Hasty Briefsbeta

All tags

#devops

131 stories total

Bilingual

In retrospect, DevOps was a bad idea
a year ago
- DevOps initially improved deployment by involving developers in production processes.
- Naming DevOps led to the creation of DevOps teams, which isolated operations again.
- DevOps teams now restrict rather than empower developers, counter to the original intent.
- Standardization and compliance in DevOps often hinder rather than help development teams.
- Security should focus on transparency and reviewability, not restrictive access.
- Formalizing DevOps roles removed production-minded engineers from product teams.
- Platform Engineering is repeating the same mistakes as DevOps.
DIY automation using only Linux
a year ago
- Automating website deployment using simple Linux tools.
- Prerequisites include a Gitea repository and an Ubuntu server.
- Step-by-step guide covering static and dynamic website deployment.
- Scripts for automatic deployment, logging, and issue creation on failure.
- Setting up Nginx for serving static content or proxying to a Node.js server.
- Creating a system service to handle deployment hooks.
- Adding webhooks in Gitea to trigger deployments on code pushes.
- Optional step to automatically create issues in Gitea on deployment failures.
- Conclusion emphasizing simplicity and avoiding vendor lock-in.
Building a Firecracker-Powered Course Platform to Learn Docker and Kubernetes
a year ago
- The author built a learning-by-doing platform for DevOps, SRE, and Platform engineers to enhance technical education with interactive playgrounds.
- The platform integrates theoretical content with practical exercises, aiming to bridge the gap between learning and real-world application.
- Key design principles include simplicity, reliability, security, cost-effectiveness, scalability, and productivity.
- The architecture features a central Foreman component for management and a fleet of worker servers running Firecracker microVMs for playgrounds.
- Security measures include isolation of microVMs, rate-limiting, and restricted egress traffic to minimize risks.
- The platform uses a mix of JavaScript (Nuxt 3) for the frontend and Go for backend services, with Docker extensively used for deployment and runtime.
- Hosting solutions include fly.io for the frontend and Hetzner Auction for cost-effective bare-metal worker servers.
- Future plans include IDE integration, multi-node playgrounds, Kubernetes visualizer, and potential open-sourcing of key components.
- The platform is designed for educators, students, and companies, with possibilities for training, automated interviews, and university courses.
Coolify: Open-source and self-hostable Heroku / Netlify / Vercel alternative
a year ago
- Coolify is an open-source, self-hostable alternative to Heroku, Netlify, and Vercel.
- Supports any programming language and framework for deploying static websites, APIs, backends, databases, and more.
- Deployable on any server including VPS, Raspberry Pi, EC2, DigitalOcean, Linode, and Hetzner via SSH.
- Flexible deployment options: single server, multiple servers, or Docker Swarm clusters (Kubernetes support coming soon).
- Compatible with any Docker service, with many one-click services available.
- Git integration with platforms like GitHub, GitLab, Bitbucket, and Gitea for push-to-deploy functionality.
- Automatic Let's Encrypt SSL certificate setup and renewal for custom domains.
- No vendor lock-in; all settings and data are stored on your own servers.
- Automatic database backups to S3-compatible solutions for easy restoration.
- Webhooks for CI/CD pipeline integration with GitHub Actions, GitLab CI, Bitbucket Pipelines, etc.
- Powerful API for automating deployments, managing resources, and integrating with existing tools.
- Real-time terminal in the browser for server management without leaving Coolify.
- Collaborative features: share projects with team members and control permissions.
- Pull request deployments for quick review and team collaboration.
- Server automations to handle tasks automatically, letting you focus on coding.
- Monitoring for deployments, servers, disk usage, and notifications for issues.
- Notifications via Discord, Telegram, email, etc., for server and deployment events.
Show HN: SwiftHive – A Swift Package Registry for Faster Builds
a year ago
- SwiftHive is a private registry for Swift packages that ensures build reproducibility.
- It speeds up dependency resolution by 10x and reduces bandwidth usage by up to 90%.
- Versions in SwiftHive are immutable once published, guaranteeing reproducible builds.
- Optimized archives reduce package sizes by up to 90% compared to Git repositories.
- SwiftHive provides role-based access controls for centralized package management.
- It offers both fully managed and self-hosted deployment options.
- SwiftHive improves resolution speed by downloading optimized archives instead of cloning Git repositories.
- The registry supports standard SPM syntax for referencing private packages.
Show HN: Koreo – The platform engineering toolkit for Kubernetes
a year ago
- Koreo is a Kubernetes configuration management and resource orchestration tool.
- It offers programmable workflows for automating complex Kubernetes operations.
- Structured configuration management treats configurations as structured data, not just templates.
- Dynamic resource materialization allows combining configurations from multiple sources.
- Configuration as functions promotes modularity and reusability.
- Declarative operator model ensures consistency across infrastructure.
- First-class testing and tooling includes built-in testing frameworks and IDE integration.
- Koreo serves as a meta-controller programming language for Kubernetes.
- Workflows and functions provide composable primitives for building platforms.
- Dynamic resource materialization enables configuration layers and pluggable logic.
- Koreo supports various use cases like Internal Developer Platforms, Multi-Cloud IaC, and Policy as Code.
- The team behind Koreo has extensive platform engineering expertise.
That's a Lot of YAML
a year ago
- YAML is humorously criticized for its quirks and potential for errors in DevOps environments.
- Common issues include parsing 'NO' as a boolean instead of Norway, and inconsistent handling of numbers (e.g., '07' vs '08').
- YAML's lack of official documentation and the complexity of its specification are highlighted as drawbacks.
- Security vulnerabilities related to YAML parsing are mentioned, including executable YAML risks.
- The post critiques YAML's handling of multi-line strings, octal numbers, and time formats.
- Alternatives to YAML for configuration and DevOps are suggested, such as Dhall, CUE, and Jsonnet.
- The website itself is a subject of criticism for its unconventional and user-unfriendly design.
EKS Auto Mode: Simplify Kubernetes Operations
a year ago
- EKS Auto Mode is a new deployment option in Amazon EKS that automates infrastructure provisioning and scaling.
- It eliminates the need to manage EC2 nodes by using serverless compute, optimized for Kubernetes workloads.
- EKS Auto Mode incorporates Kubernetes capabilities directly into AWS-managed infrastructure, including compute autoscaling, networking, load balancing, and GPU support.
- It is designed for users who want to run Kubernetes workloads without managing the underlying compute infrastructure.
- Key features include automated cluster management, dynamic scaling, enhanced security, and cost optimization.
- EKS Auto Mode is compared to AWS Fargate and EKS Managed Node Groups, highlighting differences in flexibility and management.
- A tutorial section demonstrates how to create an EKS Auto Mode cluster, authenticate, deploy a sample application, and manage scaling.
- Best practices include setting pod resource requests, isolating workloads, and tagging resources for cost tracking.
- Limitations include immutable nodes, pre-defined AMIs, and opinionated defaults that may not suit all use cases.
- Spacelift is introduced as a platform for managing infrastructure-as-code tools, enhancing collaboration and governance.
Show HN: Attune - Build and publish APT repositories in seconds
a year ago
- Attune is a tool for securely publishing and hosting Linux packages.
- It offers flexible deployment options: self-hosted or managed cloud infrastructure.
- Designed for security: CLI performs repository index signing locally, keeping signing keys private.
- Optimized for speed: incremental repository index rebuilds make package management fast.
- Currently supports APT (Debian and Ubuntu) repositories, with more to come.
- Quick setup guide provided for setting up an APT repository in about 5 minutes.
- Setup involves cloning the repository, configuring environment variables, and starting services with Docker.
- CLI installation, repository creation, and package addition steps are outlined.
- GPG key generation and usage for signing and deploying the repository are detailed.
- Apache 2 licensed, with a user guide available for more detailed instructions.
Ansible: Pure (only in its) pragmatism
a year ago
- Ansible is a tool for remote and local system administration tasks, focusing on idempotence.
- It allows declarative state management for files, permissions, users, groups, services, and more.
- Ansible uses a YAML-based DSL with Jinja2 templating and has a large community and plugin ecosystem.
- Key features include SSH/WinRM support, idempotent operations, and integrations with various APIs.
- Design choices include reliance on file hierarchy, global variables, and lack of namespacing.
- Ansible is not purely declarative; task order matters and imperative commands are possible.
- Compared to shell scripts, Ansible offers a consistent interface, idempotence, and reliability.
- It is agentless, simplifying operations but not enforcing continuous configuration like Puppet or Salt.
- Ansible is slower than agent-based solutions due to SSH transport but has a larger community.
- Best suited for pet servers and ad-hoc tasks; less ideal for full infrastructure-as-code.
- Potential improvements include better variable scoping, role simplification, and code reuse.
- Ansible remains irreplaceable for its niche, though modern tools like Terraform + Packer are better for full IaC.
Nerdlog: Fast, multi-host TUI log viewer with timeline histogram
a year ago
- Nerdlog is a fast, multi-host TUI log viewer with a timeline histogram, inspired by Graylog/Kibana but without the bloat.
- It was developed as a lightweight alternative to centralized logging systems like Graylog and Splunk, which were either slow or required significant maintenance.
- Nerdlog connects to remote nodes via SSH, analyzes log files directly on the hosts, and only downloads necessary data (logs and histogram info), making it efficient.
- Key features include simultaneous log querying from multiple hosts, time range filtering, and a histogram for visualizing log distribution.
- The tool uses standard Unix utilities (bash, awk, tail, head) and supports logstreams, which can be configured via SSH or a YAML config file.
- Nerdlog is designed for small teams or personal projects where SSH access to production hosts is feasible, though it has limitations like CPU/IO usage on hosts and dependency on log rotation policies.
- It is not a replacement for tools like lnav, which focus on local log files, but excels at efficiently handling remote logs from multiple nodes.
Atuin Desktop: Runbooks That Run
a year ago
- Atuin Desktop is an executable runbook editor designed for terminal workflows.
- It combines script blocks, embedded terminals, database clients, and Prometheus charts in one place.
- Aims to make workflows repeatable, shareable, and reliable, preventing knowledge rot.
- Features include reducing context switching, dynamic runbooks with templating, and autocomplete from shell history.
- Local-first and CRDT-powered, with sync and share capabilities via Atuin Hub.
- Used for releasing software, migrating infrastructure, and managing database queries.
- Future plans include team accounts and auto-generating runbooks from shell history.
- Early access is available for those interested in improving their workflow efficiency.
Writing "/etc/hosts" breaks the Substack editor
a year ago
- Author encountered a 'Network Error' in Substack editor when typing system file paths like '/etc/h*sts'.
- Investigation revealed the error was triggered by Web Application Firewall (WAF) security filters.
- WAFs block paths resembling common system files to prevent path traversal and command injection attacks.
- Examples of blocked paths include '/etc/pass*d' and '/etc/ssh/sshd_conf*g'.
- Historical posts show variations in filtering behavior over time.
- The issue highlights the tension between security measures and usability for technical content.
- Suggested improvements include contextual filtering and clearer error messages.
- Workarounds involve using alternative spellings or quoting paths.
Show HN: Logchef – Schema-agnostic log viewer for ClickHouse
a year ago
- Logchef is a modern, high-performance log analytics platform.
- It is lightweight and operates as a single binary, using ClickHouse for storage and querying.
- Features schema-agnostic log exploration, flexible query options, and high performance.
- Includes team-based access control with fine-grained permissions.
- Simplifies deployment with single binary installation.
- Can be started using Docker Compose with provided commands.
- Accessible via http://localhost:8125 after setup.
- Documentation and more details available at logchef.app.
- Distributed under AGPLv3 License.
Show HN: Kubetail – Real-time log search for Kubernetes
a year ago
- Kubetail is a real-time logging dashboard for Kubernetes, usable in browser or terminal.
- It merges logs from all containers in a workload into a single chronological timeline.
- Kubetail uses the Kubernetes API to fetch logs directly, ensuring data privacy.
- Features include filtering by workload, time range, node properties, and grep.
- Installation options include Homebrew, direct download, Helm chart, and Glasskube.
- Supports multiple clusters (Desktop-only) and tracks container lifecycle events.
- Development contributions are welcome in UI/UX design, React frontend, and issue reporting.
How we saved time (and money) on continuous integration
a year ago
- The Datawrapper app team reduced their continuous integration (CI) workflows by 80% and costs by 45%.
- They switched from NPM to PNPM, which made dependency installation 74% faster by avoiding redundant installations.
- Implemented caching for dependencies and builds, reducing setup time to 35 seconds (87.5% improvement).
- Split workflows to run steps in parallel, achieving a 67% improvement in real-time execution.
- Switched from GitHub runners to Blacksmith runners, resulting in 22.14% faster job execution and 45% cost savings.
- The team now spends less time waiting for CI tasks and more time building, with linting being the next bottleneck to address.
Cloud-Based Cron Jobs
a year ago
- Schedo.dev offers a developer-first platform for running scheduled tasks in the cloud with zero infrastructure setup.
- Features include automatic scaling, concurrency handling, retries, and reliability guarantees.
- Eliminates the need for manual DevOps configuration, unlike traditional cron jobs which are infrastructure-dependent and lack built-in reliability.
- Provides built-in distributed locking to prevent race conditions and ensure single execution of jobs.
- Supports local development with the same API as production, allowing testing without deployment.
- Works across any cloud provider with no infrastructure lock-in, enabling easy migration between environments.
- Offers real-time execution logs, performance metrics, status monitoring, and failure alerts for full observability.
- Uses standard cron syntax or interval-based scheduling for job definition.
- Ensures jobs run exactly once across the entire infrastructure, scaling with confidence.
- Manages different environments (dev, staging, prod) with separate configurations and monitoring.
Cloudflare's approach to global service health metrics and software releases
a year ago
- Cloudflare's Health Mediated Deployments (HMD) automates software updates using data-driven metrics to prevent widespread issues.
- HMD uses Prometheus and Thanos to monitor service health, reverting problematic updates before they affect users.
- Service Level Objectives (SLOs) and Indicators (SLIs) help HMD detect and respond to performance degradation.
- Backtesting with historical incident data ensures HMD can quickly react to future issues.
- Cloudflare stores 4.5 billion time series in R2, totaling 8 petabytes of data for a year of retention.
- Optimizations like recording rules and distributed query processing reduce batch runtimes from 30 hours to 2 hours.
- Congestion control mechanisms prioritize critical queries and smooth out batch processing spikes.
- Experimental Parquet-based storage shows promise for optimizing time series data handling.
Show HN: CLI that spots fake GitHub stars, risky dependencies and licence traps
a year ago
- StarGuard is a CLI tool designed to detect risks in open-source projects, including fake stars, dependency hijacks, and license issues.
- It automates open-source due diligence for CTOs, security teams, and VCs, providing a fast and repeatable trust score.
- Key checks include star burst detection, dependency analysis, license scanning, maintainer activity, and code pattern scans.
- Output formats include JSON, Markdown, plaintext reports, and visual star-history plots.
- StarGuard uses GitHub API for data collection and includes modules like BurstDetector and User Profiler for analysis.
- It requires Python ≥ 3.9 and a GitHub Personal Access Token for high-rate API limits.
- Usage examples include full scans with star plots or burst-only fast scans.
- Target users include CTOs, security teams, VCs, and open-source maintainers.
- The tool is open-source, licensed under Apache-2.0, and focuses on static analysis without executing code.
We Fixed 2k+ Security Issues (2023)
a year ago
- Dgraph Labs Inc. implemented Continuous Security Audits to detect and remediate security issues in their OpenSource offerings.
- Integrated toolsets with GitHub Actions for improved visibility and faster resolution of security issues.
- Addressed over 2k+ security issues in ~3 months, enhancing SOC2 compliance.
- Security audits focus on three layers: Code, Binary Artifacts, and Docker Images.
- Code layer audits include dependency package checks (CVEs) and static analysis (Linters).
- Binary Artifacts layer ensures secure environment and SHA validation for integrity.
- Docker Images layer focuses on updated Linux packages to prevent vulnerabilities.
- CVEs (Common Vulnerabilities and Exposures) are identifiers for tracking software vulnerabilities.
- Linters are automated tools for detecting code issues, improving quality and security.
- Continuous Security Audits involve Code & Build Phase (Aqua Trivy Scans) and Post Release Phase (Snyk Scans).
- GitHub Security tab used for visualizing, triaging, and resolving security issues.
- DependaBot enabled for auto-remediation of security issues.
- Tracked fixes via GitHub, showing a decrease in reported issues over time.
- Achieved significant results: resolved 2k+ security issues and 1k+ CVEs across projects.
- Future focus includes exploring AI-driven security tools and auto-remediation solutions.

firstprev0next

About|Login

#devops

In retrospect, DevOps was a bad idea

DIY automation using only Linux

Building a Firecracker-Powered Course Platform to Learn Docker and Kubernetes

Coolify: Open-source and self-hostable Heroku / Netlify / Vercel alternative

Show HN: SwiftHive – A Swift Package Registry for Faster Builds

Show HN: Koreo – The platform engineering toolkit for Kubernetes

That's a Lot of YAML

EKS Auto Mode: Simplify Kubernetes Operations

Show HN: Attune - Build and publish APT repositories in seconds

Ansible: Pure (only in its) pragmatism

Nerdlog: Fast, multi-host TUI log viewer with timeline histogram

Atuin Desktop: Runbooks That Run

Writing "/etc/hosts" breaks the Substack editor

Show HN: Logchef – Schema-agnostic log viewer for ClickHouse

Show HN: Kubetail – Real-time log search for Kubernetes

How we saved time (and money) on continuous integration

Cloud-Based Cron Jobs

Cloudflare's approach to global service health metrics and software releases

Show HN: CLI that spots fake GitHub stars, risky dependencies and licence traps

We Fixed 2k+ Security Issues (2023)