Hasty Briefsbeta

All tags

performance optimization

27 stories total

Bilingual

Golang Maps: how Swiss Tables replaced the old bucket design
2 months ago
- Go 1.24 replaces the old bucket-plus-overflow map implementation with a Swiss Table-inspired design, improving cache locality and reducing pointer chasing.
- Swiss Tables use compact per-slot metadata (control bytes) and group-oriented layout to filter candidates via fingerprint before full key comparison.
- The new design supports incremental growth with segmented tables and directory routing, avoiding global stop-the-world rehashes.
- Performance gains are significant in microbenchmarks (30-35% faster for large-map access) but more modest in full applications (around 1.5% geometric mean speedup).
- Memory efficiency improves due to higher load factors and elimination of overflow chains, with reductions of 0-25% in many workloads.
- Trade-offs include potential regressions in cold-cache scenarios, sparse large maps, and delete-heavy patterns with tombstone accumulation.
- Developers should profile their own workloads, as gains depend on map dominance, access patterns, and Go version (ongoing optimizations for 1.27).
Beej's Bit Bucket
2 months ago
- Object pools reuse objects to reduce garbage collection overhead and performance hitches.
- Garbage collectors handle memory automatically but can cause delays when many temporary objects are created.
- An object pool maintains a list of reusable objects; objects are borrowed and returned instead of allocated and freed.
- If no idle object is available, the pool allocates a new one; otherwise, it reuses an existing object from the free list.
- The number of allocated objects stabilizes as the pool adjusts to demand, avoiding unnecessary allocations.
- The free list can be cleared (e.g., with a 'Collect' button) to shrink the pool when it becomes too large.
- Objects should be zeroed or sanitized when returned to the pool to prevent data leaks.
- In JavaScript, a stack (Array with push and pop) can implement the object pool, passing a constructor to define the object type.
Optimizing optimizing: some insights that led to a 400% speedup of PowerDNS - Bert Hubert's writings
2 months ago
- Performance optimization requires a mindset shift and systematic approach, not just writing better code.
- Avoid broad, unscientific advice like avoiding STL or exceptions; focus on project-specific profiling.
- After initial easy wins, further improvements require deep analysis via call-graph profiling, not just flat profiles.
- Modern compilers, CPUs, and OSes often outsmart manual optimizations, so avoid premature optimization.
- The most impactful optimization is reducing work: 'do less' by understanding what your code truly needs.
- Small incremental speedups (e.g., 15%) compound significantly after other optimizations, leading to exponential returns.
- Super-heroic micro-optimizations are rare and project-specific; leverage existing fast libraries instead.
Modern C++ for C Programmers: part 1 - Bert Hubert's writings
2 months ago
- C and C++ are closely related; C code often compiles as valid C++ and C++ includes the entire C library.
- C++ follows the zero-overhead principle: you don't pay for features you don't use, and hand-coded equivalents are not better.
- C++ std::sort outperforms C qsort by 40% due to template inlining and lambda expressions; parallel sort further speeds up sorting.
- C++ std::string offers easy concatenation, searching, and interoperates with C via c_str().
- Modern std::string uses Small String Optimization (SSO) to store short strings without heap allocation, improving cache performance.
- The article encourages C programmers to adopt these C++ features incrementally without overwhelming complexity.
Ghostty Devlog 006
2 months ago
- Focus on IO throughput speed improvements in Ghostty terminal emulator.
- SIMD optimization for plain text processing: 7.3x faster ASCII, 16.6x faster UTF-8 decode.
- Precomputed lookup tables for codepoint width: 2.8x faster ASCII, 5x faster random UTF-8.
- Optimized grapheme break detection using small lookup table, nearly 8x faster.
- CSI sequence fast-path parser improved throughput by 1.4-2x in control-heavy workloads.
- Real-world results: cat'ing large files 2-5x faster, outperforming other terminals even with grapheme clustering.
- Community contributions and future plans to further improve performance.
Lessons Learned Building High-Performance Rust Profiler
2 months ago
- hotpath-rs is an all-in-one Rust profiler/debugging toolkit that combines wall-clock timing, memory allocation, and CPU sampling into unified reports.
- Different performance signals (timing, allocations, CPU) can reveal different bottlenecks, e.g., async sleep dominating wall time but using little CPU.
- The core instrumentation uses RAII guards; performance improved from ~500ns to ~80ns per measurement by batching channel messages to avoid Thread::unpark overhead.
- Memory allocation tracking uses a thread-local stack, but for async functions that may migrate threads, an InstrumentedFuture wrapper is used to correctly attribute allocations per poll call.
- Cache-line contention was reduced by adding #[repr(align(64))] to thread allocation stats, improving multi-threaded performance by up to 86%.
- CPU profiling integrates samply via a separate process; raw CPU traces are resolved by mapping symbol ranges from the binary's debug info.
- Exclusive vs inclusive CPU attribution uses linked-list traversal of stack frames; CPU attribution uses threadCPUDelta rather than sample count to distinguish active CPU time from idle.
- Future work includes DWARF-based symbol resolution, more CPU instrumentation modes, and a hosted version for sharing and comparing profiling reports.
Escape Analysis in Go: Stack vs. Heap Allocations Explained
2 months ago
- Escape analysis in Go is a compiler optimization that determines whether a value should be allocated on the stack or the heap.
- Values escape to the heap when they are still needed after the function returns, such as when returning pointers, using closures or goroutines, passing values through interfaces, or storing in long-lived data structures.
- The Go compiler provides escape analysis insights via the 'go build -gcflags="-m"' command, but the output can be noisy and hard to interpret.
- GoLand's escape analysis tool integrates compiler output into the editor, making it easier to understand allocation decisions with gutter markers and organized logs.
- Escape analysis is most valuable for optimizing hot paths in performance-sensitive code, but not all heap allocations need optimization.
- Best practices include measuring performance first, focusing on hot paths, understanding escape reasons, and verifying changes with benchmarking.
How we made our LeRobot video reader up to 15× faster
2 months ago
- LeRobot is the dominant open format for robot learning data, but decoding frames is expensive and memory-intensive.
- The original Daft LeRobot reader opened each MP4 shard per frame, causing high latency (3 seconds per frame) and linear cost growth.
- The fix uses a batch UDF that processes 16 consecutive rows at a time, grouping rows by shard, sorting and clustering targets by timestamp, and seeking once per cluster.
- The batched reader is 4–13× faster on diverse datasets and up to 15× faster on a 1080p dataset (632 frames in under 2 minutes vs. 29 minutes).
- A hand-tracking pipeline using the optimized reader improved end-to-end time from 44.8s to 9.8s with identical detections.
SPIR-V on ROCm: A Portable IR for AMD GPUs
2 months ago
- ROCm adopts SPIR-V as a portable intermediate representation for AMD GPUs, enabling compile-once, specialize-on-device model to replace ahead-of-time per-target compilation.
- SPIR-V addresses scalability issues: eliminates long build times, large fat binaries, and lack of forward compatibility associated with AOT-per-target approach, allowing single binary to run on future GPU architectures.
- Key feature is late-resolved feature predicates (ZCFS), which enable per-arch dispatch without runtime overhead via builtins like __builtin_amdgcn_processor_is and __builtin_amdgcn_is_invocable.
- Benchmark shows SPIR-V provides flat build time and binary size regardless of target count, while fat binaries scale linearly; first-launch JIT latency (~70-100 ms) is mitigated by caching.
- Current status includes shipping amdgcnspirv target and JIT caching; in-flight developments include in-tree LLVM SPIR-V backend and JIT-at-package-install; library coverage remains a limitation.
Refs: Mutable Arrays in jax
2 months ago
- JAX Arrays are immutable, aiding optimized compilation but can limit expressiveness and performance for tasks like memory control.
- Refs are mutable arrays compatible with JAX transformations like jit and grad, allowing in-place reads and writes via indexing.
- Refs have constraints: they cannot be returned from jit functions, passed multiple times, or used in vmap with closed-over refs.
- Automatic differentiation works with pure functions using Refs internally, and ref arguments can be used for gradient accumulation via vjp.with_refs.
- Refs support in-place updates, enabling efficient gradient accumulation over microbatches without materializing separate arrays.
- foreach decorator uses Refs to write flexible loops with non-sequential access, avoiding scan's fixed pattern for better performance.
Zero-dependency streaming tar parser and writer for JavaScript
2 months ago
- The NPM ecosystem suffers from legacy dependencies, which slow down applications, increase install times, and expand the attack surface for supply chain attacks.
- e18e.dev is a movement to modernize the ecosystem by promoting cleaner, more efficient libraries like modern-tar, a zero-dependency, standards-compliant TAR parser.
- The architecture of modern-tar decouples parsing logic from I/O, using a pure, synchronous state machine that works across runtimes, including browsers and Node.js.
- A zero-copy ring buffer minimizes memory allocations and garbage collector pressure by storing references to incoming data rather than copying it.
- To handle concurrency and security, modern-tar uses a path-caching mechanism with promise chains, enabling parallel processing while maintaining strict sequential order and preventing vulnerabilities like TOCTOU.
- Benchmarks show modern-tar is 1.4x faster than node-tar and 2.6x faster than tar-fs for archives with many small files, while being stable, actively maintained, and widely adopted in the industry.
We rewrote our custom visualisation renderers from SVG to be in Canvas
2 months ago
- The new profiler dashboard replaces SVG with Canvas for rendering flame graphs, eliminating the need for DOM nodes per frame and reducing browser workload.
- Canvas uses an immediate-mode API, where drawing commands create pixels directly, avoiding style, layout, and garbage collection overhead associated with retained-mode SVG.
- Data is handled via columnar Arrow format over Arrow Flight, enabling zero-copy reads and efficient rendering by accessing arrays for frame attributes like x-positions, widths, and depths.
- Rendering optimizations include viewport culling to draw only visible frames and precomputed positions, making zooming, scrolling, and resetting interactions feel instant.
- Performance improvements show the Canvas renderer is about 3.6× faster in isolated tests, with live interactions like zooming and scrolling becoming 4-5 times quicker, though accessibility and text selection remain trade-offs.
Just Use Postgres for Job Queues
2 months ago
- Scaling Postgres-backed job queues is challenging but possible with optimizations.
- SKIP LOCKED helps solve contention by allowing workers to lock and skip already locked rows.
- Adjusting transaction isolation levels from REPEATABLE READ to READ COMMITTED reduces serialization failures.
- Optimizing indexes, such as using selective and partial indexes, cuts CPU usage and improves performance.
Count the Bytes, Not the FLOPs
2 months ago
- The article describes how tsbootstrap's batched design, which materialized a large intermediate tensor (160 MB for 10,000 observations and 2,000 replicates), caused memory-bound performance issues due to low arithmetic intensity, leading to slower performance than the incumbent library.
- A fused, streaming path was implemented using kernel fusion to avoid materializing the full replicate tensor, reducing memory traffic by keeping the working set in cache and only storing the final statistics (e.g., 16 kB instead of 160 MB).
- A second performance wall was addressed by eliminating O(B) seed object allocations through a counter-based random number generator (Philox-4x32-10), reducing setup time from 13.1 ms to 0.55 ms for small workloads and ensuring bitwise reproducibility.
- The redesign achieved speedups of 3.1x to 20x on sustained workloads compared to the incumbent, with the fused path winning all 16 benchmark cells, though the materializing path still remains slower.
- Key principles: never materialize arrays only consumed by reductions, fuse operations to minimize memory traffic, and derive state rather than carrying it, leading to significant performance gains through subtraction rather than addition of complexity.
Lobste.rs is now running on SQLite
2 months ago
- SQLite deployment completed successfully with improved performance and reduced costs.
- Migration process involved multiple PR attempts and performance fixes.
- Lessons learned include SQLite's concurrency model and backup strategies.
- Community collaboration and healthy expectations contributed to the project's success.
- SQLite chosen over PostgreSQL for simplicity and meeting project needs.
Lobste.rs is now running on SQLite
2 months ago
- SQLite migration was successfully deployed on Saturday, with positive results including reduced CPU and memory usage, faster site performance, and lower VPS costs.
- The migration project was initiated due to concerns about MariaDB's future after K1's acquisition, with initial plans to move to PostgreSQL but ultimately choosing SQLite for pragmatism and simplicity.
- Multiple pull request attempts were made, with a failed first deploy due to performance issues from full table scans and n+1 queries, which were fixed in subsequent attempts.
- A successful second deploy occurred on July 11th, after thorough testing and fixes, with the site handling Monday traffic spikes well and minor issues promptly addressed.
- Key lessons included SQLite's concurrency model (multiple readers, single writer via WAL mode), the importance of performance tuning, and the benefits of using simpler solutions like SQLite over more complex databases like PostgreSQL.
- Backup strategy involves a nightly job using restic, and deployment uses Hatchbox with Capistrano for zero-downtime updates, leveraging SQLite's local file system and WAL mode for concurrent access.
- Community contributions were acknowledged, and the migration is seen as a step toward ensuring the site's long-term sustainability, with SQLite proving suitable for the site's needs despite initial skepticism about concurrency.
Migrating a production AI agent to GPT-5.6: 2.2x faster, 27% cheaper
2 months ago
- Ploy's agent now uses GPT-5.6 Sol as the default model, replacing Claude Opus after it performed better in quality, speed (2.2x faster), and cost (27% cheaper).
- Migrating to GPT-5.6 required fixing evaluation harness biases, tool call schemas (to handle optional parameters), prompt caching (workspace-scoped keys), and reasoning replay (self-contained blobs).
- GPT-5.6 excels at clean, modern designs but can be generic without guidance; tool call verbosity initially caused empty file reads, resolved via schema transforms.
- Prompt caching differences between providers significantly impacted costs; OpenAI's model requires explicit keys per workspace for effective cache hits.
- The model shows promise with lean code output and improved visual scores, but configuration adjustments are crucial for fair performance and cost comparisons.
We scaled PgBouncer to 4x throughput
2 months ago
- PgBouncer is single-threaded, using only one CPU core even on multi-core systems, which can bottleneck throughput.
- In ClickHouse Managed Postgres, multiple PgBouncer processes are run, scaled to available cores, using so_reuseport for kernel load balancing.
- Query cancellation is addressed via peering: processes forward cancel requests to the correct session owner, ensuring functionality across the fleet.
- Transaction mode pooling returns server connections after commits, with connection budgets split across processes to avoid oversubscribing Postgres.
- Performance tests show a fleet of 16 processes achieves ~336k TPS (4x higher) and utilizes ~8 cores, while a single process peaks at ~87k TPS and uses only one core.
- A single PgBouncer suffices at low concurrency, but under high load, multi-process fleets prevent the pooler from becoming a bottleneck.
Pg_re2: Postgres extension for fast, RE2-powered regular expressions in Postgres
2 months ago
- ClickHouse introduces pg_re2, a Postgres extension for fast RE2-powered regular expressions, emphasizing speed and compatibility.
- RE2 uses finite automata for faster, more predictable performance compared to Postgres's backtracking POSIX regex, with benchmarks showing 1.8x to 8.6x speedups.
- pg_re2 adds index support (@~ operator) with btree and GIN indexes, providing 1.1x to 1.8x acceleration and better trigram handling for patterns.
- Compatibility with ClickHouse regex functions is key, as pg_re2 addresses syntax differences (e.g., \b meaning word boundary vs. backspace) to ensure consistent results when using pg_clickhouse.
- pg_re2 functions push down to ClickHouse, eliminating discrepancies and improving performance for analytical queries, and is recommended for a unified high-performance data platform.
Girls Just Wanna Have Fast MPMC Queues with Bounded Waiting
3 months ago
- The post presents a wait-free (bounded waiting) MPMC queue design based on a ticket lock system.
- The queue uses atomic counters for producers and consumers, with separate data and state buffers.
- Operations avoid CAS loops, minimizing cache contention and head-of-line blocking.
- Bounded waiting is ensured unless threads are suspended or fail, preventing starvation.
- Drivable enqueue/dequeue operations allow non-blocking polling for async/event loop integration.
- A rare race condition risk exists at high reservation number overflow boundaries (e.g., 2^63+1).
- Benchmarks show good scalability with multiple producers/consumers, though performance varies by core count.

firstprev0next

performance optimization

Golang Maps: how Swiss Tables replaced the old bucket design

Beej's Bit Bucket

Optimizing optimizing: some insights that led to a 400% speedup of PowerDNS - Bert Hubert's writings

Modern C++ for C Programmers: part 1 - Bert Hubert's writings

Ghostty Devlog 006

Lessons Learned Building High-Performance Rust Profiler

Escape Analysis in Go: Stack vs. Heap Allocations Explained

How we made our LeRobot video reader up to 15× faster

SPIR-V on ROCm: A Portable IR for AMD GPUs

Refs: Mutable Arrays in jax

Zero-dependency streaming tar parser and writer for JavaScript

We rewrote our custom visualisation renderers from SVG to be in Canvas

Just Use Postgres for Job Queues

Count the Bytes, Not the FLOPs

Lobste.rs is now running on SQLite

Lobste.rs is now running on SQLite

Migrating a production AI agent to GPT-5.6: 2.2x faster, 27% cheaper

We scaled PgBouncer to 4x throughput

Pg_re2: Postgres extension for fast, RE2-powered regular expressions in Postgres

Girls Just Wanna Have Fast MPMC Queues with Bounded Waiting