Hasty Briefsbeta

All tags

#performance optimization

122 stories total

Bilingual

Cache-Friendly B+Tree Nodes with Dynamic Fanout
7 months ago
- B+Tree nodes require contiguous memory blocks for high performance, avoiding std::vector due to indirection.
- The 'struct hack' or flexible array member technique allows dynamic sizing of node entries in C/C++.
- C++11 supports flexible array members as 'arrays of unknown bound' when declared last in a struct.
- Placement new is used to construct B+Tree nodes in preallocated memory buffers for contiguous layout.
- Manual memory management is required, including careful destruction and deallocation to prevent leaks.
- Derived classes cannot add new members without risking data corruption due to memory layout constraints.
- Using raw arrays means reimplementing vector-like functionality, increasing complexity and maintenance.
- The implementation implicitly requires trivially copyable types, limiting generic use with complex types like std::string.
Why We Need SIMD
7 months ago
- SIMD (Single Instruction, Multiple Data) delivers significant performance speedups at a modest area cost.
- SIMD instructions reuse existing CPU infrastructure like caches and decoding hardware, making them cost-effective.
- First SIMD on x86 was MMX, which performed 8 byte-sized operations per instruction, followed by wider SSE, AVX, and AVX512.
- SIMD requires software updates to utilize new instructions, unlike transparent CPU improvements like superscalarity.
- Video and cryptography benefit most from SIMD, but 3D rendering shifted to dedicated hardware, limiting SIMD's impact.
- AVX512, Intel's latest SIMD, includes advanced features like per-lane predication and mask registers for better performance.
- SIMD adoption is hindered by the need for developers to manually optimize code, unlike automatic hardware improvements.
We didn't rewrite our feed handler in Rust
7 months ago
- Decision to rewrite market data feed handler in C++ despite Rust's popularity in fintech.
- Existing implementation had technical debt, complex concurrency model, and lock contention issues.
- New implementation requirements: minimal shared state, basic parallelism, and predictable sub-100μs latencies.
- Rust's advantages: built-in tooling, compiler error clarity, safety for finance applications.
- Challenges with Rust: buffer reuse optimization, self-referential structs, compile-time generics limitations.
- C++ benefits: code reuse, control over resources, flexibility with templates, team expertise.
- Future considerations: C++26 compile-time reflection, Rust's Polonius borrow checker improvements.
- Conclusion: Both Rust and C++ have niches in fintech; pragmatic language choice based on requirements.
Bulk Operations in Boost.Bloom
7 months ago
- Boost.Bloom introduces bulk operations in Boost 1.90 to speed up insertion and lookup by separating position calculation from memory access.
- Bulk-mode insertion prefetches addresses to minimize CPU stalling, improving performance for larger bit arrays.
- Lookup operations for k > 1 are challenging due to branching; bulk-mode reduces iterations using bitmask techniques.
- The bulk version of may_contain reduces conditional branches and superfluous memory fetches, achieving speedups up to 3x.
- Performance gains vary based on filter configuration and size, with some cases showing less performance than non-bulk operations.
- The key technique involves iteration reduction with std::countr_zero, applicable beyond Boost.Bloom for efficient pipelining.
Implementing ZADD If Key Exists
7 months ago
- Performance optimizations required implementing a Redis ZADD operation only if the sorted set exists, without creating a new set.
- Initial non-atomic solution involved checking key existence before ZADD, but needed an atomic approach.
- AI suggested using a pipeline, which was incorrect as it always performed ZADD regardless of key existence.
- AI then proposed a LUA script for atomicity, but LUA was rejected due to potential performance issues.
- WATCH command was introduced as an alternative, but testing revealed it didn't work as expected with key expiration.
- Final solution used TTL check to ensure key existence with a buffer, leveraging simplicity and performance.
- The approach tolerates rare cases of non-TTL keys, which can be cleaned up with SCAN.
JIT: So you want to be faster than an interpreter on modern CPUs
7 months ago
- The blog discusses the challenges of optimizing a JIT compiler for PostgreSQL, focusing on performance improvements over the interpreter.
- Modern CPUs with out-of-order execution and branch prediction complicate performance optimization efforts.
- The interpreter's main loop with opcode dispatch can be inefficient due to unpredictable branching, leading to performance losses.
- Using 'computed gotos' instead of a switch statement can improve interpreter performance by making jumps more predictable.
- Optimizing a simple query like 'SELECT a FROM table WHERE a = 42' involves reducing unnecessary null checks and inlining common operations.
- Inlining the int4eq function call significantly reduces instruction count and improves performance.
- The author explores various optimizations, including removing null checks and inlining, showing measurable performance gains.
- Despite CPU optimizations, the interpreter remains a tough competitor, but the JIT compiler shows promise with further improvements.
- The author seeks collaboration and support for the project, including code contributions, sponsoring, or job offers.
Intro to Practical SIMD for Graphics
7 months ago
- SIMD (Single Instruction, Multiple Data) instructions allow CPUs to process multiple data points with a single instruction, significantly speeding up operations like matrix multiplication.
- Different CPU architectures support various SIMD instruction sets: x86 (SSE, AVX, AVX512), ARM (NEON, SVE), and RISC-V (RVV), each with unique features and compatibility challenges.
- For game development, targeting AVX2 is recommended for high-end games, while SSE4 ensures compatibility with virtually all PCs. NEON is essential for mobile and Nintendo Switch development.
- SIMD programming can be done using intrinsics, libraries like XSIMD, or ISPC, each offering different trade-offs between performance and ease of use.
- Practical examples include vectorized array addition and 4x4 matrix multiplication, demonstrating significant performance improvements over scalar code.
- Frustum culling can be optimized using SIMD by processing multiple spheres simultaneously, leveraging AVX for 8-wide operations and FMA for further performance gains.
- FMA (Floating Multiply-Add) instructions can provide substantial performance improvements for dot product operations, common in graphics algorithms.
Rethinking Async Loops in JavaScript
6 months ago
- Using `await` in loops can cause sequential execution, leading to inefficiency for independent network calls.
- Avoid using `await` inside `map()` without handling promises, as it returns an array of unresolved promises.
- Use `Promise.all()` with `map()` to run API calls in parallel and get all results at once.
- `Promise.all()` fails fast; a single rejection causes the entire operation to fail.
- For error resilience, use `Promise.allSettled()` to process all results, including failures.
- Handle errors inside the mapping function to prevent unhandled promise rejections.
- Use `for...of` with `await` for sequential execution when order or rate limits matter.
- Use `Promise.all()` with `map()` for parallel execution of independent tasks.
- For controlled concurrency, use throttling utilities like `p-limit` to balance speed and API limits.
- Never use `await` in `forEach()` as it doesn’t wait for async operations to complete.
- Choose the right async pattern based on needs: order (`for...of`), speed (`Promise.all`), safety (`allSettled`), or balance (`p-limit`).
A Fast Bytecode VM for Arithmetic: The Virtual Machine
6 months ago
- The article details the creation of a fast bytecode compiler and virtual machine (VM) for arithmetic expressions in Haskell.
- Key components include parsing, compilation to bytecode, and execution via a stack-based VM, with a focus on performance optimization.
- Property-based testing with QuickCheck ensures correctness across arbitrary inputs, highlighting the importance of testing in compiler development.
- Performance benchmarks compare the Haskell VM against a C implementation, showing Haskell's VM is 1.5x to 2.6x slower but with significantly less code.
- Optimizations in the Haskell VM include using mutable, unboxed arrays for the stack and careful loop structuring to aid branch prediction.
- Future improvements suggested include parser optimizations, superinstructions, register-based VMs, and JIT compilation for further performance gains.
Principles and Methodologies for Serial Performance Optimization
7 months ago
- The paper outlines eight methodologies for serial performance optimization: Batching, Caching, Precomputing, Deferring, Relaxation, Contextualization, Hardware Specialization, and Layering.
- The authors fine-tuned GPT-4o into SysGPT, a model that suggests optimization strategies based on problem descriptions and observations.
- SysGPT outperforms baseline GPT-4o in selecting appropriate optimization techniques for given problems.
- The paper focuses on serial code optimization but hints at potential extensions for parallelizable problems.
- Future work could automate the generation of performance observations from source code and profiling data.
A deep dive into BPF LPM trie performance and optimization
6 months ago
- BPF LPM trie maps are critical for IP and IP+Port matching in network routing but face performance issues with large datasets.
- Performance bottlenecks include slow entry lookups (hundreds of milliseconds) and CPU lockups during map freeing (over 10 seconds).
- Tries are efficient for prefix matching, especially with IP routing using CIDR, due to their memory-efficient and search-optimized structure.
- Multibit tries offer faster traversal than binary tries by comparing multiple bits at once, trading space for time.
- Path compression and level compression are optimizations that can reduce memory usage and improve lookup times in tries.
- BPF LPM trie implementation lacks full optimization, leading to inefficiencies like high cache and TLB miss rates as entry count grows.
- Future work includes refactoring to use Level Compressed tries from net/ipv4/fib_trie.c to improve BPF LPM trie performance.
Show HN: Incremental JSON parser for streaming LLM tool calls in Ruby
6 months ago
- Streaming AI responses enhances user experience by displaying real-time generation.
- Incomplete JSON during streaming causes UI issues like loading spinners and lack of feedback.
- Traditional JSON parsing methods are inefficient (O(n²)) and cause lag as data grows.
- Stateful incremental parsing (O(n)) processes only new data, maintaining performance.
- Edge cases like context-dependent strings and incomplete escapes require careful handling.
- JsonCompleter gem enables real-time UI updates by parsing incomplete JSON efficiently.
- Performance benchmarks show O(n) parsing is 388x faster than O(n²) for 12 KB data.
- Streaming is critical for modern AI interfaces to avoid perceived system freezes.
Helion: A High-Level DSL for Performant and Portable ML Kernels
6 months ago
- Helion is a high-level Python-embedded domain-specific language (DSL) that compiles into optimized Triton code, bridging PyTorch's simplicity with low-level performance.
- It automates tensor indexing, memory management, and hardware-specific tuning, allowing developers to focus on algorithmic logic rather than implementation details.
- Helion's programming model, 'PyTorch with Tiles', minimizes boilerplate and leverages existing PyTorch knowledge, making kernel development more intuitive.
- The autotuning engine in Helion automatically constructs and explores a vast search space for optimal kernel configurations, significantly reducing manual effort.
- Performance benchmarks show Helion outperforming torch.compile and hand-written Triton kernels, with notable speedups on both NVIDIA and AMD GPUs.
- Case studies demonstrate Helion's ability to match or exceed the performance of highly optimized hand-written kernels, such as those written in CuTe DSL or TileLang.
- Helion's compiler architecture efficiently lowers Python functions into optimized Triton code, applying performance-critical configurations only at the final code generation stage.
- Helion is set to be released in Beta on Oct. 22nd, 2025, aiming to provide a productive paradigm for performant machine learning kernels.
FlashPack: Fast Model Loading for PyTorch
6 months ago
- FlashPack is a high-throughput file format and loading mechanism for PyTorch designed to speed up model checkpoint I/O.
- It makes model loading 3–6× faster than current methods like `accelerate` or `load_state_dict()` and `to()`.
- FlashPack treats model weights as a single data stream instead of individual files, improving load times.
- Key features include flattening the state_dict into a contiguous byte stream, memory-mapped reads, and overlapping disk, CPU, and GPU operations with CUDA streams.
- Benchmarks show 2–6× faster checkpoint loading compared to existing methods.
- Limitations include requiring all weights to be the same data type and not supporting device mapping or state dictionary transformations.
- FlashPack can be installed via PyPI or GitHub and integrates easily with existing workflows, including Hugging Face models.
Why Your Social.org Files Can Have Lines Without Issues?
6 months ago
- Org Social 2.3+ optimizes performance by using concurrent queue processing and HTTP Range-based partial fetching.
- Traditional sequential downloads waste bandwidth and time by downloading entire feeds when only recent posts are needed.
- Concurrent Queue Processing: Uses a worker pool to manage parallel downloads, configurable via `org-social-max-concurrent-downloads`.
- HTTP Range-Based Partial Fetching: Downloads only necessary parts of files (headers and recent posts) using HTTP Range headers.
- Compatibility: Automatically detects and adapts to server limitations (e.g., Cloudflare CDN, Codeberg.org, GitHub Raw Content).
- UI Pagination: Renders only 10 posts per page to keep the interface responsive, even with large downloads.
- Performance Benchmarks: Partial fetching maintains consistent download sizes (~8 KB) for feeds with over 100 posts.
- Tuning Recommendations: Users can adjust `org-social-max-concurrent-downloads` and `org-social-max-post-age-days` for optimal performance.
- Conclusion: The three-layer approach ensures scalability, efficiency, and non-blocking UI for Org Social users.
Opportunistic Mutation in Roc
6 months ago
- Roc is a single-paradigm functional language with a focus on simplicity and performance.
- Roc uses opportunistic mutation to optimize performance while maintaining semantic immutability.
- Semantic immutability in Roc eliminates data races and reference cycles, enhancing reliability and performance.
- Roc disallows reassignment and shadowing, making code easier to read and rearrange without regressions.
- Roc employs managed effects (Tasks) instead of side effects, ensuring all functions are pure.
- Pure functions in Roc enable compiler optimizations like loop fusion, compile-time evaluation, and hoisting.
- Roc's design choices support easier debugging, testing, and optimization, though some algorithms may be harder to express.
We Deleted Tokio from Our Payment System and Cut Cloud Costs by $127,000
6 months ago
- The author deleted Tokio from their Rust microservice to reduce costs and improve performance.
- After removing Tokio and switching to blocking I/O, memory usage dropped by 42% and P99 latency improved from 340ms to 180ms.
- The change resulted in an annual cost saving of over $127,000.
- The author reflects on the widespread belief that async programming is always better, questioning why they didn't make the change sooner.
The Green Tea Garbage Collector
6 months ago
- Go 1.25 introduces an experimental garbage collector called Green Tea, which reduces GC time by 10-40% for many workloads.
- Green Tea is production-ready and used at Google, with plans to make it default in Go 1.26.
- The traditional Go garbage collector uses a mark-sweep algorithm, which involves marking reachable objects and sweeping unreachable ones.
- Green Tea optimizes GC by working with memory pages instead of individual objects, improving cache utilization and reducing memory stalls.
- Vector hardware acceleration (e.g., AVX-512) further enhances Green Tea's performance by enabling efficient page-wide operations.
- Early benchmarks show 10-40% reduction in GC CPU costs, with additional improvements expected from vector acceleration.
- Green Tea is available in Go 1.25 via GOEXPERIMENT=greenteagc, with opt-out possible in Go 1.26 using GOEXPERIMENT=nogreenteagc.
- Feedback from users is encouraged to refine and improve Green Tea before its full release.
Vectorizing for Fun and Performance
6 months ago
- IBM Power processors feature a vector processing facility (AltiVec, VMX, VSX) enabling SIMD operations.
- POWER8 has 64 vector-scalar registers (VSRs), each 128 bits, capable of holding multiple floating-point quantities.
- Vector instructions can perform operations like add, subtract, multiply, divide, and multiply-add simultaneously.
- Compilers support auto-vectorization but often require explicit implementation for optimal performance.
- Example provided: vectorized code for finding the maximum value in an array shows significant performance gains for larger arrays.
- Performance comparison: Non-vectorized vs. vectorized code shows a drastic reduction in execution time for 32 floats.
What Caused Performance Issues in My Tiny RPG
6 months ago
- Performance issues in a tiny RPG built with web technologies were noticed when wrapped as a desktop app compared to running in Firefox.
- Feedback from the KAPLAY Discord server and a wider audience via Substack and Hacker News helped identify the issues.
- The game was packaged using NW.js and GemShell, with GemShell initially chosen for quick executable generation across platforms.
- KAPLAY was used for its ease of prototyping, despite being less performant than alternatives like Phaser or Excalibur.
- Issue #1: Setting a max FPS in KAPLAY caused the game to slow down when the machine couldn't sustain the target FPS, resolved by removing the FPS cap.
- Issue #2: Poorer performance on Mac due to Webkit's limitations compared to Chromium, with potential fixes from GemShell's developer.
- Issue #3: Better performance on Firefox than Chrome and Safari, attributed to inefficient text rendering and lack of object pooling.
- Optimizations included direct text rendering in the draw loop, batching draw calls, and implementing object pooling for projectiles.
- The game now runs smoothly on Chrome and better on Safari, though not as consistently as on Firefox.
- Lessons learned include the importance of learning more performant frameworks and engines to avoid future development roadblocks.

first prev3next

About|Login

#performance optimization

Cache-Friendly B+Tree Nodes with Dynamic Fanout

Why We Need SIMD

We didn't rewrite our feed handler in Rust

Bulk Operations in Boost.Bloom

Implementing ZADD If Key Exists

JIT: So you want to be faster than an interpreter on modern CPUs

Intro to Practical SIMD for Graphics

Rethinking Async Loops in JavaScript

A Fast Bytecode VM for Arithmetic: The Virtual Machine

Principles and Methodologies for Serial Performance Optimization

A deep dive into BPF LPM trie performance and optimization

Show HN: Incremental JSON parser for streaming LLM tool calls in Ruby

Helion: A High-Level DSL for Performant and Portable ML Kernels

FlashPack: Fast Model Loading for PyTorch

Why Your Social.org Files Can Have Lines Without Issues?

Opportunistic Mutation in Roc

We Deleted Tokio from Our Payment System and Cut Cloud Costs by $127,000

The Green Tea Garbage Collector

Vectorizing for Fun and Performance

What Caused Performance Issues in My Tiny RPG