optimization

Bilingual

Faster floating point math with Rust's new API
2 months ago
- Floating point math is slower than integer math due to compiler conservatism to preserve exact results.
- Rust 1.98 introduces algebraic operators (e.g., `algebraic_add`) that allow compiler optimizations for floats.
- Pairwise summation with algebraic operators can match or beat NumPy speed while maintaining accuracy.
- Algebraic operations enable SIMD optimizations, reducing CPU instructions per value significantly.
- Combining strict and algebraic operators in algorithms balances performance and numerical precision.
Why we write our own C and C++ inference engines
2 months ago
- Deploying C++ ports yields a single shared library (66 MiB) instead of a multi-gigabyte Python virtualenv (9.1 GiB for vLLM), with predictable memory usage.
- vllm.cpp achieves throughput parity or slight advantage over vLLM across concurrency levels, with token-for-token identical output and lower peak memory (24.88 GiB vs 28.18 GiB).
- depth-anything.cpp runs 1.31x faster than PyTorch on CPU while using 27% of the memory, thanks to caching positional embeddings that were recomputed unnecessarily.
- face-detect.cpp and voice-detect.cpp prioritize exact output matching (cosine 1.0, pixel-level accuracy) over speed, enabling drop-in replacement without re-enrolling biometric templates.
- The porting method follows a strict order: convert weights to GGUF, port graph with per-component parity gates, optimize only after parity holds, and expose a flat C ABI.
- Maintenance cost is high: each engine needs its own CI, benchmarks, conversion scripts, and updates for new checkpoints; GPU kernels lag behind tuned cuDNN for convolution-heavy models.
- Porting is selective: many backends still wrap existing projects (llama.cpp, vLLM, whisper.cpp) when they are already excellent, large, and fast-moving.
Pulse Path – a browser puzzle where placement is the only decision
2 months ago
- You have two rounds with the same six links; only their placement changes.
- Pulse DMG starts at 10 and each placed link drains 7% (after max cut).
- Each path has a Max Pulse DMG; any pulse above it is discarded.
- Paths can split, loop, and recombine; balanced splits yield bonus.
- Special nodes: Twin boosts adjacent same links, Overhead removes split loss, Amplifier doubles pulse (watch max), Bypass removes link drain after it.
- Goal: maximize pulse DMG reaching the TARGET at the bottom.
Help, My Sin Is Slow and My FPU is Inaccurate - Bert Hubert's writings
2 months ago
- The author encountered a mysterious slowdown in sin() when computing exact multiples of π, which triggered glibc's slow path.
- FPU hardware instructions for trigonometry are inaccurate for large inputs due to the difficulty of range reduction.
- The Payne-Hanek-Corbett algorithm (1983) solved accurate range reduction for large angles in software.
- The slowdown occurred because the code computed sin(2π * Hz * t) with integer steps, producing exact multiples of π.
- As of 2021, glibc's libm has been optimized (patch by Wilco Dijkstra) eliminating the drastic slowdown for such inputs.
MicroAlloc
2 months ago
- The author created a custom memory allocator (MicroAlloc) inspired by studying the esp-idf heap allocator.
- General-purpose heap allocators involve trade-offs between speed, overhead, and fragmentation.
- The implementation uses a first-fit free list allocator with a metadata structure that stores offsets instead of full pointers to minimize overhead (2 bytes per allocation).
- Embedded engineers should generally avoid dynamic allocation; static allocation, scratch buffers, block allocators, and arena allocators are safer alternatives.
- Free() implementation includes merging adjacent free blocks, but merging can be deferred via a compile-time flag for better performance.
Matt Godbolt's blog
2 months ago
- Switch statements can compile to jump tables, but compilers often use more clever optimizations like arithmetic or bitmasks.
- For dense case values, compilers may build lookup tables or use bitmasks to avoid branches entirely.
- For sparse values, compilers may use binary search trees of comparisons or fall back to conditional branches.
- Different compilers (e.g., GCC vs Clang) employ different optimization strategies for switch statements.
- Programmers should write clear switch statements and trust the compiler to choose the best implementation.
Before You memo() — overreacted
2 months ago
- Before applying memo() or useMemo(), first try to split components that change from those that don't.
- Solution 1: Move state down by extracting the part of the UI that depends on the state into its own component.
- Solution 2: Lift content up by passing unchanged parts (e.g., as children prop) to the state-dependent component to prevent re-rendering.
- These techniques improve code clarity and data flow, with performance as a bonus.
- They complement memo() and useMemo()—use the Profiler and memo() only when these simpler methods aren't enough.
Welcome to the age of decadence without pleasure
2 months ago
- Modern wealth offers abundant distractions but often results in emptiness and boredom, creating an era of 'decadence without pleasure'.
- Historical decadence, like that of the late 19th century, involved extravagant art and a philosophy of 'useless pleasure' enjoyed for its own sake.
- Contemporary culture mimics decadence superficially (luxury, excess) but subordinates pleasure to optimization and algorithmic stimulation, leading to joylessness.
- Tech elites promote optimization regimes (e.g., Bryan Johnson's anti-aging, looksmaxxing) that treat pleasure as suspect and prioritize self-improvement over sensation.
- Pleasureless decadence trickles down from the wealthy, with inequality turning delights into status symbols and reducing fun to work-like routines.
- AI and algorithms amplify this trend by producing hollow content and fostering dependency, leaching pleasure from language and experience.
- A 'New Decadence' could reclaim everyday pleasures through art and mindful sensation, resisting optimization and reviving embodied joy.
GNU C Library 2.44 Released With /etc./tunables.conf, More Optimizations
2 months ago
- Introduced /etc/tunables.conf for system-wide tunables, more persistent than environment variable
- Implemented FMA-enhanced cosh function for ~35% performance improvement
- Imported optimized and correctly-rounded math functions from CORE-MATH project
- Added optimizations for LoongArch CPUs and introduced LoongArch32 32-bit support
- Optimized various functions on AArch64 and RISC-V
Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?
2 months ago
- Claude Fable 5 outperformed GPT-5.6 Sol on an NP-hard optimization problem, demonstrating superior consistency and producing the best overall solution.
- The /goal mode had mixed results; it improved outcomes in some runs but caused significant regressions in others, leading to worse average performance despite winning most individual trials.
- The study highlights that /goal is not a generic 'try harder' switch but alters the control loop and search path, which can amplify both good and bad decisions.
AttoChess, a complete, playable chess program for 16-bit x86 DOS in 278 bytes
2 months ago
- AttoChess optimizes by rethinking board display, move decoding, and search loop, saving 10 bytes.
- Board display draws directly to console using int 29h, eliminating render buffer and DOS function 09h.
- Startup removes BIOS mode-set, using deterministic assumptions and cld for smaller code.
- Input decoder folds constants into base address, using 16-bit pointer math and imul for efficiency.
- Search loop frees CX by comparing pointers, avoiding depth reload from stack.
- Pawn direction is folded into color bit with xor, handling both colors in one path.
From Muon to Gradient Clipping: Some Thoughts on QK Stability
2 months ago
- Muon optimizer works in function space by constraining the spectral norm of weight updates, aiming to limit the functional change for any input, contrasting with parameter-space optimizers like Adam.
- Applying standard Muon to Transformer's Q and K matrices leads to training instability because it separately constrains updates to W_Q and W_K, failing to control their coupled product in attention scores, which can cause spectral norm explosion.
- A principled attempt to modify Muon for QK involves directly constraining the change in attention score matrix S, leading to an optimization problem with a constraint coupling ΔW_Q and ΔW_K.
- Decoupling this constraint via the triangle inequality and solving subproblems results in a theoretically clean update formula, but it requires expensive pseudoinverse computations, making it impractical.
- A mathematical simplification reveals that the gradient G_Q naturally contains W_K, leading to a geometric interpretation involving a projection matrix, yet the external pseudoinverse remains, not solving the computational bottleneck.
- Viewing attention through a unified bilinear matrix B = W_Q W_K^T simplifies the Muon update to a single-matrix constraint, but this approach is impractical due to parameter explosion (d^2 vs. 2dh parameters).
- Multi-head attention can be seen as a low-rank approximation of full bilinear attention, connecting to LoRA's structure, suggesting potential for a Muon-LoRA optimizer for parameter-efficient fine-tuning.
- Due to computational hurdles, practical solutions shift to empirical approximations like gradient clipping or Kimi's MuonClip, which use batch information to approximate worst-case behavior and stabilize training.
Show HN: Small World – The "Preact" of 3D Web Engines (TypeScript, WebGPU)
2 months ago
- The text describes a hybrid rendering system with optimized Cook-Torrance PBR shading that compiles dynamically for WebGL2 and WebGPU.
- It includes a zero-allocation Finite State Machine utility executing natively in a recursive update loop for behaviors.
- Post-processing filters feature specialized shader passes for effects like night vision, VHS tracking, glitches, and Noir film.
- Conformal reflections involve mirror geometries, procedural checkerboard maps, and real-time sphere inversion math.
- Octree interactions provide DOM-like pointer events supported by optimized O(log n) raycasting.

optimization

Faster floating point math with Rust's new API

Why we write our own C and C++ inference engines

Pulse Path – a browser puzzle where placement is the only decision

Help, My Sin Is Slow and My FPU is Inaccurate - Bert Hubert's writings

MicroAlloc

Matt Godbolt's blog

Before You memo() — overreacted

Welcome to the age of decadence without pleasure

GNU C Library 2.44 Released With /etc./tunables.conf, More Optimizations

Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal help?

AttoChess, a complete, playable chess program for 16-bit x86 DOS in 278 bytes

From Muon to Gradient Clipping: Some Thoughts on QK Stability

Show HN: Small World – The "Preact" of 3D Web Engines (TypeScript, WebGPU)