Register deprivation: spills and runtime under forced register scarcity

2 days ago

Reserving registers via gcc -ffixed-<reg> reliably increases spills, but static spill count is a weak predictor of runtime cost (Pearson r=0.55).
Removing registers slows 8 of 9 kernels by 14–76% at the tightest budget; SipHash shows no slowdown despite 23 added spills.
The slowdown is specific to the register file used: e.g., reserving GP registers on SHA-256 costs +33%, while reserving XMM costs only +5.6%.
GCC auto-vectorizes part of SHA-256 with SSE, causing XMM reservation to increase stack spills 8-fold.
Per-kernel cost per added spill varies ~30x, from about 0%/spill (SipHash) to 2.2%/spill (FIR filter).
Out-of-order execution hides spill latency, making spills cheap; historical x86-32 would have had higher and more uniform cost.
The experiment used gcc 15.2.0 -O2 on an Intel Xeon E-2236; code is available on GitHub.

Coding agents send similar large prompts repeatedly, making prompt caching crucial for efficiency and cost.
The KV cache stores key and value tensors per token and layer, enabling reuse of processed prefixes.
Session affinity keeps the cache on the same GPU; distributed caching spreads it across workers, offering flexibility but with system overhead.

Claude Code uses significantly more tokens than OpenCode for the same tasks, with baseline overhead around 33,000 tokens vs 7,000 for a simple reply.
Cache inefficiency is a major factor; Claude Code rewrites tens of thousands of prompt-cache tokens mid-session, leading to up to 54x more cache writes than OpenCode.
Configuration elements like instruction files and MCP servers add substantial token overhead, with a real setup reaching 75,000–85,000 tokens before user input.

TabFM beats an Optuna-tuned XGBoost in a zero-shot setup on small-to-medium datasets.
A multi-seed check led to demoting two narrow wins to ties due to measurement noise.
The GPU memory footprint is about 16.95 GB, not 22.75 GB, after adjusting for XLA allocator artifacts.

AI costs are rising, leading companies to replace token leaderboards with daily quotas and seek cost reduction strategies.
Compacting agent traces after periods of user inactivity can save up to 50% on coding workloads, using a model that runs at over 50k tok/s without harming user experience.
In long-running conversations, input tokens (not output tokens) dominate costs due to cache reads and rewrites, with costs growing quadratically with session length.

Zig's incremental compilation detects changed functions/declarations, recompiles only those, and patches the binary for fast rebuilds.
The feature is now viable for real-world projects and used daily by the Zig core team.
A demo shows rebuilds taking 50-70ms after an initial 5-second build.

Coding agents send similar large prompts repeatedly, making prompt caching crucial for efficiency and cost.
The KV cache stores key and value tensors per token and layer, enabling reuse of processed prefixes.
Session affinity keeps the cache on the same GPU; distributed caching spreads it across workers, offering flexibility but with system overhead.

Claude Code uses significantly more tokens than OpenCode for the same tasks, with baseline overhead around 33,000 tokens vs 7,000 for a simple reply.
Cache inefficiency is a major factor; Claude Code rewrites tens of thousands of prompt-cache tokens mid-session, leading to up to 54x more cache writes than OpenCode.
Configuration elements like instruction files and MCP servers add substantial token overhead, with a real setup reaching 75,000–85,000 tokens before user input.

TabFM beats an Optuna-tuned XGBoost in a zero-shot setup on small-to-medium datasets.
A multi-seed check led to demoting two narrow wins to ties due to measurement noise.
The GPU memory footprint is about 16.95 GB, not 22.75 GB, after adjusting for XLA allocator artifacts.

AI costs are rising, leading companies to replace token leaderboards with daily quotas and seek cost reduction strategies.
Compacting agent traces after periods of user inactivity can save up to 50% on coding workloads, using a model that runs at over 50k tok/s without harming user experience.
In long-running conversations, input tokens (not output tokens) dominate costs due to cache reads and rewrites, with costs growing quadratically with session length.

Zig's incremental compilation detects changed functions/declarations, recompiles only those, and patches the binary for fast rebuilds.
The feature is now viable for real-world projects and used daily by the Zig core team.
A demo shows rebuilds taking 50-70ms after an initial 5-second build.

Hasty Briefsbeta