- Author switches from DS4 Flash to Qwen 3.5 122B model due to high prefill latency (3-5 minutes for 50k tokens) impacting pair programming workflow.
- qMLX is a fork of rapid-mlx, specialized for hybrid Qwen models on Apple Silicon, adding disk KV restore, eviction, and metrics.
- Three bugs were fixed: removing unique timestamps from system prompts to ensure cache hits, persisting streamed replies on interrupts, and fixing checkpoint eviction to prioritize matchable entries.
- After fixes, cache hits increased dramatically; prefill times dropped from minutes to sub-second for repeated prompts, improving conversational throughput.