Don't trust large context windows
7 hours ago
- #Attention Degradation
- #Coding Agents
- #LLM Context Windows
- LLM context windows have a 'smart zone' up to about 100k tokens where the model performs well, beyond which attention declines into a 'dumb zone'.
- Large advertised context windows (e.g., 200k, 1M, 2M) are largely marketing, as studies show effective context is much smaller and performance degrades as the window fills.
- Coding agents can quickly push sessions into the dumb zone through token-heavy tasks like file reads or debugging, making the advertised window size unreliable.
- Modern agents, such as Claude Code, use auto-compaction to summarize history and reset sessions, but this occurs after degradation and relies on a degraded model.
- A recommended strategy is to open new sessions with a self-written spec or artifact, providing higher signal handoffs to keep workflows in the smart zone.
- Projects like obra/superpowers and mattpocock/skills structure agent workflows around small, named artifacts (e.g., PRDs, plans) to maintain context efficiency.
- Treating the context window like a budget involves assuming only the initial chunk is usable and moving information into written artifacts to reduce cognitive load.