Context Language Models
4 hours ago
- Context Language Models (CLMs) natively manage their own context by treating it as a file with unrestricted updates.
- Zero-shot CLMs outperform current SOTA context management strategies across multiple tasks, improving accuracy by up to 11.4% and reducing FLOPs by up to 59%.
- CLMs naturally extend to multi-agent systems, where multiple agent contexts coexist as files.
- CLMs can be steered via natural-language instructions evolved through a skill-optimization loop, achieving up to 35.9-point accuracy gains on context-management tasks.
- An online reinforcement learning method improves CLM performance by 47.6% on BrowseComp-Plus while using 12% fewer FLOPs, and co-designed Suffix Cache Reuse reduces server-side compute by 35%.