3 months ago
- Schema harness enables frontier models to achieve ~99% performance on ARC-AGI-3 Public by treating latent world representations as executable programs, allowing interpretable, verifiable, and searchable modeling.
- ARC-AGI-3 challenges agents to infer game rules, objects, and goals from raw 64x64 color grids without explicit instructions, using the Relative Human Action Efficiency (RHAE) metric for evaluation.
- The approach combines state grounding (identifying objects and variables from observations) and mechanism discovery (learning transition rules), solved jointly within an editable program that mirrors physicist-like reasoning.
- Controlled comparisons show Schema improves Claude Code baseline by 56.15%, with efficiency gains from verifying models against complete history and planning inside reusable simulators, reducing action counts significantly.
- Case studies highlight Fable models discovering world models more efficiently than Opus by better experimental decisions, representational revisions, and earlier identification of key mechanisms.
- The harness reduces theory usage costs through persistence and searchability, while the underlying model affects discovery efficiency; search completeness depends on the accuracy of the world model.
- Self-reported results on the Public set (98.98% with Claude, 95.35% with Sol) indicate progress but lack independent verification; extrapolation to Semi-private performance remains uncertain.