Harness Handbook – Making Agent Harnesses Understandable, Auditable and Editable
10 hours ago
- Agent harness behavior is scattered across many implementation sites, making it hard to understand, audit, or adapt.
- Harness Handbook organizes implementation into a behavior-level manual with three levels (L1, L2, L3) linked to code evidence.
- Key problems include complex codebases (e.g., Codex has 2,267 files) and need for behavior localization to answer specific questions.
- The Handbook supports three goals: understand system flow, audit behavior against expectations, and adapt for building custom agents.
- Behavior-Guided Progressive Disclosure (BGPD) allows users to start from a behavior question and trace to code evidence layer by layer.
- Experiments show Handbook-assisted coding agents achieve higher localization accuracy with lower token cost, especially for complex tasks.
- Handbook Studio provides an interactive workbench for reading, verifying, and making changes on the behavior map.
- Example change (temporary environment variables) demonstrates how Handbook reduces fan-out of edits across 10 files and 14 sites.
- Takeaway: complex harnesses need a behavior-level map for understanding, auditing, and adapting, with code evidence at every step.