Simple Meta-Harness on Islo.dev
3 hours ago
- #Sandbox Infrastructure
- #Automated Optimization
- #LLM Agents
- A harness combines the prompt, tools, and scaffolding for an LLM agent, while a meta-harness is an automated loop that improves the harness by analyzing diagnostic logs and proposing enhancements.
- The key bottleneck for meta-harness is diagnostic context; instead of summary statistics, it provides up to 10 million tokens of raw execution traces for analysis.
- Islo sandboxes provide the necessary runtime primitives for meta-harness, including snapshot saving, parallel forks, and durable diagnostic logs, enabling cheap trace production and storage.
- A proof-of-concept orchestrator, written in ~200 lines of bash, integrates Islo's primitives with a deterministic simulator and a pattern-matching proposer, demonstrating optimization from 0/5 to 5/5 tasks in four iterations.
- The meta-harness loop revealed a transfer-fix where a hint for one task inadvertently fixed another, highlighting the value of full diagnostic traces over summary scores.
- The implementation includes a live dashboard for visualization, task specifications, and extensible components for swapping simulators with real agents like Claude.