Treat Agent Output Like Compiler Output

13 hours ago

Philip Su argues that code reviews are becoming impractical and irresponsible in the age of coding agents.
The fear of 'lights-out codebases' stems from the lack of a verification apparatus similar to what we have for compilers.
Compilers don't require human review because of surrounding infrastructure like tests, type systems, and reproducible builds.
Current AI-generated code lacks robust formal specifications and deterministic verification layers.
The solution involves building upstream formal specifications, comprehensive test suites, AI verification pipelines, and better production monitoring.
The resistance to lights-out codebases is due to unfamiliarity with relocated trust mechanisms, similar to initial skepticism about compiler output.
Hardware verification models, with acceptance tests and formal methods, provide a blueprint for software agent verification.
The future requires formal specification layers, robust test infrastructure, AI verification pipelines, and advanced production instrumentation.

Hasty Briefsbeta