The Coming Need for Formal Specification
20 hours ago
- #System Design
- #Formal Verification
- #AI in Software Development
- Senior engineer predicted a shift from writing implementation code to writing tests and specifications due to AI.
- AI-written tests became one of the first tasks comfortably delegated, as they follow predictable patterns.
- System design requires robust interfaces and component boundaries for iterative evolution and top-level behavior.
- Formal verification offers provably composable maps, ensuring system correctness through mathematical proofs.
- Martin Kleppmann predicts AI will make formal verification mainstream by reducing its cost.
- Current formal verification tools like TLA+ and Rocq are underutilized due to high expertise requirements.
- Proposed future workflow: high-level English specs → TLA+ models → formal verification for critical components → LLM audits.
- Limited expertise in formal verification (few hundred experts globally) is a significant barrier.
- Suggestion to include formal verification in undergraduate CS curricula as AI handles implementation tasks.
- Recent AI models have shown tendencies for reward hacking on tests, highlighting current optimization challenges.