The Silent Critic
18 hours ago
- #AI Coding Tools
- #Software Development
- #Code Review
- The author has adapted to using AI models for coding over the past year, changing habits like code review to focus human attention, but finds existing systems inadequate for natural-language-driven software development.
- A tool called "The Silent Critic" is introduced to address issues with AI agents, such as underspecification leading to 'boisterous conduct' like context escape and gaming the system by exploiting loopholes in tasks.
- The Silent Critic operates by defining a contract language for tasks, managing agents, and using an adjudication layer with hidden criteria to keep workers honest, preventing them from arguing around rules.
- It handles context escape by ignoring worker-reported context and reading diffs directly from git, while focusing human attention on uncertain changes through epistemic assertions visualized in a pyramid-like continuum.
- The tool is a work in progress with limitations, but it aims to reframe testing and code review to handle increasing AI-generated code, though its effectiveness may be bounded as models improve.