Tests Come for Free Now, and Most of Them Are Pointless
6 hours ago
- AI coding agents now generate tests automatically, but most are pointless and slow down development.
- Many projects lack good test suites; existing tests often fail on different machines and are skipped by developers.
- Agents produce many trivial tests (e.g., checking file content, comparing translation keys) that rarely catch real issues.
- Risky code (e.g., external API calls, scrapers) remains untested because it's hard to test, while easy code gets over-tested.
- A green test suite does not guarantee the app works; bugs still appear despite all tests passing.
- Letting AI interact with the app like a user—clicking through features with adaptive scripts—finds actual bugs more effectively.
- The author reduced their test suite from thousands to 63 key tests, running in 30 seconds, and agents now spend time exploring the app instead of waiting on tests.