Which Design Doc Did a Human Write?
2 days ago
- The author created three design docs for the same open-source web app: one handwritten (16 hours), one by Claude Opus 4.6 (medium effort), and one by GPT-5.4 (high effort), then asked readers to identify the human version.
- Readers correctly identified the human-written doc by its personal opinions, anecdotal experience, unusual technology choices (e.g., NixCI, PolyForm-Noncommercial license), and non-generic architecture headings.
- AI-generated docs were spotted by 'bloat'—verbose, fluffy sentences, overuse of bold text, unrealistic time estimates (like '10 hours' milestones), and generic security/privacy sections lacking app-specific threats.
- Images were a dead giveaway: AI-generated diagrams had layout problems with colliding elements, so the author stripped diagrams from all versions.
- Both AI versions missed the hard problems—such as balancing security and usability in a photo-sharing app—and focused on generic best practices instead of the specific challenges of the project.
- The author concluded that human-written design docs convey the tough design trade-offs and competing interests, while AI tends to produce generic, bloated content.