3 months ago
- Author tests DeepWiki, a tool that generates documentation for GitHub repositories.
- Documentation for mod_blog (7,400 lines) was mostly accurate but had inconsistencies like incorrect layer count and interface issues.
- Documentation for a09 (9,500 lines) contained significant errors, including incorrect backend matrix tables and misrepresentations of features.
- Author attributes errors in a09 documentation to higher cyclomatic complexity and potential LLM context window limitations.
- Concerns raised about updating documentation for evolving codebases and potential inaccuracies in legacy projects.
- Overall assessment: less problematic than LLM-generated code but too unreliable for unfamiliar codebases.