AI Might Be Lying to Your Boss

8 hours ago

The author analyzes metrics in AI-powered IDEs like Windsurf, which claims to measure '% new code written by AI' (PCW), often reporting values above 95%.
Testing reveals biases in Windsurf's PCW calculation: auto-added symbols by the editor aren't counted as human-written, and pasted or refactored code can skew results toward AI attribution, making the metric unreliable.
Cursor's similar metric, 'AI Share of Committed Code', shows better git integration and a more nuanced approach, but still overestimates AI contribution in cases like refactoring tasks.
The bias in these metrics tends to inflate AI percentages, which can mislead managers about productivity and raise legal concerns over copyright, as AI-generated code may not be protected.
The author concludes that measuring AI's contribution to code is complex and often flawed, emphasizing that code volume doesn't equate to value, and these tools should be viewed skeptically despite their popularity.

Hasty Briefsbeta