- The author expresses skepticism about LLM coding assistants, not due to IP, ecological, or quality issues, but based on scientific evidence questioning their ability to improve coding speed or quality.
- A key problem is the high error rate of LLM coding assistants, akin to an 'intern' producing incorrect code, with the proposed solution being thorough code review.
- Empirical research shows effective code reviews have limits: they should not exceed 1 hour or 400 lines of code (LOC) per hour to maintain efficiency.
- Applying these review limits to LLM-generated code means a developer can only review a few thousand LOC per day at best, imposing a productivity ceiling.
- Evidence suggests human reviewers of LLM-generated code are less effective—finding fewer defects while being overconfident—compared to reviewing human-written code.
- Proponents often promote LLM assistants for writing complex code like Bash scripts, which is hardest to review, exacerbating the review problem.
- To address skepticism, the author calls for empirical research on human reviewers' ability to detect defects in LLM-generated code and how reviewing such code differs from human code.
- The author is frustrated by proponents ignoring these issues and relying on anecdotal evidence instead of engaging with scientific data on code review practices.