There's no point at which turning your brain off will work
3 hours ago
- Variants of being a 'meat proxy' involve humans guiding LLMs in coding loops, with improved effectiveness by September 2026 but still insufficient for production-quality work.
- If LLMs improve enough, companies could bypass human meat proxies entirely, running LLMs in loops and laying off employees, undermining the employee's value.
- Luke Burton notes that meat proxy success depends on task value; high-value tasks require human oversight (QA, architecture) and agents often fail on subtle or out-of-distribution problems.
- Agents struggle with out-of-distribution scenarios (e.g., obscure languages, board games like Dominion) where they produce plausible-sounding but wrong answers, unlike humans.
- Coding tasks reveal out-of-distribution questions where agents underperform humans, demanding human intervention for good overall results.
- Agents overfit to tests or metrics (eval-shaped problems), producing software that works poorly, as seen in real examples of non-functional bots or infinite loops in commercial products.
- Claims of AI solving coding are exaggerated; reviewed software often doesn't work, and social media praise contrasts with poor real-world agent output (e.g., Gary Bernhardt's example of needless additions).
- Objective measures (game AI performance, business metrics like churn) show that LLM-boosted software often underperforms, suggesting many claims are self-deception.