Experiences with Local Models for Coding

a day ago

Small local models for agentic coding were evaluated using a viability funnel covering RAM, speed, tool calling, functional correctness, conversation handling, task complexity, and code quality.
The evaluation process went through manual evals, automated evals, and day-to-day use, with tasks including sorting/cumulating a bar chart and creating a bar chart from access log data in JavaScript/TypeScript, plus Bash and Python scripts.
Results varied: some models like Qwen Coder Next 80B MoE produced functional code but crashed in extended conversations, while others like Gemma 4 26B succeeded manually but failed in automated tests, with performance differing between machines.
Day-to-day use with Qwen3.6 35B MoE showed success for small, well-defined tasks (e.g., Bash/Python scripts, website updates) but struggled with complex logic, requiring careful task selection and more code review.
Key factors affecting viability include task complexity, number of files to edit, specificity of instructions, and tech stack, with local models offering a 'detox' by encouraging slower, more thoughtful coding but lagging behind larger models in autonomy.

Hasty Briefsbeta