Building a (Bad) Local AI Coding Agent Harness from Scratch
2 days ago
- #Local-GPU
- #AI
- #Node.js
- Built a terminal-based coding agent in ~400 lines of vanilla Node.js, running locally on a GPU with no cloud dependencies.
- Utilized Claude Sonnet 4.6 and Google Gemma 3 4GB model under Ollama for development.
- Demonstrated core concepts: Agent Loop, Tool Use, and Sandboxing in a rudimentary form.
- Implemented a simple protocol for LLM to interact with filesystem via fenced code blocks (read, write, ls).
- Sandboxing achieved by restricting file operations to a user-defined working directory.
- The agent loop maintains conversation history and drives turn-by-turn interaction with the model.
- Tool use protocol allows the model to perform file operations by emitting specially-tagged blocks.
- System prompt carefully designed to guide the model in following the custom protocol.
- Includes a /test command to verify the protocol adherence of the model.
- Despite some failures, the basic framework for a local coding agent was successfully demonstrated.