Files over tools: how we built our agent with a virtual filesystem and bash

6 hours ago

Knock Agent is an AI agent for managing customer messaging resources, capable of creating and updating workflows, templates, audiences, segmenting users, and answering performance questions.
The agent was built using a virtual filesystem and bash, inspired by Vercel's approach, allowing it to explore and script on resources via a filesystem instead of exposing numerous API tools.
A custom Elixir-based virtual bash interpreter called just-bash was developed to provide a lightweight, in-memory environment without the performance overhead of a full sandbox.
The agent uses a durable workflow with Oban and Postgres, lazy-loading resources into the filesystem and providing tools like bash, read_file, edit_file, write_file, and upsert_resource for operations.
Skills are introduced to teach the agent resource-specific nuances, and a CLI (knock command) is integrated for querying dynamic data like logs, reducing the need for additional tools.
Observability is achieved through OpenTelemetry traces to Honeycomb and Braintrust, with a homegrown evals system using LLM-as-judge for testing and monitoring agent performance.
Future improvements include better session compaction, expanded evals, and exploring subagents for focused tasks, building on the filesystem-based foundation to enhance agent capabilities.

Hasty Briefsbeta