The octopus architecture for AI agents
4 hours ago
- #LLM Systems
- #AI Agents
- #Software Architecture
- The Octopus Architecture involves a central brain dispatching tasks to semi-autonomous appendages, each with its own brain, reporting back.
- Key components include static lanes (long-lived appendages like Curator), lane templates (capabilities for bounded purposes), and sandbox snapshots (filesystem starting points).
- The architecture balances responsiveness (bounded turns, avoiding I/O), capability (delegation for complex tasks), and continuity (single LLM conversation maintaining personality and memory).
- All user interactions across surfaces (e.g., threads, channels) go through the same foreground conversation to foster emergent behavior and cross-platform links.
- Foreground lanes communicate with other lanes via text, using templates or messages, while lanes handle messy work like tool calls and I/O, containing context locally.
- Lanes communicate through chat or shared virtual filesystem artifacts, keeping the foreground clean for intent and synthesis, with asynchronous and synchronous compaction to manage context size.
- Benefits include low mean-time-to-interaction (foreground stays responsive), context and cache efficiency (stable prompt for better API cache hits), and curation/compaction to prevent context bloat.