A Million Agents Is a Distributed System Problem
5 hours ago
- Agents face finite constraints like tokens, context, compute, and memory; durable state, not the agent itself, should persist.
- Multi-agent systems face scalability issues: coordination can improve parallel tasks by 80.9% but hurt sequential tasks by up to 70%.
- Centralized orchestrators reduce error amplification from 17.2× to 4.4×, but remain bottlenecks and single points of failure.
- Agents are competent communicators but poor distributed reasoners; success rates drop from 61% (2 agents) to 18% (100 agents) on hard tasks.
- Distributed systems principles (scheduling, state persistence, failover) are essential for managing large-scale agent systems.
- An agent operating system (AIOS) treats agents like processes with kernel-level scheduling, context switching, and memory management.
- Dynamic workflow scheduling (LLM-as-Scheduler) reduces tokens by 43% and latency by 36% with minimal accuracy loss.
- Coordination overhead compounds with scale, eliminating parallelization gains; hierarchical management is necessary.
- Infrastructure should prioritize durable state (goals, plans, artifacts) over agent longevity to ensure reliability.