- Running many isolated agents in microVMs becomes expensive due to high RAM and disk usage for real workloads, despite Firecracker’s low overhead.
- The goal is to minimize cost while maximizing the number of concurrent sandboxes, using a linear cost model of CPU, memory, and disk hours.
- Simple ephemeral sandboxes with one-shot tasks are not suitable for long-lived agent sessions that need full state preservation.
- Snapshotting the entire microVM state (CPU, device, memory, disk) results in massive files, e.g., 144 GiB per sandbox, causing latency and cost issues.
- Content-addressable storage splits disk into 16 MiB chunks, hashed with SHA-256, and shared across sandboxes; only dirty chunks are stored per session.
- An NBD server with copy-on-write handles reads and writes, fetching chunks on demand and creating private copies only for modified data, reducing snapshot size and latency.
- Memory is handled similarly using MAP_PRIVATE and userfaultfd: a shared base memory file is populated on demand, with copy-on-write for dirty pages, keeping unique chunks small.
- This approach allows sandboxes to be captured and evicted when idle, shifting the cost model from requested resources to actual unique resources (base plus dirty chunks).
- The result is improved session density, where active sessions use physical resources and inactive sessions only store small manifests and unique chunks.