Scaling to 1M concurrent sandboxes in seconds
5 hours ago
- #container
- #scalability
- #sandbox
- Modal has rebuilt its sandbox platform to handle massive scale, enabling millions of concurrent sandboxes and creation rates of tens of thousands per second.
- Traditional container platforms like Kubernetes face scaling bottlenecks from O(containers) or O(nodes) operations, such as centralized scheduling and etcd writes.
- Modal's new architecture eliminates central coordination, using a fleet of scheduling servers and workers as independent sources of truth, with no durable stores in the critical path.
- Benchmarks show the system can create 1 million sandboxes in under a minute, with median start times under half a second and no significant degradation at scale.
- The development involved a focused prototype effort and required reimplementing features and optimizing container startup to handle high concurrency.