We built a cloud GPU notebook that boots in seconds
20 days ago
- #gpu
- #cloud-computing
- #jupyter-notebooks
- Modal Notebooks is a cloud Jupyter notebook that boots GPUs and custom images in seconds with real-time collaboration.
- The system uses Modal Sandboxes for secure, isolated processes with high-performance capabilities like GPUs and lazy-loading filesystems.
- A daemon called modal-kernelshim translates Jupyter protocol messages into HTTP calls for remote kernel access.
- Lazy-loading container images reduce startup latency by fetching files on demand through a content-addressed tiered cache.
- Notebooks share a pool with Modal functions, balancing workloads across CPUs and GPUs, with automatic pausing to save costs.
- VolumeFS provides global, mutable, and fast storage, essential for AI workloads and persistent data.
- Real-time collaboration is enabled by Rushlight, using Redis Streams for broadcasting edits and CodeMirror for presence.
- Editor features include Language Server Protocol integration with Pyright, Ruff for auto-formatting, and AI completion with Claude 4.
- Modal Notebooks is built on years of foundational infrastructure work, focusing on speed, performance, and ease of use.