Blockdiff: We built our own file format for VM disk snapshots
9 hours ago
- #open-source
- #systems engineering
- #VM optimization
- Blockdiff is an open-source tool for rapid block-level diffs and snapshots of VM disks.
- Developed to address slow VM startup and snapshot times, reducing EC2 snapshot times from 30+ minutes to seconds.
- Uses a custom file format for instant block-level diffs, leveraging Linux's CoW implementation in XFS.
- Designed for dev environments, enabling efficient snapshotting without redundant storage of OS data.
- Supports features like sleep/wake with minimal data transfer and disk rollback via incremental snapshots.
- Avoids solutions like OverlayFS and ZFS due to limitations in performance and functionality.
- Implemented in Rust, focusing on simplicity, zero overhead, and instantaneous operations.
- Utilizes sparse files and copy-on-write (CoW) for efficient storage and quick operations.
- Includes a performance comparison showing significant speed advantages over traditional methods like tar and qcow2.