- NVIDIA's cuda-checkpoint allows freezing a CUDA process and serializing its GPU state into host memory, later restoring it exactly as it was.
- Checkpointing involves transferring GPU state into a host-side anonymous buffer, which can be manipulated directly (e.g., modifying a GPU counter) before restoration.
- The checkpoint operation is performed by a service thread within the target process, communicating via pipes, and uses ordinary driver ioctls rather than a dedicated checkpoint syscall.
- Checkpoint and restore speeds are limited by the allocation and deallocation of staging buffers, with checkpoint copying at ~3 GiB/s and restore at ~8 GiB/s, well below PCIe bandwidth limits.