Sparse resources helped my GPU-driven renderer memory usage
18 hours ago
- GPU-driven rendering requires efficient management of persistent GPU buffers, which can range from a few KiB to multi-GiB; reallocation strategies and suballocation are critical.
- Grow-or-shrink buffers cause peak memory spikes, require data copying, can invalidate referencing addresses, and suffer from external fragmentation, often needing defragmentation.
- A static buffer with a fixed maximum capacity avoids reallocation and fragmentation issues for small, bounded data but wastes memory when peak usage far exceeds typical usage.
- Block-based allocation (fixed-size blocks) reduces peak memory use and gives stable addresses but loses contiguous address space, complicating access; it's a viable option when sparse resources aren't available.
- CPU-style virtual memory using VirtualAlloc-like patterns outperforms static and block approaches: it provides stable pointers, contiguous address space, and commits only needed physical memory.
- Sparse resources (reserved resources in D3D12) offer a GPU-side virtual memory model: they provide a virtual address space with 64 KiB pages, and physical memory is mapped via UpdateTileMappings, supporting partial residency.
- Mapping and unmapping sparse resources is done via queue operations (UpdateTileMappings), which must be synchronized with GPU work; CPU-only mapping is required, and performance has improved significantly on modern drivers.
- Sparse resources enable persistent memory without reallocation, reduce physical cost by decommitting unused pages, and facilitate stable pointers for streaming systems, eliminating relocation work.
- A practical application is binning: predefined virtual address ranges can be used (e.g., for meshlet types) with sparse mapping to control physical memory usage while keeping a single buffer, though it requires CPU-GPU coordination for counts.
- Sparse resources are recommended for large, persistent buffers with a large gap between average and peak usage; for small buffers, simpler placed allocations are preferable.
- Challenges include page granularity (64 KiB), potential performance issues with small pages on NVIDIA, and the need for careful synchronization and CPU-side management.