8 hours ago
- NVLink on the RTX 3090 was tested for AI workloads, including inference and training, using two GPUs.
- Initial tests with NVLink showed benefits in tensor parallel prompt processing (~30% boost) and FSDP training (~3x improvement), but layer split, token generation, and DDP training were unaffected.
- The non-NVLink tests initially used NCCL_P2P_DISABLE, which forced GPUs to use shared host memory (SHM) rather than PCIe peer-to-peer, skewing results.
- After unplugging NVLink and applying a third-party driver patch to enable true P2P on consumer GPUs, the difference between NVLink and PCIe P2P largely disappeared for the author's server platform with full x16 slots.
- On consumer platforms with limited PCIe lanes (e.g., x8/x8 or x4), NVLink provides a significant advantage because it bypasses slot limitations, maintaining full bandwidth regardless of PCIe configuration.
- NVLink is worth considering if you are on a consumer platform with multiple GPUs, use workloads like FSDP training or tensor parallel prompt processing, and cannot apply the unofficial Linux-only P2P driver patch.