thunderbolt-ibverbs: We have InfiniBand at home
2 days ago
- #AI-Inference
- #RDMA
- #USB4
- A Linux kernel module developed to emulate InfiniBand devices using USB4/Thunderbolt ports on AMD mini PCs.
- Enables AI runtimes to distribute inference or training across multiple consumer systems without enterprise networking hardware.
- Achieved experimental RDMA-over-USB4 with 128 GB Strix Halo mini PCs, supporting tensor-parallel inference and FSDP workloads.
- Performance metrics: ~95 Gb/s bidirectional raw RDMA throughput, ~7 µs one-way latency, and significant speed improvements in AI tasks compared to Ethernet.
- Experimental and AI-generated code with risks, not intended for production use, based on research with potential issues.