1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5
a day ago
- #High Performance Computing
- #AI Clustering
- #Apple Silicon
- Apple provided access to a Mac Studio cluster to test RDMA over Thunderbolt in macOS 26.2, enhancing memory pooling for AI models.
- The Mac Studio cluster, costing nearly $40,000, features 1.5 TB of unified memory and is loaned by Apple for testing purposes.
- RDMA support reduces memory access latency from 300μs to <50μs, significantly boosting performance for large AI models.
- The M3 Ultra Mac Studio excels in local AI model performance, creative applications, and small-scale scientific computing with low power consumption.
- Testing involved comparing the M3 Ultra with Dell Pro Max and AMD AI Max+ systems, showing superior performance and efficiency.
- Clustering Mac Studios presents challenges, including macOS management hurdles and the lack of Thunderbolt 5 switches for networking.
- Exo 1.0 introduces RDMA support for Mac clustering, enabling efficient running of massive AI models across multiple nodes.
- Stability issues with RDMA over Thunderbolt were noted, alongside concerns about Exo's development transparency and future support for older Mac models.
- Unanswered questions remain about future Apple hardware developments, including the potential for an M5 Ultra and improved networking options.
- Despite the hype, Thunderbolt 5 limitations and macOS management complexities may hinder the Mac Studio cluster's broader adoption.