I clustered four Framework Mainboards to test LLMs
16 days ago
- #AI Clustering
- #Hardware Testing
- #Framework Desktop
- Framework tested a mini-rack AI cluster using their AMD Ryzen AI Max+ 395-powered Mainboard.
- The cluster was built with four pre-release Mainboards, Noctua CPU Fan kits, and a 2U Mini ITX tray.
- Performance tests included single node and cluster AI inference, with mixed results across different tools.
- The cluster achieved over 1 TFLOP of FP64 performance but faced challenges with AI clustering tools.
- AI inference using the GPU showed solid numbers, but NPU support was untested due to driver issues.
- Networking limitations were noted, with Thunderbolt/USB4 ports only reaching 10 Gbps.
- The total cost for the cluster setup was around $8,004, comparing it to other high-end options.
- The conclusion highlights that AI clustering is still in early days and not yet mainstream.