Precision over perception: Why architecture matters in benchmarking
4 days ago
- #Kubernetes density
- #benchmark analysis
- #hybrid cloud
- VMware's claim of 5.6x pod density advantage for VCF 9.0 with VKS over Red Hat OpenShift is based on a study that compared 300 virtual nodes vs 4 bare-metal nodes, skewing the results.
- The test highlighted per-node density: OpenShift achieved 1,850 pods per node, 13 times higher than VKS's 140 pods per node, but the headline focused on aggregate totals.
- The methodology involved asymmetric configurations, such as VKS using 200 maxPods per node vs OpenShift's 5,000, and omitted a direct virtual-to-virtual comparison with OpenShift Virtualization.
- Pod readiness advantages for VKS were attributed to queuing theory from using many nodes, not software superiority, and the synthetic workload (kube-burner) didn't reflect real-world business applications.
- Real-world pods require significant resources, I/O, and networking, which the test ignored, making high-density results from empty or lightweight pods irrelevant for production capacity planning.
- Running 300 VMs introduced virtualization overhead with overcommitted resources, a setup that wouldn't hold up under actual workloads, but dense virtualized workers are valid in proper configurations.
- Red Hat advocates for fair benchmarks, comparing equivalent architectures with production-like workloads, and promotes OpenShift Virtualization for VM and container integration on a level playing field.