- A key mistake in database benchmarking is oversizing workloads to the point where response times reach hundreds of milliseconds or seconds, making throughput numbers misleading if latency is high.
- HammerDB v6.0 introduces new response time metrics, including full percentile reporting and box plots, to visualize latency profiles for complex transactions, moving beyond averages to show medians, spread, and outliers.
- The update adds reservoir sampling for long runs, ensuring practical response time analysis without data bottlenecks, even with large transaction volumes.
- It emphasizes reviewing throughput and latency together; if throughput rises while latency spikes to hundreds of milliseconds, it indicates workload overload, and HammerDB v6.0 makes this easier to detect.