4-Bit Rotational Quantization: -45% RAM, <1% recall drop vs. TurboQuant
5 hours ago
- Weaviate 1.39 introduces 4-bit Rotational Quantization (RQ) with significant performance improvements over earlier versions.
- SIMD support for Fast Walsh-Hadamard Transforms (FWHT) speeds up vector rotation, with up to 6.1× faster transforms on certain CPUs.
- Encoding performance improves across all RQ variants, with RQ8 up to 3.8× faster and new RQ4 options being efficient.
- Distance kernels use SIMD nibble functions and optimized byte dot products, increasing speed up to 2.2× for 8-bit and enabling efficient 4-bit operations.
- Prefetching enhancements on both AMD64 and ARM64 architectures boost HNSW query throughput by 7-11% and imports by 12%.
- Centering and storing extreme coordinates improve recall, with RQ4c (centered) adding +0.1 to +6.1 percentage points in recall@10 across datasets.
- RQ4 demonstrates flat recall from 1M to 250M vectors, showing scalability with stable performance.
- RQ4c outperforms TurboQuant on most datasets, especially with centering and renormalization, and supports 8-bit where TurboQuant does not.
- 4-bit RQ reduces memory usage by about 45% compared to 8-bit, with default 8-bit retained in Weaviate Cloud, but 4-bit is available for cost savings.