How to pack ternary numbers in 8-bit bytes
a day ago
- #ternary packing
- #SIMD optimization
- #BitNet b1.58
- The article discusses packing ternary numbers (trits) into 8-bit bytes efficiently for SIMD-friendly unpacking, focusing on a block size of 5 trits which fits into 8 bits with 99.06% efficiency.
- Packing involves converting trits (values -1, 0, 1) to base-3 digits (0, 1, 2) and using fixed-point arithmetic to avoid divisions during unpacking, enabling fast SIMD operations.
- A key technique uses multiplication and bit shifting to extract digits without modulo operations, as implemented in llama.cpp for TriLMs and BitNet b1.58, with SIMD support for AVX2 and ARM NEON.