<antirez>
8 hours ago
- HyperLogLog is a randomized algorithm that approximates the number of unique elements in a set using constant, small memory.
- It uses hashing to assign elements to registers and counts the longest run of leading zeros to estimate cardinality.
- Redis implements HyperLogLog with 12KB per key, 16384 registers, and a standard error of 0.81%.
- The API includes PFADD, PFCOUNT, and PFMERGE commands, all operating on string-encoded HLL values.
- Performance optimizations include unrolled loops, precomputed lookup tables, and caching of the last computed cardinality.
- Bias correction for small cardinalities uses polynomial regression based on a four-order polynomial, improving accuracy in the range 40k–72k.
- The implementation uses a 64-bit hash function and 6-bit counters, eliminating practical limits on cardinality up to 2^64.