Querying 3B Vectors
4 days ago
- #numpy
- #optimization
- #vector-search
- The author explores optimizing a vector search problem involving 3 billion vectors and 1,000 query vectors.
- Initial naive implementation took 2 seconds for 3,000 vectors, which was inefficient.
- Vectorizing the numpy operations improved performance significantly, reducing time to 0.017 seconds for the same dataset.
- Further optimization by converting data to float32 reduced the time to 0.0045 seconds for 3,000 vectors.
- Scaling up to 3 million vectors took 12 seconds, projecting to ~3,216 minutes for 3 billion vectors.
- Memory constraints became a problem, with 3 billion vectors requiring ~8.6 TB of memory.
- Potential solutions include using generators, batching, memory mapping, or rewriting in optimized languages like Rust or C.
- The author highlights the importance of clarifying requirements, such as the nature of queries, hardware specs, and data representation, before proceeding with optimizations.