A visual exploration of vector embeddings
a year ago
- #vector-embeddings
- #similarity-metrics
- #machine-learning
- Vector embeddings map inputs like words or images to lists of floating-point numbers representing them in a multidimensional space.
- Different embedding models (e.g., word2vec, text-embedding-ada-002, text-embedding-3-small) have unique dimensions, input types, and similarity characteristics.
- Similarity spaces allow comparing vectors using metrics like cosine similarity, with rankings varying across models.
- Vector similarity metrics include cosine similarity, dot product, Euclidean distance, and Manhattan distance, each suited for different scenarios.
- Vector search enables finding semantically similar items across languages or media types, using exhaustive or approximate nearest neighbor (ANN) algorithms.
- Vector compression techniques like quantization (scalar, binary) and dimension reduction (e.g., MRL) save storage and computation while preserving semantic information.
- Compression with rescoring combines compressed vectors for indexing with original vectors for high-quality search results.
- Resources for further learning include Jupyter notebooks, talks, and documentation on embedding models and vector databases.