Vector search is a method of finding items whose numerical vector representations, called embeddings, are closest to the vector of a query, using a distance measure such as cosine similarity. Text, images, audio and other data are converted by a model into lists of numbers in which similar items lie near each other, so searching for neighbors in this space finds similar items. Vector search is the engine behind semantic search, recommendation systems and the retrieval step of retrieval-augmented generation.
Embeddings and vector space
An embedding is a vector, often with 384 to 3,072 numbers, produced by a neural network trained so that related items receive similar vectors. A sentence about dogs lies near a sentence about puppies and far from one about tax law. Each dimension has no simple human meaning; the pattern across all of them carries the information. Because one model can embed text and images into a shared space, vector search can find pictures from a text description.
Measuring similarity
- Cosine similarity: compares the angle between vectors and ignores length. It is the most common choice for text.
- Dot product: multiplies matching components and adds them, and is equivalent to cosine similarity when vectors are normalized.
- Euclidean distance: the straight-line distance between points.
- Manhattan and Hamming distance: used for some data types, including binary codes.
The model’s documentation usually says which measure it was trained for, and using a different one may reduce quality.
Exact and approximate search
Exact nearest neighbor search compares the query with every vector, which is accurate but slow for millions or billions of items. Approximate nearest neighbor search, or ANN, builds an index that finds very close matches much faster, accepting a small loss of recall. The trade-off between speed, memory and accuracy is tuned for each application.
Common ANN algorithms
- HNSW: builds layered graphs of neighbors and navigates from coarse to fine layers. It offers high recall and speed but uses much memory.
- IVF: divides the space into clusters and searches only the nearest few clusters.
- Product quantization: compresses vectors into short codes to save memory.
- Locality-sensitive hashing: maps similar vectors to the same buckets.
- Tree-based methods: split the space recursively, as in the Annoy library.
- DiskANN-style indexes: keep much of the index on solid-state drives for very large collections.
These structures are a specialized form of database indexing for high-dimensional data.
Vector databases and libraries
Vector databases store embeddings together with metadata and provide indexing, filtering, updates and scaling. Examples include Pinecone, Weaviate, Milvus, Qdrant and Chroma. Libraries such as Faiss and ScaNN provide the core algorithms, and established databases such as PostgreSQL with the pgvector extension, Elasticsearch and OpenSearch now offer vector features. Choosing among them depends on scale, update frequency, filtering needs, cost and operational skills.
Filtering and hybrid queries
Real applications rarely search vectors alone. They filter by metadata, such as language, date or access rights, and may combine vector scores with keyword scores in hybrid search. Filtering can interact with the ANN index in tricky ways, and database designers use pre-filtering, post-filtering or filtered graph traversal to keep results both correct and quick.
Applications
- Retrieval for large language models and enterprise assistants.
- Product, music and video recommendations.
- Image and audio similarity search.
- Duplicate and near-duplicate detection.
- Fraud and anomaly detection by finding unusual points.
- Code and document search.
- Face and biometric matching, which raises privacy concerns.
Limitations
- Approximation: results may miss the true nearest item.
- Quality depends on the embedding model: a poor model gives poor neighbors.
- The curse of dimensionality: in very high dimensions, distances become less distinct.
- Cost: large indexes need substantial memory or storage.
- Updates: adding and deleting vectors can degrade some indexes until rebuilt.
- Opacity: it is hard to explain why two items are neighbors.
- Exact matches: identifiers and rare terms may be missed without keyword support.
A worked example
An online retailer has two million product descriptions. It embeds each into a 768-number vector and builds an HNSW index. A shopper types: lightweight waterproof jacket for hiking in autumn. The system embeds the query, finds the 50 nearest product vectors in a few milliseconds, filters out items that are out of stock and passes the rest to a reranker. The shopper sees jackets that match the intent, including ones that never use the word lightweight in their descriptions.
Tuning and evaluating a vector index
The main settings of an ANN index control a trade-off. In HNSW, the number of connections per node and the search breadth determine recall, speed and memory. In IVF, the number of clusters and the number probed at query time play the same role. Teams measure recall against an exact search on a sample of queries, then choose the cheapest configuration that meets a target such as 95 percent recall at a given latency. They also test with realistic filters and with the real mix of queries, since benchmarks on generic data can mislead.
Compression and efficiency
Vectors consume memory quickly: a hundred million vectors of 1,024 numbers in 32-bit form need about 400 gigabytes. Techniques that reduce this include storing numbers at lower precision, product quantization, binary codes and embedding models that produce shorter vectors with little loss, sometimes called Matryoshka embeddings because the first dimensions can be used alone. A common design uses compressed vectors for a fast first pass and full-precision vectors to rescore the best candidates.
Operations and governance
Running vector search in production involves monitoring recall and latency, rebuilding indexes after large changes, scaling by splitting data across machines and planning for model upgrades, which require re-embedding all content. Access control must be applied at query time so that users only see permitted results, and sensitive data should be handled with the same care as the source text, because embeddings can leak information about it.
History
Nearest neighbor search is an old problem in computer science, studied since the 1960s for pattern recognition. Tree structures such as k-d trees worked in low dimensions but slowed in high ones, which led to hashing and graph methods in the 2000s and 2010s. The growth of deep learning, which turns text and images into embeddings, made the problem central to modern software, and the arrival of retrieval for language models after 2020 brought vector databases into mainstream engineering.
FAQs
Is vector search the same as semantic search? Vector search is the technique, and semantic search is a goal that it helps achieve.
Do I need a vector database? Small collections can use a library or a database extension. Dedicated systems help at larger scale.
What does approximate mean? The index returns results that are very likely, but not guaranteed, to be the true nearest neighbors.
More in Retrieval and Knowledge Systems
Key facts
- Hierarchical Navigable Small World graphs, introduced by Malkov and Yashunin in 2016, are among the most widely used algorithms for approximate nearest neighbor search.1
- Facebook AI Research released the open-source library Faiss in 2017 for efficient similarity search over dense vectors, including on GPUs.2
- The ANN-Benchmarks project compares approximate nearest neighbor algorithms on recall and speed across standard datasets.3
Related concepts
Quick checkWhat is vector search?Show answer
It is finding the items whose vector embeddings are closest to a query's embedding, usually with an approximate nearest neighbor index so that results return quickly even for very large collections.
Sources
- Malkov and Yashunin, Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs, https://arxiv.org/abs/1603.09320
- Johnson, Douze and Jegou, Billion-scale similarity search with GPUs, https://arxiv.org/abs/1702.08734
- Aumueller, Bernhardsson and Faithfull, ANN-Benchmarks, https://arxiv.org/abs/1807.05614
Editorially reviewed by Specialty Digest Editorial TeamLast reviewed November 17, 2026Researched and drafted with AI assistanceReport an issue
