Specialty Digest

Global English editionLearners’ Study Dashboard — in development
The Knowledge Library for All

Vector Database

nounLarge Language Modelsalso vector store, vector search engine, vector DB

In one line

A vector database is a database designed to store embedding vectors and quickly find the ones most similar to a query, which supports semantic search and retrieval for AI applications.

A vector database is a database designed to store embedding vectors and to find quickly the ones most similar to a query. An embedding is a list of numbers that represents the meaning of a piece of text, an image or another item. A vector database holds millions or billions of these lists, each linked to its original content and metadata, and when given a new vector, it returns the stored items that lie closest to it. This makes it possible to search by meaning instead of exact keywords. Vector databases became popular after 2022 because they provide the retrieval layer for large language models that must answer questions using private or current documents.

History of vector search

Nearest-neighbor search is an old problem in computer science, studied since the 1960s with structures such as k-d trees, which work well in low dimensions but degrade in the hundreds of dimensions typical of embeddings. In the 2000s, locality-sensitive hashing offered approximate methods. In 2017, Facebook AI Research released the FAISS library for efficient similarity search on GPUs, and Google’s ScaNN and Spotify’s Annoy followed. The HNSW graph algorithm of 2016 and 2018 became the leading method for fast, accurate search. Dedicated vector databases, including Pinecone, Milvus, Weaviate, Qdrant and Chroma, appeared in the late 2010s and early 2020s, while established systems such as PostgreSQL with pgvector, Elasticsearch, OpenSearch, MongoDB and Redis added vector features. The rise of retrieval-augmented generation in 2023 drove rapid adoption.

How a vector database works

  1. Create embeddings: an embedding model turns documents, images or records into vectors, and often each long document is first split into chunks.
  2. Store vectors with metadata: the database saves each vector with an ID, the original text or a link to it and metadata such as source, date or access rights.
  3. Build an index: to avoid comparing a query with every vector, the database builds an index, a structure that narrows the search.
  4. Query: an application embeds the user’s question with the same model and asks the database for the k nearest vectors, optionally with filters on metadata.
  5. Return results: the database returns the matching items ranked by similarity, which the application can show or pass to a language model.

Similarity is typically measured by cosine similarity, dot product or Euclidean distance.

Index methods

Exact search compares the query with every vector, which is accurate but slow at scale. Approximate nearest neighbor methods trade a small loss of recall for big speed gains.

  • HNSW: a layered graph in which the search hops from coarse to fine layers toward the closest vectors. It is fast and accurate but uses a lot of memory.
  • IVF, inverted file: vectors are grouped into clusters, and the search looks only in the clusters nearest to the query.
  • Product quantization: vectors are compressed into short codes, reducing memory at some cost in precision. See quantization.
  • Locality-sensitive hashing: hash functions place similar vectors in the same buckets.
  • Disk-based indexes: designs such as DiskANN keep much of the index on solid-state drives to save memory.

Key features

Production systems offer filtering by metadata, hybrid search that combines keyword and vector scores, updates and deletions, sharding and replication for scale and reliability, access control, multi-tenancy and integrations with machine learning frameworks. Managed cloud services handle operations, while open-source options can run on a company’s own servers.

Applications

  • Retrieval-augmented generation: find relevant passages for a question and give them to a language model, improving accuracy and allowing citations.
  • Semantic search: search documents, products, code and support articles by meaning.
  • Recommendation: find items similar to those a user liked.
  • Image and video search: find visually similar pictures or scenes.
  • Deduplication and clustering: detect near-duplicate content.
  • Anomaly and fraud detection: flag items far from normal patterns.
  • Long-term memory for agents: store past interactions so that an assistant can recall them.

Vector database compared with traditional databases

A relational database stores structured rows and answers exact queries, such as all orders from a customer in March. A search engine matches keywords. A vector database answers similarity queries, such as the passages most like this question. Many teams use all three, and many traditional databases now include vector indexes, so a separate system is not always needed. A small collection of a few thousand documents can be searched with a simple library or even brute force, while very large or fast-changing collections benefit from specialized engines.

Challenges and limitations

  • Quality depends on embeddings: poor embeddings or chunking give poor results, so evaluation on real queries is essential.
  • Approximation: approximate search may miss some true neighbors.
  • Cost and memory: large indexes need substantial memory and compute.
  • Updates: changing the embedding model means re-embedding everything.
  • Security and privacy: vectors can leak information about the source text, and retrieval must respect permissions, so that users cannot retrieve documents they should not see.
  • Poisoning and prompt injection: malicious documents in the store can try to manipulate a model, so sources should be vetted.

Choosing a solution

Consider data size, query speed, filtering needs, update frequency, hosting preferences, cost, ecosystem and the team’s skills. Start with a simple option, measure retrieval quality and scale only as needed. Keep the original documents and embedding model version recorded so that the index can be rebuilt.

A worked example

A hospital wants nurses to search its care guidelines. It splits 5,000 guideline documents into paragraphs, creates embeddings and stores them in a vector database with metadata for department and date. A nurse asks how to handle a patient with a particular allergy. The system embeds the question, filters to current guidelines and returns the five closest paragraphs, which a language model summarizes with citations. A clinician reviews the answers on a sample of queries each month and flags errors.

Chunking and retrieval quality

The quality of answers from a retrieval system depends as much on how documents are prepared as on the database. Chunks that are too small lose context, while chunks that are too large dilute the match and waste space in the model’s context window. Common practice is to split on natural boundaries such as headings and paragraphs, keep an overlap between neighboring chunks and attach titles and section names as metadata. Teams then measure recall, the share of questions for which the right passage appears in the top results, and adjust chunk size, the embedding model and the number of results. Rerankers, which rescore the top candidates with a more precise model, often improve precision further.

Operations and cost

Running a vector database in production involves capacity planning, since memory use grows with the number and dimension of vectors, monitoring of latency and recall, backups and a plan for re-indexing when the embedding model changes. Compression, shorter embeddings and disk-based indexes can cut costs. Access control should be applied at retrieval time, so each user sees only the documents they are allowed to read, and logs should record which sources were used for each answer to support audits.

Future directions

Trends include tighter integration of vector search into general-purpose databases, better hybrid retrieval that combines keywords, vectors and structured filters, multi-vector representations that keep several vectors per document for finer matching and learned indexes that adapt to the data. As language-model applications spread, reliable, secure and explainable retrieval will remain a core requirement.

FAQs

Do I need a vector database to use embeddings? Not always. For small collections, an in-memory library works, but a database helps with scale, updates and filters.

What is the difference between a vector database and a vector index? An index is the search structure, while a database adds storage, updates, security and management around it.

Is a vector database the same as a knowledge graph? No. A vector database finds similar items by numeric closeness, while a knowledge graph stores explicit relationships between entities.

Key facts

  • A vector database stores embeddings and answers nearest-neighbor queries, returning the stored vectors most similar to a query vector.1
  • Approximate nearest neighbor algorithms such as HNSW trade a little accuracy for large gains in speed at scale.2
  • Vector databases are widely used for retrieval-augmented generation, semantic search and recommendations.3

Related concepts

Quick checkWhat does a vector database do?Show answer

It stores embeddings and quickly returns the items whose vectors are closest to a query vector, which lets applications search by meaning instead of exact keywords.

Sources

  1. Johnson, Douze and Jegou, Billion-scale similarity search with GPUs, https://arxiv.org/abs/1702.08734
  2. Malkov and Yashunin, Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs, https://arxiv.org/abs/1603.09320
  3. Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, https://arxiv.org/abs/2005.11401

Editorially reviewed by Specialty Digest Editorial TeamLast reviewed November 17, 2026Researched and drafted with AI assistanceReport an issue