Vector Database

Vector Database is a specialized storage system that indexes high-dimensional embedding vectors and retrieves the closest matches to a query vector via approximate nearest-neighbor search, typically returning results across millions of records in single-digit milliseconds using graph or cluster-based index structures. Unlike a relational store, which compares rows by exact equality, a vector database ranks items by geometric distance—cosine similarity, dot product, or L2 distance—so semantically similar content surfaces even when no keywords overlap. Modern engines such as Pinecone, Weaviate, Milvus, Qdrant, and pgvector expose this through ANN indexes like HNSW (Hierarchical Navigable Small World graphs) and IVF (Inverted File clusters), trading a small, tunable recall penalty for orders-of-magnitude speedups over brute-force scan. They are the retrieval substrate behind semantic search, recommendation, and most Retrieval-Augmented Generation pipelines.

How it works

At ingest time, each document or object is converted by an embedding model into a fixed-length float vector, then inserted into an ANN index. HNSW builds a multi-layer proximity graph where queries traverse from coarse to fine layers, pruning the search space logarithmically; IVF instead partitions vectors into Voronoi cells via k-means and probes only the nearest clusters. A query vector walks the same structure, collecting candidates whose distance is computed against the probe, then reranks the shortlist for final ordering. Parameters like efSearch (HNSW) or nprobe (IVF) let operators dial the recall-latency trade-off per workload. Most engines also attach metadata filters that narrow the candidate set before or after the vector search.

Why it matters for AI engineers

Choosing a dedicated vector database versus pgvector in an existing Postgres instance is the central shipping decision: pgvector keeps embeddings beside relational data, simplifying transactions, backups, and the security posture, but its HNSW index performance degrades past roughly 1–10 million vectors and under heavy concurrent writes. Managed services like Pinecone or Qdrant Cloud offload operational burden—sharding, replication, snapshotting—at the cost of egress fees, vendor lock-in, and a second data store to keep in sync. Latency budgets matter because ANN retrieval sits on the critical path of every RAG query; a 50 ms index lookup plus the embedding API call often dominates end-to-end response time. Recall must be measured against an eval harness, since an ANN index that returns the wrong neighbors silently corrupts grounding without raising any error.

Vector Database vs. alternatives

Related terms