Back to Study Material Deep Learning

Vector database

A vector database is a data management system designed to store high-dimensional numerical vectors and retrieve the nearest neighbors to a query vector eff...

Vector database

What a Vector Database Is and Why It Exists

A vector database is a data management system designed to store high-dimensional numerical vectors and retrieve the nearest neighbors to a query vector efficiently. Traditional relational databases are optimized for exact lookups and range queries over scalar values. Semantic search requires finding vectors that are geometrally close in an embedding space, which is a fundamentally different operation that conventional databases handle poorly.

In a RAG system, text passages are encoded into embedding vectors and stored in a vector database. When a query arrives, it is also encoded into a vector, and the database returns the passages whose vectors are most similar to the query vector. This enables semantic retrieval that matches conceptual meaning rather than exact keywords.

Exam tip: The core algorithm underlying most vector databases is approximate nearest neighbor search, abbreviated ANN. Exact nearest neighbor search scans all vectors and is too slow for large collections. ANN methods like HNSW, IVF-PQ, and ScaNN trade a small amount of recall for large speed gains by indexing the vector space. Be able to describe the trade-off between recall and query latency in ANN search.

HNSW, IVF, and Quantization

Hierarchical Navigable Small World, or HNSW, is one of the most widely used indexing algorithms for vector search. It builds a layered graph where each node represents a stored vector and edges connect nearby vectors. Search starts at the top layer with few nodes and navigates down layers, following edges toward the query vector at increasing granularity. This greedy graph traversal finds approximate nearest neighbors much faster than brute-force scanning.

Inverted file indexing, or IVF, partitions the vector space into clusters using k-means and assigns each vector to its nearest cluster centroid. A query is compared against only the vectors in the nearest few clusters rather than the entire collection, reducing the search scope at the cost of potentially missing neighbors in adjacent clusters.

Product quantization compresses vectors by splitting them into subvectors and encoding each subvector with a compact code from a small codebook, reducing storage by up to 32 times while introducing only modest recall loss. Combining IVF with product quantization, as in the FAISS library's IVF-PQ index, enables billion-scale vector search on hardware that would otherwise be memory-constrained.

Metadata Filtering and Production Considerations

Real applications often need to combine vector similarity search with structured attribute filtering. A document retrieval system might need to find the semantically closest passages written after a certain date or belonging to a specific category. Metadata filtering allows the vector search to be restricted to a subset of the index defined by scalar conditions on stored metadata fields.

Supporting metadata filtering efficiently requires the database to avoid scanning filtered-out vectors during search. Pre-filtering computes the matching subset first and then searches within it, which requires smaller specialized indexes per metadata value. Post-filtering performs vector search globally and then applies the metadata filter, but may return fewer than the requested number of results if many top vectors are filtered out.

Production vector database deployments must also handle updates: adding new vectors, updating metadata, and deleting stale entries. Many ANN indexes are optimized for batch construction rather than real-time updates, and insertions to an HNSW index are slower than queries. Evaluating the update throughput requirements of an application alongside query latency is essential when selecting a vector database for production use.