Back to Study Material Deep Learning

Semantic search

Traditional keyword search retrieves documents that contain the same words as the query. It fails when the query and the relevant document use different vo...

Semantic search

Semantic Search Versus Keyword Search

Traditional keyword search retrieves documents that contain the same words as the query. It fails when the query and the relevant document use different vocabulary to express the same meaning. A search for heart attack may not return documents that use only the term myocardial infarction even though the two phrases are synonymous. Semantic search addresses this by encoding both query and documents into an embedding space where semantically similar text is placed near each other regardless of word choice.

The enabling technology is a text embedding model, typically a transformer trained with a contrastive objective on pairs of semantically related texts. Bi-encoder architectures encode query and document independently and use vector similarity, typically cosine similarity or dot product, as the relevance score. This makes them fast enough for large-scale retrieval because documents can be encoded and indexed in advance.

Exam tip: The difference between a bi-encoder and a cross-encoder is critical for understanding retrieval system design. A bi-encoder encodes query and document separately and compares their vectors, enabling pre-computation. A cross-encoder encodes the query and document jointly, allowing richer interaction at the cost of not supporting pre-indexing. Cross-encoders are used for re-ranking rather than first-stage retrieval.

Embedding Models for Semantic Search

The quality of semantic search depends heavily on the embedding model used. General-purpose sentence embedding models like those in the sentence-transformers library produce useful representations for a wide range of queries, but specialized embedding models trained on domain-specific data consistently outperform general models within their domain.

MTEB, the Massive Text Embedding Benchmark, provides standardized evaluation of embedding models across tasks including retrieval, clustering, classification, and semantic similarity. Consulting MTEB leaderboard results for the retrieval task is a practical first step when selecting an embedding model for a new application, because it provides standardized comparable performance figures across dozens of models.

Newer embedding models support matryoshka representation learning, which trains embeddings to be useful at multiple truncated dimensions. This allows practitioners to trade retrieval quality for storage and compute cost by using shorter embedding vectors without retraining the model.

Practical Deployment and Hybrid Systems

A production semantic search system involves several components working together. An offline pipeline encodes all documents into vectors and stores them in a vector database. An online pipeline encodes incoming queries and retrieves nearest neighbors from the database. Results are typically re-ranked by a cross-encoder before being returned to the user.

Cold start is a practical challenge: before any documents are indexed, the system returns nothing. Incrementally indexing documents as they are created, rather than waiting for a bulk import, allows the system to be useful immediately, though the index may be incomplete early on.

Hybrid search, combining semantic similarity scores with keyword match scores, consistently outperforms either approach alone on benchmark evaluations. Reciprocal rank fusion is a simple fusion method that combines rankings from multiple retrieval systems without requiring tuned weight parameters. For most applications, starting with a hybrid BM25 plus dense retrieval system with reciprocal rank fusion is a strong baseline that can be refined with domain-specific embedding fine-tuning if needed.