Hybrid Search

Hybrid Search

Hybrid Search combines sparse lexical retrieval (BM25) and dense semantic retrieval (Vector Search) to provide optimal query relevance. It executes both keyword matching and vector similarity lookups in parallel, and merges the resulting rank lists using Reciprocal Rank Fusion (RRF) to leverage the strengths of both retrieval styles.

Complexity Profile

Code Implementation

def reciprocal_rank_fusion(bm25_ranks, vector_ranks, k=60):
    # bm25_ranks and vector_ranks are lists of document_ids
    rrf_scores = {}
    
    # 1. Score BM25 rankings
    for rank, doc_id in enumerate(bm25_ranks):
        rrf_scores[doc_id] = rrf_scores.get(doc_id, 0.0) + 1.0 / (k + rank + 1)
        
    # 2. Score Vector Search rankings
    for rank, doc_id in enumerate(vector_ranks):
        rrf_scores[doc_id] = rrf_scores.get(doc_id, 0.0) + 1.0 / (k + rank + 1)
        
    # 3. Sort by highest RRF score
    sorted_docs = sorted(rrf_scores.items(), key=lambda item: item[1], reverse=True)
    return [doc_id for doc_id, score in sorted_docs]

Real-World Applications

• High-accuracy AI Search platforms (Perplexity, Cohere Rerank, Elasticsearch).
• Enterprise Retrieval-Augmented Generation (RAG) to fetch context.
• E-commerce product search engines matching both tags and semantic intent.

Architectural Analysis

[!tip] Deep Dive Best information retrieval approach. Instead of choosing between exact keyword matching (BM25) and conceptual semantic meaning (Dense Vector Embeddings), Hybrid Search runs both in parallel and fuses their scores using Reciprocal Rank Fusion (RRF). This avoids vocabulary mismatch issues while preserving structural keyword filtering.