Similarity Search

What Is Similarity Search?

Similarity search finds stored items whose vector representations are most similar to a query vector.

Instead of asking for an exact text match, the system compares the meaning represented by vectors. This makes it useful when a user's wording differs from the wording stored in the documents.

Where Similarity Search Fits

Similarity search usually sits between embedding generation and the final retrieval or generation step.

In a RAG system, the selected chunks can become context for the language model.

Query Vector vs Stored Vectors

The query is represented as a vector in the same embedding space as the stored document chunks.

The search operation calculates a similarity or distance between the query vector and candidate vectors.

Similarity Score

A similarity function produces a score that helps rank candidates. The exact meaning of the score depends on the metric.

Nearest Neighbors

Similarity search is often described as a nearest-neighbor search. The system looks for vectors that are closest according to the selected metric.

The word “nearest” does not necessarily mean physically close. It means close under the chosen vector similarity or distance function.

Similarity Search vs Keyword Search

Modern search systems often combine lexical and semantic retrieval instead of treating them as mutually exclusive.

Top-K Similarity Search

Similarity search can return the strongest K matches instead of returning every candidate.

For example, Top-K = 5 returns the five highest-ranked candidates after similarity search.

Simple Python Similarity Search

Here is a small example using normalized vectors and cosine similarity.

import numpy as np

query = np.array([1.0, 0.9, 0.1])

documents = {
    "refund policy": np.array([0.9, 0.8, 0.2]),
    "shipping policy": np.array([0.1, 0.2, 0.9]),
    "return window": np.array([0.8, 0.7, 0.2]),
}

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

ranked = sorted(
    (
        (name, cosine_similarity(query, vector))
        for name, vector in documents.items()
    ),
    key=lambda item: item[1],
    reverse=True,
)

for name, score in ranked:
    print(f"{score:.3f}  {name}")

The system calculates a score for each stored vector, sorts the candidates, and places the strongest matches first.

Applying Top-K in Python

Once candidates are ranked, slicing the sorted results gives a simple Top-K operation.

k = 2

top_results = ranked[:k]

for name, score in top_results:
    print(f"{name}: {score:.3f}")

In a production vector database, the database index performs the search efficiently rather than comparing every vector in Python.

Metadata Filtering

Similarity is not always the only requirement. A user may need results from a particular product, language, customer, or date range.

Similarity Search in RAG

Similarity search is one of the core retrieval operations in a RAG pipeline.

If the retriever returns poor evidence, a strong language model may still produce an unsupported answer. Retrieval quality is therefore a critical part of RAG quality.

Exact Search vs Approximate Search

For a small collection, an exact search can compare the query with every vector. At large scale, systems often use approximate nearest-neighbor indexes to reduce search work.

Why Similarity Scores Are Not Enough

A high similarity score does not guarantee that a chunk contains the correct answer. Embeddings capture relationships in vector space, not factual truth.

Reranking After Similarity Search

Some systems retrieve a broader candidate set first and then use a stronger reranker to order the candidates again.

Common Similarity Search Mistakes

How to Evaluate Similarity Search

Build a representative query set and measure whether the expected evidence appears in the retrieved results.