What Are Embeddings?

What Is an Embedding?

An embedding is a numerical representation of data. For text, an embedding model maps words, sentences, paragraphs, or documents into a vector of numbers.

The important idea is that the vector is designed to capture useful relationships in the original data. Texts with related meanings can have vectors that are close together in the embedding space.

From Text to a Vector

Suppose we have three pieces of text. An embedding model converts each one into a vector with many numerical values.

The actual vector values usually do not have human-readable meanings such as “password = 0.44”. The useful information comes from the position of the complete vector in the embedding space.

Embeddings Capture Semantic Similarity

Consider these sentences:

What Does “Dimension” Mean?

A vector has dimensions. A tiny educational example might represent a sentence with only three numbers:

Real embedding models generally use far more dimensions. You do not need to interpret each dimension individually. Think of the complete vector as a point in a high-dimensional mathematical space.

Think of Embeddings as a Meaning Map

A useful mental model is a map. Similar concepts tend to occupy nearby regions, while unrelated concepts may be farther apart.

How Do We Compare Two Embeddings?

Once text has been converted into vectors, we need a mathematical way to compare them. A common choice is cosine similarity.

Similarity scores depend on the embedding model and the implementation. Do not assume one universal threshold works for every dataset.

Python Example: Create an Embedding

An embedding API can convert text into a vector that your application can store, compare, or send to a vector database.

from openai import OpenAI

client = OpenAI()

text = "How do I reset my password?"

response = client.embeddings.create(
    model="YOUR_EMBEDDING_MODEL",
    input=text,
)

vector = response.data[0].embedding

print("Dimensions:", len(vector))
print("First values:", vector[:5])

Python Example: Compare Two Embeddings

You can calculate cosine similarity with Python to estimate how closely two vectors are related.

import numpy as np

def cosine_similarity(a, b):
    a = np.array(a)
    b = np.array(b)

    return np.dot(a, b) / (
        np.linalg.norm(a) * np.linalg.norm(b)
    )

similarity = cosine_similarity(
    embedding_a,
    embedding_b,
)

print(similarity)

In a real application, the two embeddings would normally come from the same embedding model so that they share the same vector space.

Embeddings Power Semantic Search

Traditional keyword search mainly looks for matching words. Semantic search can retrieve content based on meaning.

Embeddings in RAG

Embeddings are one of the key building blocks of retrieval- augmented generation. A typical RAG pipeline turns documents into embeddings, stores them, embeds a user's query, and retrieves the most relevant chunks.

Where Are Embeddings Used?

Important Embedding Decisions

Common Misconceptions