Vector Representation

What Is a Vector Representation?

A vector is an ordered collection of numbers. In machine learning, a vector can represent an object, such as text, an image, a product, or another piece of data. An embedding model creates a vector that captures useful patterns learned from data.

From Meaning to Coordinates

Imagine placing information on a map. A physical map uses latitude and longitude. A vector representation uses many numerical dimensions instead. Together, those coordinates locate an item in a mathematical space.

This drawing has only two dimensions so humans can visualize it. Real embedding vectors commonly have hundreds or thousands of dimensions.

Dimensions Are Coordinates

If a vector contains 768 numbers, it has 768 dimensions. Each position contributes one coordinate to the complete representation.

Direction and Magnitude

Two useful geometric ideas are direction and magnitude. Direction describes the orientation of a vector, while magnitude describes its length.

Cosine similarity focuses on the angle between vectors, while other metrics can also take magnitude into account. The right choice depends on the application.

Why Similar Things Can Be Close

An embedding model is trained so that useful relationships in its training objective can be represented geometrically. As a result, texts with related meanings may have vectors that are close according to an appropriate similarity measure.

The first three examples are likely to be more relevant to a password-recovery query than the unrelated weather question. Actual rankings depend on the selected model and data.

Vector Similarity vs Exact Matching

Keyword matching looks for overlapping words. Vector similarity can retrieve text that expresses a related idea even when the wording is different.

Cosine Similarity

Cosine similarity measures the cosine of the angle between two vectors. It is commonly used when comparing embedding vectors.

The exact score range and interpretation depend on the metric, implementation, and vectors being compared. Do not use an arbitrary score threshold without evaluation.

Python: Represent a Vector

Python lists or NumPy arrays are convenient ways to work with vectors in small experiments.

import numpy as np

vector = np.array([
    0.21,
    -0.48,
    0.73,
    0.15,
    -0.09,
])

print("Dimensions:", vector.shape[0])
print("Vector:", vector)

Python: Calculate Vector Magnitude

The Euclidean norm gives the length, or magnitude, of a vector.

import numpy as np

vector = np.array([0.21, -0.48, 0.73])

magnitude = np.linalg.norm(vector)

print("Magnitude:", magnitude)

Python: Compare Two Vectors

Here is a compact example of cosine similarity using NumPy.

import numpy as np

def cosine_similarity(a, b):
    return np.dot(a, b) / (
        np.linalg.norm(a) * np.linalg.norm(b)
    )

a = np.array([0.8, 0.2, 0.1])
b = np.array([0.7, 0.3, 0.1])

score = cosine_similarity(a, b)

print("Similarity:", score)

In a real embedding application, a and b would usually come from an embedding model rather than being manually written numbers.

Other Ways to Compare Vectors

Cosine similarity is not the only option. Different applications can use different distance or similarity measures.

Vector Representation in RAG

Retrieval-Augmented Generation uses vector representations to help find relevant information before the language model generates an answer.

The vector representation is therefore one part of a larger retrieval pipeline. Good retrieval still depends on chunking, metadata, indexing, query formulation, ranking, and evaluation.

Vector Representation Beyond Search

Common Misconceptions