GENERATIVE AI • LESSON 7

Cosine Similarity

Cosine similarity is an important concept in Embeddings, Semantic Search, Vector Databases, and RAG.

CORE IDEA

Cosine similarity measures how similar two vectors are by comparing the direction they point.

The Main Idea

Cosine similarity looks at the angle between two vectors.

Small angle
     ↓
Similar direction
     ↓
High similarity

Large angle
     ↓
Different direction
     ↓
Low similarity

a · b = ax bx + ay by = ∥a∥∥b∥ cos θ

4(-1) + 1(3) = -1

Obtuse angle: negative

Simple Example

Imagine:

Vector A = [1, 2]

Vector B = [2, 4]

Notice:

B = 2 × A

So they point in exactly the same direction.

Therefore:

Cosine similarity = 1

That means:

Maximum similarity in direction.

Opposite Direction

Now:

Vector A = [1, 2]

Vector B = [-1, -2]

They point in exactly opposite directions.

Therefore:

Cosine similarity = -1

So, roughly:

 1  → same direction

 0  → perpendicular / no directional similarity

-1  → opposite direction

For many modern embedding use cases, you'll mostly encounter positive similarity values, but the mathematical range is -1 to 1.

The Formula

The cosine similarity formula is:

cosine similarity(A, B) = (A · B) / (|A| |B|)

It looks complicated, but break it into three pieces:

A · B
───────
|A| |B|

A · B

This is the dot product.

|A|

This is the magnitude/length of vector A.

|B|

This is the magnitude/length of vector B.

You don't need to memorize the formula immediately. Understand what it is doing:

It compares the direction of two vectors while normalizing for their lengths.

Let's Calculate One Manually

Take:

A = [1, 2]

B = [2, 1]

Step 1 — Dot product

Multiply corresponding values:

1 × 2 = 2
2 × 1 = 2

Add them:

2 + 2 = 4

So:

A · B = 4

Step 2 — Magnitude of A

|A| = √(1² + 2²)
   = √5

Step 3 — Magnitude of B

|B| = √(2² + 1²)
   = √5

Step 4 — Cosine similarity

4
────────────
√5 × √5

Since:

√5 × √5 = 5

we get:

4 / 5 = 0.8

Cosine similarity = 0.8

That's a fairly strong directional similarity.

Python Implementation

You can calculate it yourself with Python.

</> Python

import math


def cosine_similarity(a, b):

    dot_product = sum(
        x * y for x, y in zip(a, b)
    )

    magnitude_a = math.sqrt(
        sum(x * x for x in a)
    )

    magnitude_b = math.sqrt(
        sum(x * x for x in b)
    )

    return dot_product / (
        magnitude_a * magnitude_b
    )


a = [1, 2]
b = [2, 1]

score = cosine_similarity(a, b)

print(score)

Output:

0.8

Why Is This Useful in Generative AI?

Now connect it to embeddings.

"I love Python"
       ↓
Vector A

"I enjoy Python programming"
       ↓
Vector B

We calculate:

Vector A
    ↓
Cosine Similarity
    ↑
Vector B

Suppose the score is:

0.90

We can say:

These two pieces of text have a high semantic similarity according to this embedding representation.

Now compare:

"I love Python"

"The weather is rainy today."

Maybe the similarity is much lower.

The exact score depends on the embedding model and the content.