Distance Between Data Points
KNN needs a way to decide which data points are close and which are far away. It does this by measuring the distance between data points.
Smaller distance means the data points are closer.
KNN compares a new data point with known data points. The points with the smallest distances become its nearest neighbors.
Why Does KNN Need Distance?
Remember what KNN does:
A new example needs a prediction.
Find the closest known examples.
Use the neighbors to make a prediction.
But KNN cannot simply say "this point looks close." It needs a mathematical way to measure closeness.
Think of a Map
Imagine two people standing on a map. If they are standing close together, the distance between them is small.
If they are far apart, the distance is large.
Location A
Location B
KNN uses the same basic idea with data. Instead of physical locations, it compares the values of the features.
Data Points Have Features
Suppose we want to predict whether a student will Pass or Fail.
We use two features:
How many hours the student studies.
The student's attendance percentage.
A student can therefore be represented by two numbers.
5 study hours and 88% attendance.
A Simple Distance Example
Suppose our new student is:
Now compare this student with two known students.
Student A is clearly more similar to the new student than Student B.
Their feature values are very similar.
Their feature values are much more different.
Distance in Two Dimensions
With two features, we can think of each data point as a position on a simple graph.
The new point is closer to point A than point B. Therefore, point A is a better candidate for being one of the nearest neighbors.
Euclidean Distance
One common way to measure distance is called Euclidean distance.
It is basically the straight-line distance between two points.
Don't worry about memorizing the formula yet. The important idea is what each part means:
For example, study hours.
For example, attendance.
How far apart the two points are.
Calculate a Simple Distance
Let's use a very simple example so the calculation is easy to follow.
Apply the Euclidean distance formula:
= 3² = 9
= 4² = 16
= 25
= 5
So the Euclidean distance between (2, 3) and (5, 7) is 5.
Smaller Distance vs Larger Distance
Suppose our new point has three possible neighbors.
Therefore, Point A is the nearest of these three points.
How Distance Helps KNN
Now connect distance back to the KNN process.
KNN receives a new data point.
Compare it with known points.
Smallest distances come first.
Select the K closest points.
Complete Example
Let's put everything together using a new student.
5 study hours, 88% attendance.
Find how close each student is.
The smallest distances are the nearest points.
Select the three closest students.
After finding the three nearest students, KNN can look at their labels and perform the majority vote.
Why Feature Scaling Matters
There is an important problem when features use very different scales.
Usually around 18–80.
Could be 20,000–200,000.
Income has much larger numerical values than age. If we calculate distance directly, the larger-scale feature can dominate the calculation.
You already learned about Feature Scaling in Lesson 6 — Data Preparation. Here we are seeing why it matters specifically for KNN.
The Big Picture
Distance is not the final prediction. Distance is used to find the neighbors.
KNN uses distance to find its nearest neighbors.
A smaller distance means two data points are closer. KNN calculates distances, finds the closest points, selects K of them, and then uses their labels to make the prediction.