What Is KNN?
K-Nearest Neighbors (KNN) is a Machine Learning algorithm that makes a prediction by looking at the closest known examples.
Look at the closest examples, then use them to make a prediction.
Instead of learning a complicated set of rules, KNN asks a simple question: "Which known examples are closest to this new example?"
What Does KNN Mean?
KNN stands for K-Nearest Neighbors.
How many nearby examples should we look at?
We look for examples that are closest to the new example.
The known data points around the new example.
A Simple Real-Life Example
Imagine you move to a new neighborhood and want to know whether a nearby restaurant is likely to be expensive or affordable.
You look at restaurants near it and notice:
If most of the nearby restaurants are affordable, you might reasonably guess that the new restaurant is also affordable.
KNN Does Something Similar
KNN applies this same idea to data.
Suppose we already know the classes of several data points.
Class A
Class B
Unknown class
KNN looks at the data points closest to the new point.
What Is a Neighbor?
In KNN, a neighbor is simply a known data point that is close to the new data point.
We want to classify this point.
These are its nearest neighbors.
"Near" means that the data points have similar feature values according to the distance measure being used.
What Does K Control?
The letter K tells KNN how many neighbors it should consider.
The closest example has the strongest influence.
The three closest examples can vote on the prediction.
The five closest examples can vote on the prediction.
A Simple KNN Prediction
Suppose we choose:
The three closest known examples to our new example are:
Now count the votes:
Because Pass received the majority of votes, KNN predicts Pass.
KNN Classification
KNN can be used for classification problems where the prediction is a category or class.
The model already knows the classes of the training examples.
KNN finds its closest neighbors.
The neighbors vote on the class.
Student Example
Let's use the same type of student example we used with Decision Trees.
Now imagine a new student:
This new student is close to the students who studied around 5–6 hours and had high attendance.
KNN Does Not Build a Decision Tree
KNN works differently from the Decision Tree we just learned.
Example: split the data using feature values.
It uses the closest training examples when making a prediction.
This difference is important. KNN does not need to create a tree of decisions like the previous algorithm.
The Basic KNN Process
Give KNN an example it needs to classify.
Find the closest known examples.
Check the classes of those neighbors.
Use the majority class for the prediction.
KNN predicts by looking at nearby examples.
The algorithm finds the K closest known data points to a new data point and uses their labels to make the prediction.