Choosing K
In KNN, K tells the model how many nearby data points to consider when making a prediction. Choosing K matters because different values of K can produce different predictions.
K decides how many neighbors get a vote.
A small K looks at only a few nearby examples. A larger K considers more examples. The right choice depends on the data.
What Does K Mean?
Remember that K stands for the number of nearest neighbors KNN uses for a prediction.
The Same Data Can Give Different Results
Suppose a new student needs to be classified as Pass or Fail.
Imagine the nearest students have these results:
Now watch what happens when we change K.
Example 1 — K = 1
If we choose K = 1, KNN looks at only the closest neighbor.
There is only one vote, so that one neighbor determines the prediction.
The prediction depends heavily on the single closest example.
Example 2 — K = 3
Now choose K = 3. KNN uses the three closest neighbors.
Pass wins the majority vote, so the prediction is Pass.
Example 3 — K = 5
Now choose K = 5. KNN considers five neighbors.
This time, Fail gets the majority. The prediction changes to Fail.
Small K vs Large K
The choice of K creates a trade-off.
A small number of neighbors means individual examples can have a strong influence.
More neighbors participate, so the prediction considers a wider group of examples.
Why Can a Very Small K Be a Problem?
Suppose K = 1. The prediction depends entirely on one neighbor.
What if that neighbor is unusual or noisy?
But this point is an unusual example.
Most nearby examples suggest Pass.
With K = 1, the unusual point could determine the prediction by itself.
Why Can a Very Large K Be a Problem?
A very large K considers many neighbors. This can make the model less sensitive to the local pattern around the new point.
If K becomes too large, distant points that are less similar to the new point can influence the prediction.
So What Is a Good K?
There is no single K value that is always best.
A good K depends on the dataset and the problem. In practice, we usually try several values of K and evaluate how well the model performs.
This is why we don't simply say "K should always be 3" or "K should always be 5."
Example: Comparing K Values
Suppose we test three different K values on our validation data.
In this example, K = 5 performs best on the validation data among the values we tested.
An Important Rule
When comparing K values, don't choose the K that simply looks good on the training data.
Use the validation data to compare different K values.
This connects directly to the data-splitting concepts from Lesson 4.
Don't ask "What is the correct K?" Ask "Which K works best for this data?"
Small K values focus strongly on nearby examples, while larger K values consider a broader group. The goal is to find a value that gives good performance on unseen validation data.