Evaluating Predictions
A machine learning model can make predictions, but a prediction is not automatically correct. We need to compare the model's predictions with the actual results to understand how well the model is performing.
A prediction is useful only if we know how well it performs.
Evaluation means checking the model's predictions against the correct or actual results. This helps us understand whether the model is making useful predictions or needs improvement.
Prediction vs Evaluation
These are two different steps in machine learning.
The trained model receives new data and produces an output.
We compare the prediction with the actual result to measure performance.
Simple House Price Example
Suppose a model predicts the price of a house.
The model predicts: $350,000
Later, we discover that the actual selling price was: $370,000
The prediction was not exactly correct.
The difference between the predicted value and the actual value gives us information about the model's performance.
Prediction tells us the model's answer. Evaluation tells us how close or useful that answer was.
One Prediction Is Not Enough
We should not judge a model based on just one prediction.
Imagine a model makes these predictions:
| House | Predicted | Actual |
|---|---|---|
| House 1 | $300,000 | $310,000 |
| House 2 | $400,000 | $395,000 |
| House 3 | $250,000 | $320,000 |
| House 4 | $500,000 | $490,000 |
Now we can look at several predictions together and get a better idea of how the model performs overall.
This is why machine learning evaluation normally uses multiple examples rather than relying on one prediction.
What Are We Looking For?
When evaluating a model, we want to understand whether its predictions are close to the correct results and whether it performs consistently on data it has not learned from.
Classification Example
Evaluation also works when the model predicts categories instead of numbers.
Suppose a model predicts whether emails are Spam or Not Spam.
By comparing many predictions with their actual labels, we can understand how well the classifier is working.
Later, we will learn specific metrics such as accuracy, precision, recall, and F1 score.
Why Can't We Evaluate Using Training Data?
This is an important idea.
A model has already seen the training examples while learning. If we only test it on those same examples, the result may give us a misleading picture of how well it works on new data.
The model learned from these examples during training.
This gives us a better idea of how the model performs on unseen examples.
This is one reason machine learning workflows separate training data from data used for evaluation.
Evaluation Helps Us Improve the Model
Suppose we evaluate a model and discover that its predictions are poor.
That tells us something important: the current model or the current data preparation may not be good enough.
We can then make changes and train again.
This creates an iterative machine learning workflow.
A Complete Simple Example
Let's put everything together using house-price prediction.
If the predictions are good, the model may be useful.
If the predictions are poor, we need to investigate why and improve the model or the data.
Evaluation Tells Us How Well the Model Is Performing.
We compare the model's predictions with actual results and use multiple examples to understand whether the model is accurate, reliable, and useful on new data.
Which Model Is Better?
Two house-price models make predictions for the same group of new houses.
Model A usually predicts prices very close to the actual selling prices.
Model B often predicts prices that are far away from the actual selling prices.
Model A is performing better because its predictions are closer to the actual results.
In real machine learning, we use specific evaluation metrics to measure this difference more precisely.