Improving the Model
A machine learning model is not always good on the first attempt. After evaluating its predictions, we may discover that the model is making too many mistakes. We can then investigate the problem and improve the model.
Train, evaluate, improve, and repeat.
Improving a model is an iterative process. We train the model, check its predictions, identify problems, make changes, and train it again. The goal is to make the model perform better on new data.
Why Do We Need to Improve a Model?
A model may make predictions that are not accurate enough.
For example, suppose we build a house-price model. The model predicts prices that are very different from the actual prices.
A large difference like this suggests that something about the current model or the data may need attention.
A model being trained successfully does not mean that it is performing well. We improve it based on its results.
The Improvement Cycle
Model improvement is usually a cycle rather than a one-time action.
Train the model using prepared data.
Use the trained model to make predictions.
Check how well the predictions perform.
Find problems and make useful changes.
After making improvements, we train and evaluate again.
First Find the Problem
We should not change a model randomly.
First, we need to understand why the model is not performing well.
The problem could come from several places.
Finding the actual problem is more useful than simply changing things and hoping the model becomes better.
Example: Improving a House-Price Model
Imagine that our model uses only the size of a house to predict its price.
That can work to some extent, but house prices are affected by other information too.
The model has very limited information.
Bedrooms, location, age, and other useful features can provide more information.
If those additional features contain useful information, the model may be able to make better predictions.
Predicting a house price using only size is like trying to judge a house using only one piece of information. More useful information can give the model a better picture of the problem.
Better Data Can Improve the Model
Sometimes the model is not the main problem.
The training data may contain missing values, incorrect information, duplicates, or other problems.
The model learns from unreliable examples.
The model has better examples from which to learn.
This is why data preparation is an important part of machine learning.
A sophisticated model cannot magically fix fundamentally bad data.
Try a Better Model
Sometimes the current model is simply not suitable for the problem.
In that situation, we can try another machine learning algorithm and compare the results.
Predictions are often far from the actual results.
Predictions are closer to the actual results.
Use evaluation results to decide which model is more useful.
We should choose based on evidence from evaluation, not simply because one algorithm sounds more advanced.
Improvement Is Not Just "Make the Training Score Higher"
This is an important mistake beginners make.
A model can become very good at predicting its training examples while performing badly on new data.
This can cause the model to learn the training examples too closely.
The model should learn useful patterns that generalize beyond the training examples.
The real goal is not to memorize the training data. The real goal is to build a model that works well on data it has not seen before.
Complete Improvement Example
Let's put everything together.
This cycle can be repeated until the model reaches a level of performance that is good enough for the intended problem.
Improving a Model Means Learning From Its Mistakes.
We evaluate the model, understand where it is failing, make a sensible change, train again, and evaluate again. The goal is not simply to make the model better on old training examples, but to make it perform better on new data.
What Should You Do?
You built a house-price model, but its predictions are often far from the actual prices.
Should you immediately change the algorithm?
No. First investigate the problem.
Check the data, the features, the training process, and the model's evaluation results. Then make a sensible change and evaluate the new version.