Missing Values
Missing values are pieces of information that are not available in a dataset. Before training a machine learning model, we need to decide how to handle them.
A missing value means the data for something is not available.
For example, a customer dataset may contain the customer's age and city, but the income value may be missing.
What Does a Missing Value Look Like?
Imagine we have information about customers.
Priya's income is missing. We know the customer exists, but we do not know the value of the Income feature.
Why Do Values Go Missing?
Missing information can happen for many practical reasons.
Missing data is common in real-world datasets. It does not automatically mean the entire dataset is bad.
Why Can Missing Values Be a Problem?
Machine learning models need usable input data to learn patterns.
Suppose we want to predict house prices using:
2,000 sq ft
Missing
8 years
If Bedrooms is required as an input feature, we need to decide what to do with the missing value before training the model.
One Common Solution: Remove the Row
If only a small number of rows have missing values, one possible approach is to remove those rows.
For example:
Age: 32
Income: Missing
Model trains on the remaining data.
This can be reasonable when only a small amount of data is missing.
But blindly deleting rows is a bad idea if a large percentage of the dataset is missing.
Another Solution: Fill the Missing Value
Instead of removing the row, we can sometimes replace the missing value with a reasonable value.
For example, suppose we have these customer ages:
One simple approach is to use a summary value such as the mean or median, depending on the situation.
The exact method depends on the type of data and the problem. We should not automatically fill every missing value with the same number.
Remove or Fill?
There is no single rule that works for every dataset.
Removing a few affected rows may be acceptable.
Filling values may allow us to keep more training examples.
The choice should depend on how much data is missing, which feature is affected, and why the value is missing.
A Simple Real-World Example
Imagine an e-commerce dataset:
Before training the model, we need to decide how to handle the missing Income value.
We could remove that row if appropriate, or use a suitable method to fill the missing value.
Missing Data Must Be Handled Before Training.
Missing values are normal in real-world datasets. The important part is to identify them and choose an appropriate way to handle them instead of ignoring them.
What Would You Do?
A dataset contains 10,000 customer records, but only 20 records have a missing Age value.
Because only 20 out of 10,000 records are affected, removing those rows could be reasonable. Depending on the dataset and the reason the values are missing, filling the missing values could also be appropriate. The correct choice depends on the situation.