Splitting Data Correctly
Before training a machine learning model, we need to separate our dataset into different parts. This allows us to train the model on one part and check how well it works on data it has not seen before.
Split your data so the model learns from one part and is tested on another.
The most important rule is simple: the model should not get access to the test data while it is learning.
Why Do We Split the Data?
Imagine you want to test a student before an exam. If you give the student the exact exam questions while studying, the final score will not tell you whether the student actually understands the subject.
Machine learning works in a similar way.
The model learns from training data. We then use separate data to check whether it can make good predictions on information it has not seen before.
Training Data and Test Data
Suppose we have 1,000 customer records.
Used to learn
Used to evaluate
A common starting point is an 80/20 split:
80/20 is not a strict rule. The appropriate split depends on the size and nature of the dataset.
The Important Rule: Keep Test Data Separate
The test data should represent data that the model has not seen during training.
This gives us a more honest estimate of how the model may perform on new data.
Why Random Splitting Is Common
Suppose our dataset is ordered like this:
If we simply take the first 80% for training and the last 20% for testing, we might accidentally create groups that are very different from each other.
For many ordinary datasets, randomly splitting the records gives a better mixture of examples in the training and test sets.
However, random splitting is not appropriate for every problem. Time-series data, for example, often needs to respect chronological order.
A Common Mistake: Data Leakage
One of the biggest problems when splitting data is data leakage.
Data leakage happens when information from data that should be unseen accidentally influences the training process.
Then split it into training and test data.
Fit preprocessing using training data, then transform the other data.
For example, if you are using feature scaling, do not calculate the scaling parameters using the entire dataset before the split.
The test set should remain unseen during the learning and preprocessing process.
Train, Validation, and Test
For simple projects, you may use only training and test data.
When developing and tuning a model, however, we often use three separate parts:
Used to train the model.
Used to compare settings and improve the model.
Used for final evaluation.
You will study the detailed purpose of these three sets in the earlier Lesson 4 topic, Training, Validation, and Test Data.
Splitting Data With Python
Scikit-learn provides a simple function called train_test_split().
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.2,
random_state=42
)Here is what the important parts mean:
So if we have 1,000 records, approximately 800 will be used for training and 200 for testing.
A Complete Example
Imagine we want to predict whether a customer will buy a product.
The input features are:
The label is:
We split the complete dataset before training the model.
Learn patterns
Check predictions
Never Let the Model Learn From the Test Set.
Split the data before training. Use the training data to learn, and keep the test data separate for evaluating how well the model works on unseen examples.
Which Approach Is Correct?
The correct basic workflow is to split the dataset, train the model using the training data, and evaluate it using separate test data. The test data should not influence the training process.