Putting the Data Split Together
We have seen training data, validation data, and test data separately. Now let's connect them into one complete Machine Learning workflow.
One dataset can have three different jobs: learn, choose, and evaluate.
Training data teaches the model. Validation data helps us make development decisions. Test data gives us the final evaluation.
Start With the Original Dataset
Imagine we have a dataset containing 10,000 house records.
Each record contains features such as house size, bedrooms, location, age, and the actual selling price.
These are all the examples available for the Machine Learning problem.
Split the Dataset
For this example, we will use an 80 / 10 / 10 split.
Learn
Choose
Evaluate
Step 1 — Train the Model
The first step is to give the training data to the Machine Learning algorithm.
The model learns patterns from these examples.
For example, a house-price model may learn how different features relate to the selling price.
Step 2 — Validate the Model
After training, we can evaluate the model using the validation data.
Use the results to compare approaches and make development decisions.
For example, suppose we try two different models.
Validation accuracy
Validation accuracy
Based on this validation result, we may choose Model B for the next stage.
Step 3 — Final Test
Once we have finished development and selected our final approach, we evaluate it using the test data.
Used for the final evaluation of the selected model.
For example, the final model might achieve 89% accuracy on the test set.
The Complete Workflow
10,000 samples
80 / 10 / 10
Learn patterns
Choose approach
Final evaluation
What Happens to the Test Data During Development?
Ideally, nothing.
The test data should stay separate while you are comparing models, changing settings, and improving your approach.
If you repeatedly change your model based on test results, the test set is no longer acting as a clean final evaluation.
A Bad Workflow
This is a problem because the test results are influencing development decisions.
A Better Workflow
Development decisions happen before the final test.
Do We Always Need Three Splits?
No.
A separate validation set is useful when you need to make repeated development decisions, but simpler projects may use only training and test data.
Train the model using one portion and reserve another portion for evaluation.
Useful when models, parameters, or approaches need to be compared during development.
One More Important Point
The split percentage is not the main idea.
An 80 / 10 / 10 split is only an example. Depending on the size, structure, and type of data, different strategies may be more appropriate.
A very large dataset may have enough examples for relatively small validation and test portions.
A small dataset may require a different evaluation strategy instead of simply removing a large percentage for a fixed test set.
Learn → Choose → Evaluate
Training data is used to learn
patterns.
Validation data is used during
development to compare and improve approaches.
Test data is reserved for the final
evaluation.
Final Question
You trained several models and used validation data to select the best approach.
You have not yet evaluated the selected model on the test set.
What should you do next?
Evaluate the selected final model on the test dataset.
The test result should provide the final estimate of performance on the held-out data.
You Now Understand the Data Split
You know why datasets are split, what training, validation, and test data are used for, and how they fit together in a Machine Learning workflow.