A Simple Dataset Split
Let's take one dataset and divide it into training, validation, and test data so we can see exactly how the three parts work together.
We start with one dataset and separate it into different groups with different responsibilities.
The samples remain examples from the same overall problem, but each group is used for a different stage of the Machine Learning workflow.
Start With One Dataset
Imagine we have a dataset containing 10,000 house records.
Each record contains information such as:
- House size.
- Number of bedrooms.
- Number of bathrooms.
- House age.
- Location.
- Selling price.
All available examples before splitting.
Decide the Split
For this example, we will use a simple 80 / 10 / 10 split.
With 10,000 samples, that gives us:
samples
samples
samples
The numbers add up to the original dataset:
The Dataset Is Now Divided
After splitting, the three groups have different roles.
Used to teach the model.
Used to compare and improve the approach.
Reserved for final evaluation.
What Happens to the Training Data?
The 8,000 training samples are given to the Machine Learning algorithm.
The model uses these examples to learn patterns between the input features and known outcomes.
For example, the model may learn relationships between house size and selling price.
What Happens to the Validation Data?
The 1,000 validation samples are kept separate from the training process.
We use these examples to compare models, settings, or approaches while developing the solution.
If one approach performs better on the validation data, we may choose that approach.
What Happens to the Test Data?
The 1,000 test samples are kept aside for the final evaluation.
After the development process is finished, the final model is evaluated using these examples.
Which Rows Go Where?
A simple way to think about the dataset is to number the samples.
Used for learning.
Used during development.
Used for final evaluation.
In practice, simply taking the first 8,000 rows, then the next 1,000, and so on can create problems if the original data is ordered or grouped.
Why Random Splitting Is Often Used
If the dataset is suitable for random splitting, we can randomly assign samples to the different groups.
This helps avoid accidentally putting one type of example entirely into one part of the dataset.
The actual assignment is handled by the splitting strategy rather than manually choosing individual rows.
Example With Python
Python libraries can perform these splits for us.
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.2,
random_state=42
)This creates an 80% training portion and a 20% test portion.
If we also need a validation set, we can split the remaining data again.
X_train, X_temp, y_train, y_temp = train_test_split(
X,
y,
test_size=0.2,
random_state=42
)
X_validation, X_test, y_validation, y_test = train_test_split(
X_temp,
y_temp,
test_size=0.5,
random_state=42
)The result is approximately:
- 80% training.
- 10% validation.
- 10% test.
What Does random_state Do?
You may have noticed this:
random_state=42It makes the random split reproducible.
Without a fixed random state, running the same split again can produce different assignments of samples.
Run the script today and Sample 25 goes into the training set.
Run the script again without a fixed random state and Sample 25 may end up in another group.
A fixed random state makes experiments easier to reproduce.
Random Splitting Is Not Always Correct
This is an important limitation.
Randomly splitting every dataset is not automatically the right solution.
For forecasting future sales, using future records to train while testing on earlier records can leak information.
If multiple records belong to the same customer or patient, random splitting can place related records in different sets and make evaluation overly optimistic.
The split strategy must match the structure of the real-world problem.
The Complete Picture
Learn
Choose
Evaluate
Splitting Data Is About Protecting the Evaluation
The goal is not to follow one magic percentage. The goal is to create useful separation between learning, development decisions, and final evaluation.
You Have 20,000 Samples
You decide to use an 80 / 10 / 10 split.
How many samples should go into each group?
Training: 16,000 samples.
Validation: 2,000 samples.
Test: 2,000 samples.