Splitting Data
A Decision Tree works by splitting data into smaller groups using questions. Each split tries to make the groups more useful for making a prediction.
A split divides one large group of data into smaller groups.
The Decision Tree asks a question about a feature. The answer determines which group each data point belongs to.
What Does "Splitting Data" Mean?
Imagine that we have a group of students and want to predict whether they will Pass or Fail.
At the beginning, all students are mixed together.
The tree needs a way to separate these examples. It can ask a question such as:
One Question Creates Two Groups
Suppose the question is:
The students are now separated into two groups.
Students who studied 3 hours or less.
Students who studied more than 3 hours.
This is what we mean by a split.
Why Does the Tree Split the Data?
The goal is not simply to divide the data randomly.
The tree wants to create groups that are easier to classify.
For example, imagine these two groups:
These groups are easier to work with than one large mixed group containing many Pass and Fail examples.
Example: Study Hours
Let's use a small dataset.
A natural split is:
Study Hours > 3?Fail, Fail, Fail
Pass, Pass, Pass
This is an excellent split for this small dataset because each resulting group contains only one class.
Not Every Split Is Good
The tree could choose many different questions. But not every question creates useful groups.
For example, suppose we split using:
Study Hours > 1?That would create:
Fail
Fail, Fail, Pass, Pass, Pass
The second group is still mixed. So this split is less useful than:
Study Hours > 3?The Tree Can Split Again
Sometimes one split is not enough.
After the first split, the tree can take one of the resulting groups and split it again.
This process can continue as the tree grows.
How Does the Tree Choose a Split?
During training, the Decision Tree considers possible splits in the available features.
It compares the resulting groups and looks for a split that separates the target classes well.
Example: Study Hours.
Example: Study Hours > 3.
Are the groups more organized?
Use the split that improves separation.
There are mathematical methods for measuring how good a split is. We will cover those ideas only when they become necessary.
A Real-World Example
Imagine a bank wants to classify transactions as Normal or Suspicious.
Mostly normal transactions.
More transactions may need additional checking.
The tree could then split the high-value group again using another feature, such as location or transaction frequency.
Splitting Is How the Tree Grows
Every time the tree makes a useful split, it creates new branches.
This repeated splitting is what creates the tree structure we studied in the previous pages.
Splitting Data in One Picture
Splitting data means dividing a group into smaller groups using a useful question.
A Decision Tree repeatedly searches for useful splits that make the resulting groups easier to classify.