Decision Tree Visualizer

Explore decision tree splits, Gini impurity and overfitting interactively. See how feature thresholds divide data and build a classifier.

1A game of 20 questions

Each question checks one thing and has a yes or no answer. Follow the answers down until you reach a leaf: that's the prediction.

2Each question cuts the plot in two

A question looks at one feature, so each cut is straight across or straight up. Together the cuts carve the space into boxes, one answer per box.

3Pick the question that sorts best

Gini impurity measures how mixed a group is: 0 when every point is the same class, 0.5 for a 50/50 mix of two. The best question leaves both sides as unmixed as possible.

Watch it learn

Build a tree, one question at a time

Each point has two numbers, x and y, and a colour (its class). The tree learns questions like “Is x ≤ 4.2?” that sort the colours apart. Step through it: the amber line is the question being tried, and the shaded boxes are the leaves' answers. When it's finished, click anywhere on the plot to send a new point down the tree.

Numbers under each box: how many training points of each colour reach it.

Questions: 0
Leaves: 0
Training accuracy: 100.0%
On 400 new points: 97.0%

Accuracy is for the finished tree at this depth.

Data

Max depth

Measure of mixed-up-ness

Three clusters. A few straight cuts separate them cleanly.

Step 0 / 9

Speed900 ms

Press Step to watch the first question being chosen.

The big trap

Too deep? Training accuracy vs new points

A tree with no limit keeps asking questions until every training point is in its own pure box. That looks perfect on the training data, but on noisy data it's memorising mistakes. The chart trains a tree at every depth from 1 to 12 on the current data and tests each one on 400 new points it never saw. Try it on “Noisy”.

training accuracy accuracy on 400 new points (dot = best depth)

The fixes all stop the tree before it memorises: a maximum depth, a minimum number of points per leaf, or growing it fully and then pruning back branches that don't earn their keep (scikit-learn calls this cost-complexity pruning).

No magic, just counting

The maths, step by step

Take ten points on a line, at x = 1 to 10, five teal (T) and five violet (V). The tree wants one question “Is x ≤ t?” that splits them into the purest two groups. Every number below is calculated by the same code that builds the tree above.

V10

x →

1
How mixed is a group? Gini impurity
Gini impurity adds up p × (1 − p) for every class, where p is that class's share of the group. It's also the chance that two points picked at random (with replacement) are different colours. 0 means pure; 0.5 is the most mixed two classes can be.
Gini = Σ p · (1 − p)
All 10 points (p = 0.5 each) = 0.5 × 0.5 + 0.5 × 0.5 = 0.5000
A pure group = 1 × 0 + 0 × 1 = 0.0000
2
The other measure: entropy
Entropy comes from information theory: how many yes/no questions, on average, you'd need to guess a point's colour. It's 0 for a pure group and 1 bit for a 50/50 split. It usually picks the same questions as Gini.
entropy = −Σ p · log₂(p)
All 10 points = −(0.5·log₂0.5 + 0.5·log₂0.5) = 1.0000

Score every possible question

Cut halfway between each pair of neighbours. For each cut, work out each side's Gini, then average them weighted by how many points each side has. Lower is better.

Question	Yes side (T / V)	No side (T / V)	Weighted Gini	Gain
x ≤ 1.5?	1 / 0	4 / 5	0.4444	0.0556
x ≤ 2.5?	2 / 0	3 / 5	0.375	0.125
x ≤ 3.5?	3 / 0	2 / 5	0.2857	0.2143
x ≤ 4.5?	3 / 1	2 / 4	0.4167	0.0833
x ≤ 5.5?	4 / 1	1 / 4	0.32	0.18
x ≤ 6.5?	5 / 1	0 / 4	0.1667	0.3333
x ≤ 7.5?	5 / 2	0 / 3	0.2857	0.2143
x ≤ 8.5?	5 / 3	0 / 2	0.375	0.125
x ≤ 9.5?	5 / 4	0 / 1	0.4444	0.0556

Gain = the parent's Gini (0.5) minus the weighted Gini. The best question has the biggest gain.

4
Check the winner by hand: x ≤ 6.5
Here is the best row worked out in full.
Yes side (5 T, 1 V) = (5/6)·(1/6) + (1/6)·(5/6) = 0.2778
No side (0 T, 4 V) = pure = 0.0000
Weighted = (6/10)·0.2778 + (4/10)·0 = 0.1667
Gain = 0.5 − 0.1667 = 0.3333
With entropy instead, the same question wins too: its gain is 0.61 bits.
5
Repeat inside each side
The no side (x > 6.5) is already pure, so it becomes a leaf that predicts violet. The yes side still holds one violet point, so the tree asks another question there, the same way. That's the whole algorithm: score every question, keep the best, repeat.
For two features, the tree simply scores every question on x and every question on y, and keeps the best of all of them. That's what the amber line shows above.

In practice

Pros and cons

Good at

Easy to understand and explain: you can read the questions (a “white box” model).
Little data preparation: no need to scale features.
Fast to use: a new point only answers one question per level.
Works for more than two classes, and for numbers too (regression trees).

Watch out for

Overfitting, if it's allowed to grow too deep.
Instability: a small change in the data can give a completely different tree.
Only straight cuts along one feature, so diagonal borders become staircases.
XOR-like patterns are hard, because the greedy first question can't see that two questions together would work.
It's biased towards classes that have more examples, so balance the data if one class dominates.

Using many trees together (random forests and boosted trees) fixes much of the instability and overfitting. These lists follow scikit-learn's documentation on decision trees.

How it works: Decision Tree

Step by step

Class: teal
Class: violet
The question being tried

Press Start or Step: the step the animation is on lights up below.

1
Start with every point in one group
The root holds all the training data. Measure how mixed it is.
node = all points
2
Try every question, keep the best
For each feature and each cut-off between neighbouring values, score the two sides by their size-weighted impurity. The lowest score wins.
best = min over (feature, t) of weighted impurity
3
Split into yes and no
Points that answer yes go left, the rest go right. Each side becomes a new, smaller group.
left = x[f] ≤ t; right = x[f] > t
4
Or stop: make a leaf
Stop when the group is pure, the depth limit is hit, or no question helps. The leaf predicts its most common class.
leaf.predict = majority class
5
Repeat inside every branch
The same steps run inside each side until every branch ends in a leaf. Then a new point just answers the questions from the top.
grow(left); grow(right)

A decision tree is a game of 20 questions. Each question looks at one feature and asks yes or no, such as “Is the weight 150 g or less?”. Each answer sends you down a branch to the next question, until you reach a leaf that gives the answer.

To grow one, the computer tries every possible question at every possible cut-off and keeps the one that sorts the points best: each side should be as close as possible to a single class. It measures “mixed-up-ness” with Gini impurity or entropy. Then it does the same again inside each side.

Every question cuts straight across one feature, so on a plot the tree carves the space into rectangles. That makes trees easy to read and explain, but it also means they can keep cutting until every training point has its own box, which is memorising rather than learning.

Where you'll meet it

When you need to explain a decision: loan approvals, medical triage, any rule a person must be able to follow.
Mixed, messy tabular data: trees need little preparation (no scaling).
As the building block of random forests and boosted trees, which are among the strongest models for tabular data.

time about O(features × n log n) per levelspace O(nodes)

Each level sorts the points on every feature to try every cut-off. Asking a new point is fast: one question per level, so roughly log(n) questions for a balanced tree.