Learn decision trees, split criteria, overfitting, random forests, feature importance and ensemble intuition. Use the dedicated Decision Trees & Random Forest dataset and notebook to reproduce the lesson rather than reading it passively. This page targets the practical intent behind tree ensembles, not repetition of the phrase itself.
Editorial note: Publication metadata for Decision Trees & Random Forest is intentionally unset until this URL is actually released; the lesson already includes its own examples, CSV and notebook.
Frame the prediction problem before choosing a model
A fraud tree may split first on transaction amount, then device risk and time of day. A random forest averages many such trees so a single unstable split has less influence.
- A decision tree recursively splits the feature space into regions that reduce impurity or prediction error.
- Deep trees can memorize training data, so depth, leaf size and pruning control variance.
- Random forests reduce variance by averaging many trees trained on resampled data with random feature subsets.
- Feature importance from trees can be useful for exploration but should not be treated as causal explanation.
Build a baseline and a leakage-safe evaluation
For Decision Trees & Random Forest, predict one visible result before each snippet runs—a row count, shape, statistic, ranking or error pattern—then compare it with the printed checkpoint.
# Decision Trees & Random Forest · example 1
from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(n_estimators=300, max_depth=8, random_state=42, class_weight='balanced')
model.fit(X_train, y_train)
print(model.score(X_test, y_test))Decision Trees & Random Forest checkpoint 1: Run the cell and inspect the printed values before changing anything.
# Decision Trees & Random Forest · example 2
import pandas as pd
from sklearn.model_selection import train_test_split
df = pd.read_csv('practice.csv')
target = df.select_dtypes('number').columns[-1]
X = df.drop(columns=[target]).select_dtypes('number')
y = df[target]
X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=.25,random_state=42)
print(X_train.shape, X_test.shape)Decision Trees & Random Forest checkpoint 2: Training and test shapes are printed so the split is explicit.
# Decision Trees & Random Forest · example 3
from sklearn.dummy import DummyRegressor
from sklearn.metrics import mean_absolute_error
baseline=DummyRegressor(strategy='mean').fit(X_train,y_train)
pred=baseline.predict(X_test)
print('baseline MAE=', round(mean_absolute_error(y_test,pred),3))Decision Trees & Random Forest checkpoint 3: The baseline score becomes the minimum standard a more complex model should beat.
# Decision Trees & Random Forest · example 4
from sklearn.linear_model import LinearRegression
model=LinearRegression().fit(X_train,y_train)
pred=model.predict(X_test)
print('model MAE=', round(mean_absolute_error(y_test,pred),3))Decision Trees & Random Forest checkpoint 4: Compare model MAE with the baseline instead of judging the model in isolation.
# Decision Trees & Random Forest · example 5
errors=(y_test-pred).rename('residual')
print(errors.describe().round(3))
print('largest absolute errors:')
print(errors.abs().sort_values(ascending=False).head())Decision Trees & Random Forest checkpoint 5: Residual statistics and the largest errors reveal cases worth investigating.
Inspect errors instead of chasing one score
Random forests reduce variance by averaging many trees trained on resampled data with random feature subsets.
Feature importance from trees can be useful for exploration but should not be treated as causal explanation.
Self-check with answers
For Decision Trees & Random Forest, what result should you be able to explain without the code?
A decision tree recursively splits the feature space into regions that reduce impurity or prediction error.
Which assumption matters most in this Decision Trees & Random Forest exercise?
Random forests reduce variance by averaging many trees trained on resampled data with random feature subsets.
What failure mode would make this Decision Trees & Random Forest result untrustworthy?
Feature importance from trees can be useful for exploration but should not be treated as causal explanation.
Repeat the experiment with a documented change
Try it yourself workspace
Edit this Decision Trees & Random Forest starter, copy it, and run it in the downloadable notebook or your local Jupyter environment.
Questions specific to this lesson
What should I establish before tuning Decision Trees & Random Forest?
For Decision Trees & Random Forest, begin with a leakage-safe split and a simple baseline so later tuning has a credible reference point.
How do I check overfitting in Decision Trees & Random Forest?
For Decision Trees & Random Forest, compare training and validation behavior, inspect held-out errors, and repeat the evaluation with a stable split or cross-validation when appropriate.
Which metric should I report for Decision Trees & Random Forest?
Feature importance from trees can be useful for exploration but should not be treated as causal explanation.
How should I explain Decision Trees & Random Forest in a portfolio or interview?
When presenting Decision Trees & Random Forest, show the problem definition, baseline, split strategy, metric, one error-analysis finding and one limitation.
Primary references and further reading
For Decision Trees & Random Forest, use these primary references when exact syntax, defaults or formal definitions matter beyond the examples on this page.
