Learn practical feature engineering: one-hot encoding, scaling, transformations, date features, interactions and leakage avoidance. Work through the feature preparation examples in order, then change one assumption and explain whether your interpretation still holds. This page targets the practical intent behind feature preparation, not repetition of the phrase itself.
Editorial note: Publication metadata for Feature Engineering Techniques is intentionally unset until this URL is actually released; the lesson already includes its own examples, CSV and notebook.
Frame the prediction problem before choosing a model
For customer churn, create account age, days since last purchase and support tickets per month, but calculate every training-time aggregate using only information available before the prediction date.
- Feature engineering converts raw fields into representations that make a pattern easier for a model to learn.
- One-hot encoding is appropriate for unordered categories, while ordinal encoding assumes an order.
- Scaling is important for distance-based and gradient-based models but usually unnecessary for tree splits.
- Transformations must be fit on training data only to avoid leaking information from validation or test sets.
Build a baseline and a leakage-safe evaluation
For feature preparation, predict one visible result before each snippet runs—a row count, shape, statistic, ranking or error pattern—then compare it with the printed checkpoint.
# feature preparation · example 1
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder, StandardScaler
pre = ColumnTransformer([
('num', StandardScaler(), numeric_cols),
('cat', OneHotEncoder(handle_unknown='ignore'), categorical_cols)
])feature preparation checkpoint 1: Run the cell and inspect the printed values before changing anything.
# feature preparation · example 2
import pandas as pd
from sklearn.model_selection import train_test_split
df = pd.read_csv('practice.csv')
target = df.select_dtypes('number').columns[-1]
X = df.drop(columns=[target]).select_dtypes('number')
y = df[target]
X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=.25,random_state=42)
print(X_train.shape, X_test.shape)feature preparation checkpoint 2: Training and test shapes are printed so the split is explicit.
# feature preparation · example 3
from sklearn.dummy import DummyRegressor
from sklearn.metrics import mean_absolute_error
baseline=DummyRegressor(strategy='mean').fit(X_train,y_train)
pred=baseline.predict(X_test)
print('baseline MAE=', round(mean_absolute_error(y_test,pred),3))feature preparation checkpoint 3: The baseline score becomes the minimum standard a more complex model should beat.
# feature preparation · example 4
from sklearn.linear_model import LinearRegression
model=LinearRegression().fit(X_train,y_train)
pred=model.predict(X_test)
print('model MAE=', round(mean_absolute_error(y_test,pred),3))feature preparation checkpoint 4: Compare model MAE with the baseline instead of judging the model in isolation.
# feature preparation · example 5
errors=(y_test-pred).rename('residual')
print(errors.describe().round(3))
print('largest absolute errors:')
print(errors.abs().sort_values(ascending=False).head())feature preparation checkpoint 5: Residual statistics and the largest errors reveal cases worth investigating.
Inspect errors instead of chasing one score
Scaling is important for distance-based and gradient-based models but usually unnecessary for tree splits.
Transformations must be fit on training data only to avoid leaking information from validation or test sets.
Self-check with answers
For feature preparation, what result should you be able to explain without the code?
Feature engineering converts raw fields into representations that make a pattern easier for a model to learn.
Which assumption matters most in this feature preparation exercise?
Scaling is important for distance-based and gradient-based models but usually unnecessary for tree splits.
What failure mode would make this feature preparation result untrustworthy?
Transformations must be fit on training data only to avoid leaking information from validation or test sets.
Repeat the experiment with a documented change
Try it yourself workspace
Edit this feature preparation starter, copy it, and run it in the downloadable notebook or your local Jupyter environment.
Questions specific to this lesson
What should I establish before tuning feature preparation?
For feature preparation, begin with a leakage-safe split and a simple baseline so later tuning has a credible reference point.
How do I check overfitting in feature preparation?
For feature preparation, compare training and validation behavior, inspect held-out errors, and repeat the evaluation with a stable split or cross-validation when appropriate.
Which metric should I report for feature preparation?
Transformations must be fit on training data only to avoid leaking information from validation or test sets.
How should I explain feature preparation in a portfolio or interview?
When presenting feature preparation, show the problem definition, baseline, split strategy, metric, one error-analysis finding and one limitation.
Primary references and further reading
For feature preparation, use these primary references when exact syntax, defaults or formal definitions matter beyond the examples on this page.
