Module 13: Advanced Real-World Projects & CapstoneBy Ashok Kumar Yadav4 min read

Project: End-to-End Customer Churn Prediction

Churn prediction is a classic business problem and a strong portfolio project because it covers the full path from messy data to a deployed service.

Customer churn project flow from raw data to trained pipeline and Dockerised prediction API

Quick answer: Split data first, build a scikit-learn pipeline with imputation, scaling and one-hot encoding, compare models using cross-validated ROC AUC, tune the decision threshold for business cost, save the whole pipeline with joblib and serve it with FastAPI in Docker.

Project workflow

Define the target (churned in the next 30 days), choose the metric (recall and precision at a business-driven threshold, plus ROC AUC), split train and test, then build features only from information available before the churn date.

Build the preprocessing and model pipeline

Putting preprocessing inside the pipeline prevents leakage during cross-validation and keeps serving identical to training. The Telco churn dataset from Kaggle or IBM works well for practice.

Compare two models with cross-validation

import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

df = pd.read_csv("churn.csv")                       # target column: Churn (Yes/No)
y = (df.pop("Churn") == "Yes").astype(int)
num = df.select_dtypes("number").columns.tolist()
cat = df.select_dtypes("object").columns.tolist()
Xtr, Xte, ytr, yte = train_test_split(df, y, test_size=0.2, stratify=y, random_state=42)

pre = ColumnTransformer([
    ("num", Pipeline([("imp", SimpleImputer(strategy="median")), ("sc", StandardScaler())]), num),
    ("cat", OneHotEncoder(handle_unknown="ignore"), cat)])

for name, clf in {"logreg": LogisticRegression(max_iter=1000),
                  "hgb": HistGradientBoostingClassifier(random_state=0)}.items():
    pipe = Pipeline([("pre", pre), ("clf", clf)])
    auc = cross_val_score(pipe, Xtr, ytr, cv=5, scoring="roc_auc").mean()
    print(f"{name:7s} CV AUC = {auc:.3f}")

Tune the decision threshold

The default 0.5 cutoff rarely matches business cost. If a retention offer costs 10 and a lost customer costs 100, flag more customers.

Choose a threshold with an F2 score

import numpy as np
from sklearn.metrics import precision_recall_curve

best = Pipeline([("pre", pre), ("clf", LogisticRegression(max_iter=1000))]).fit(Xtr, ytr)
proba = best.predict_proba(Xte)[:, 1]
prec, rec, thr = precision_recall_curve(yte, proba)
f2 = 5 * prec * rec / np.maximum(4 * prec + rec, 1e-9)       # favours recall
t = thr[np.argmax(f2[:-1])]
print("chosen threshold:", round(float(t), 3))

Save the pipeline and serve it

Save the single fitted pipeline object, then reuse the FastAPI and Docker pattern from the deployment lesson.

Persist model and threshold together

import joblib
joblib.dump({"pipeline": best, "threshold": float(t), "columns": list(df.columns)}, "churn.joblib")

# app.py (FastAPI)
# from fastapi import FastAPI
# import joblib, pandas as pd
# art = joblib.load("churn.joblib"); app = FastAPI()
# @app.post("/predict")
# def predict(record: dict):
#     row = pd.DataFrame([record])[art["columns"]]
#     p = float(art["pipeline"].predict_proba(row)[0, 1])
#     return {"churn_probability": round(p, 4), "will_churn": p >= art["threshold"]}

Make it portfolio-ready

Add a README with the business problem, metric table, confusion matrix and a short note on limitations. Include a Dockerfile, a requirements file with pinned versions and a few automated tests for the API.

Hands-on practice

Run the examples in the downloadable Jupyter notebook. The extra checks below focus on mistakes that appear in real projects, not just syntax.

Use a stratified split

Practice example

from sklearn.model_selection import train_test_split
X=list(range(100)); y=[0]*80+[1]*20
Xtr,Xte,ytr,yte=train_test_split(X,y,test_size=.2,stratify=y,random_state=42)
print(sum(ytr),sum(yte))

16 4

Stratification preserves class ratios in small imbalanced datasets.

Tune the probability threshold

Practice example

import numpy as np
p=np.array([.2,.4,.6,.8]); y=np.array([0,1,1,1])
for t in [.3,.5,.7]:
    pred=(p>=t).astype(int); print(t,(pred==y).mean())

0.3 0.75
0.5 0.75
0.7 0.50

Choose thresholds from business cost and validation metrics, not 0.5 by habit.

Self-check

Can you explain why the first practice result is trustworthy rather than just plausible?
What would you monitor or test before using this Churn Project workflow in production?
Which assumption in the example would break first on a larger or messier dataset?

Frequently asked questions

Which model is best for churn prediction?

Gradient boosting often leads on tabular data, but logistic regression is a strong, explainable baseline. Compare both.

How do I handle class imbalance?

Use stratified splits, class weights or resampling inside cross-validation, and evaluate with precision-recall metrics rather than accuracy.

Why save the threshold with the model?

The decision cutoff is part of the model's behaviour. Saving it keeps the API consistent with your evaluation.

Primary references

Use these official sources when exact syntax, defaults or definitions matter beyond the examples here.

Written by Ashok Kumar Yadav
Ashok Kumar Yadav writes Xenors tutorials on applied AI, programming and data workflows. Every lesson is built around code you can run and check yourself.
GitHub · LinkedIn