{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Decision Trees & Random Forest: Ensemble Learning Guide\n",
        "Practice notebook for the Xenors lesson: https://xenors.in/data-science/decision-trees-random-forest/\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "df = pd.read_csv('../datasets/decision-trees-random-forest.csv')\n",
        "df.head()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Baseline example\n",
        "Expected checkpoint: Run the cell and inspect the printed values before changing anything.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from sklearn.ensemble import RandomForestClassifier\n",
        "model = RandomForestClassifier(n_estimators=300, max_depth=8, random_state=42, class_weight='balanced')\n",
        "model.fit(X_train, y_train)\n",
        "print(model.score(X_test, y_test))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Create a leakage-safe split\n",
        "Expected checkpoint: Training and test shapes are printed so the split is explicit.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "df = pd.read_csv('practice.csv')\n",
        "target = df.select_dtypes('number').columns[-1]\n",
        "X = df.drop(columns=[target]).select_dtypes('number')\n",
        "y = df[target]\n",
        "X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=.25,random_state=42)\n",
        "print(X_train.shape, X_test.shape)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Build a simple baseline\n",
        "Expected checkpoint: The baseline score becomes the minimum standard a more complex model should beat.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from sklearn.dummy import DummyRegressor\n",
        "from sklearn.metrics import mean_absolute_error\n",
        "\n",
        "baseline=DummyRegressor(strategy='mean').fit(X_train,y_train)\n",
        "pred=baseline.predict(X_test)\n",
        "print('baseline MAE=', round(mean_absolute_error(y_test,pred),3))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Fit a transparent model\n",
        "Expected checkpoint: Compare model MAE with the baseline instead of judging the model in isolation.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from sklearn.linear_model import LinearRegression\n",
        "model=LinearRegression().fit(X_train,y_train)\n",
        "pred=model.predict(X_test)\n",
        "print('model MAE=', round(mean_absolute_error(y_test,pred),3))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Inspect errors, not only a score\n",
        "Expected checkpoint: Residual statistics and the largest errors reveal cases worth investigating.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "errors=(y_test-pred).rename('residual')\n",
        "print(errors.describe().round(3))\n",
        "print('largest absolute errors:')\n",
        "print(errors.abs().sort_values(ascending=False).head())\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.x"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}