{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# ML Model Monitoring & Drift Detection\n",
        "Xenors practice notebook. Run each cell and compare the output with the tutorial page.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Calculate population stability index\n",
        "PSI is a monitoring signal, not a universal retraining rule.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "def psi(expected, actual, bins=10):\n",
        "    q=np.quantile(expected,np.linspace(0,1,bins+1)); q[0]-=1e-9; q[-1]+=1e-9\n",
        "    e=np.histogram(expected,q)[0]/len(expected); a=np.histogram(actual,q)[0]/len(actual)\n",
        "    e=np.clip(e,1e-6,None); a=np.clip(a,1e-6,None)\n",
        "    return np.sum((a-e)*np.log(a/e))\n",
        "print(round(psi(np.random.normal(0,1,5000), np.random.normal(.4,1,5000)),3))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "**Expected/checkpoint**\n",
        "```text\n~0.15 to 0.25 depending on sample\n```\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Separate feature drift from performance drift\n",
        "Monitor both data and outcome quality when labels become available.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "signals={'age_psi':0.18,'auc_baseline':0.84,'auc_current':0.81}\n",
        "print('feature_drift',signals['age_psi']>0.1)\n",
        "print('performance_drop',signals['auc_baseline']-signals['auc_current']>0.02)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "**Expected/checkpoint**\n",
        "```text\nfeature_drift True\nperformance_drop True\n```\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.x"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}