{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Exploratory Data Analysis (EDA) in Python: Step-by-Step Guide\n",
        "Practice notebook for the Xenors lesson: https://xenors.in/data-science/exploratory-data-analysis-python/\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "df = pd.read_csv('../datasets/exploratory-data-analysis-python.csv')\n",
        "df.head()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Baseline example\n",
        "Expected checkpoint: Run the cell and inspect the printed values before changing anything.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "df = pd.read_csv('houses.csv')\n",
        "print(df.info())\n",
        "print(df.describe(include='all'))\n",
        "print(df.isna().mean().sort_values(ascending=False).head())\n",
        "df.plot.scatter(x='area_sqft', y='price', alpha=.4)\n",
        "plt.show()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Inspect plotting columns\n",
        "Expected checkpoint: Check that numeric axes are numeric and categories are not accidentally encoded as measures.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "df = pd.read_csv('practice.csv')\n",
        "print(df.dtypes)\n",
        "print(df.head(3).to_string(index=False))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Plot a distribution\n",
        "Expected checkpoint: A histogram appears; look for skew, gaps and extreme values before summarizing the center.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "num = df.select_dtypes('number').columns[0]\n",
        "ax = df[num].plot(kind='hist', bins=12, edgecolor='black')\n",
        "ax.set(title=f'Distribution of {num}', xlabel=num)\n",
        "plt.tight_layout()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Compare groups\n",
        "Expected checkpoint: The table shows group sizes beside mean and median so small groups are visible.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "cat = df.select_dtypes(exclude='number').columns[0]\n",
        "num = df.select_dtypes('number').columns[0]\n",
        "print(df.groupby(cat)[num].agg(['count','mean','median']).round(2))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Save a reproducible figure\n",
        "Expected checkpoint: practice-chart.png is saved with fixed dimensions and resolution.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "fig, ax = plt.subplots(figsize=(8,4.5))\n",
        "df[num].plot(kind='box', ax=ax, vert=False)\n",
        "ax.set_title(f'{num}: spread and possible outliers')\n",
        "fig.tight_layout()\n",
        "fig.savefig('practice-chart.png', dpi=160)\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.x"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}