{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Spark SQL & DataFrames Tutorial for Beginners\n",
        "Xenors practice notebook. Run each cell and compare the output with the tutorial page.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Verify a join with explicit keys\n",
        "A left join preserves every order even when the customer dimension is incomplete.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "orders = spark.createDataFrame([(1,10),(2,20)], ['customer_id','amount'])\n",
        "customers = spark.createDataFrame([(1,'A'),(3,'C')], ['customer_id','name'])\n",
        "orders.join(customers,'customer_id','left').show()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "**Expected/checkpoint**\n",
        "```text\n+-----------+------+----+\n|customer_id|amount|name|\n+-----------+------+----+\n|1          |10    |A   |\n|2          |20    |NULL|\n```\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Read the physical plan\n",
        "Use explain() before guessing about Spark performance.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "q = orders.groupBy('customer_id').sum('amount')\n",
        "q.explain(mode='formatted')\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "**Expected/checkpoint**\n",
        "```text\nPhysical Plan\n+- HashAggregate ...\n```\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.x"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}