{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Kafka Streaming Tutorial for Data Pipelines\n",
        "Xenors practice notebook. Run each cell and compare the output with the tutorial page.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Create a compact JSON event\n",
        "Stable event schemas make downstream consumers easier to evolve.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import json, time\n",
        "event = {'device_id':'M-17','temp_c':71.2,'ts':int(time.time())}\n",
        "payload = json.dumps(event).encode()\n",
        "print(payload.decode())\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "**Expected/checkpoint**\n",
        "```text\n{\"device_id\": \"M-17\", \"temp_c\": 71.2, \"ts\": ...}\n```\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Make the consumer idempotent\n",
        "At-least-once delivery means consumers should tolerate repeats.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "seen=set()\n",
        "def process(event_id, value):\n",
        "    if event_id in seen: return 'duplicate skipped'\n",
        "    seen.add(event_id)\n",
        "    return f'processed {value}'\n",
        "print(process('e1',42)); print(process('e1',42))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "**Expected/checkpoint**\n",
        "```text\nprocessed 42\nduplicate skipped\n```\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.x"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}