{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Fine-Tune LLMs with Hugging Face\n",
        "Xenors practice notebook. Run each cell and compare the output with the tutorial page.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Format one instruction example\n",
        "Consistent formatting matters more than adding decorative prompt text.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "sample={'instruction':'Summarize','input':'RAG retrieves documents.','output':'RAG grounds answers with retrieved context.'}\n",
        "text=f\"Instruction: {sample['instruction']}\\nInput: {sample['input']}\\nAnswer: {sample['output']}\"\n",
        "print(text)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "**Expected/checkpoint**\n",
        "```text\nInstruction: Summarize\nInput: RAG retrieves documents.\nAnswer: RAG grounds answers with retrieved context.\n```\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Estimate trainable LoRA parameters\n",
        "PEFT reduces optimizer memory by updating a small fraction of weights.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "total=7_000_000_000\n",
        "trainable=8_000_000\n",
        "print(f'{100*trainable/total:.3f}% trainable')\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "**Expected/checkpoint**\n",
        "```text\n0.114% trainable\n```\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.x"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}