XenorsHow to Fine-Tune LLMs with Hugging Face: LLaMA and Mistral
Full fine-tuning of a 7B model needs more memory than most people have. LoRA and QLoRA train a few million extra parameters instead, so one consumer GPU is enough.
Prompting, RAG or fine-tuning?
Fine-tune when you need consistent format, tone or a narrow skill. Use RAG when answers depend on documents that change. Use prompting first, because it is the cheapest to test.
| Need | Best tool |
|---|---|
| New facts, private documents | RAG |
| Fixed output structure or style | Fine-tuning |
| Quick experiment | Prompt engineering |
Prepare an instruction dataset
Use a consistent chat or instruction template. Quality beats quantity: a few thousand clean, varied examples usually outperform a large noisy set.
from datasets import Dataset
rows = [
{"messages": [
{"role": "user", "content": "Summarise: The server crashed at 2am due to a full disk."},
{"role": "assistant", "content": "Incident: disk full caused a server crash at 2am."}]},
]
ds = Dataset.from_list(rows)
print(ds[0])Load a 4-bit model with LoRA adapters
QLoRA stores base weights in 4-bit and trains small low-rank matrices on top. Model names change over time, so check the Hugging Face Hub for the current checkpoint and license before you start.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import LoraConfig
name = "mistralai/Mistral-7B-Instruct-v0.3" # check licence and access
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16)
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(name, quantization_config=bnb,
device_map="auto")
lora = LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05, task_type="CAUSAL_LM",
target_modules=["q_proj", "k_proj", "v_proj", "o_proj"])Train with SFTTrainer and save the adapter
Keep the learning rate low and watch validation loss. Save only the adapter, which is typically tens of megabytes.
from trl import SFTTrainer, SFTConfig
args = SFTConfig(output_dir="out", num_train_epochs=2, learning_rate=2e-4,
per_device_train_batch_size=2, gradient_accumulation_steps=8,
logging_steps=10, bf16=True)
trainer = SFTTrainer(model=model, args=args, train_dataset=ds,
peft_config=lora, processing_class=tok)
trainer.train()
trainer.model.save_pretrained("my-adapter")Evaluate before you ship
Hold out examples the model never saw and compare base versus fine-tuned answers side by side. Check for regressions on general tasks, since over-training can cause forgetting. Track exact-match or rubric scores for structured outputs, and review samples by hand.
Hands-on practice
Run the examples in the downloadable Jupyter notebook. The extra checks below focus on mistakes that appear in real projects, not just syntax.
Format one instruction example
sample={'instruction':'Summarize','input':'RAG retrieves documents.','output':'RAG grounds answers with retrieved context.'}
text=f"Instruction: {sample['instruction']}\nInput: {sample['input']}\nAnswer: {sample['output']}"
print(text)Instruction: Summarize Input: RAG retrieves documents. Answer: RAG grounds answers with retrieved context.
Consistent formatting matters more than adding decorative prompt text.
Estimate trainable LoRA parameters
total=7_000_000_000
trainable=8_000_000
print(f'{100*trainable/total:.3f}% trainable')0.114% trainable
PEFT reduces optimizer memory by updating a small fraction of weights.
Self-check
- Can you explain why the first practice result is trustworthy rather than just plausible?
- What would you monitor or test before using this LLM Fine-Tuning workflow in production?
- Which assumption in the example would break first on a larger or messier dataset?
Frequently asked questions
What is the difference between LoRA and QLoRA?
LoRA trains small adapter matrices on top of frozen weights. QLoRA does the same while keeping the frozen base model quantised to 4-bit, which cuts memory sharply.
How much GPU memory do I need?
A 7B model with QLoRA typically fits on a single GPU with roughly 16 to 24 GB, depending on sequence length and batch size.
Can fine-tuning teach a model new facts?
It is unreliable for facts. Use retrieval-augmented generation for knowledge and fine-tuning for behaviour.
Primary references
Use these official sources when exact syntax, defaults or definitions matter beyond the examples here.
