Parameter-efficient fine-tuning of Mistral-7B using LoRA, demonstrating production-grade ML engineering: dataset preparation, 4-bit quantized training, evaluation with automated metrics, and efficient inference serving.
┌─────────────────────────────────────────────────────────────────┐
│ LoRA Fine-Tune Lab │
├─────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ Data Prep │───▶│ Fine-Tune │───▶│ Evaluation │ │
│ │ │ │ │ │ │ │
│ │ • Load HF │ │ • QLoRA 4bit │ │ • Perplexity │ │
│ │ dataset │ │ • r=16 α=32 │ │ • ROUGE scores │ │
│ │ • Tokenize │ │ • Grad accum │ │ • Task metrics │ │
│ │ • Train/val │ │ • Checkpoints│ │ • W&B logging │ │
│ │ split │ │ │ │ │ │
│ └──────────────┘ └──────┬───────┘ └──────────────────┘ │
│ │ │
│ ┌────────▼────────┐ │
│ │ LoRA Adapter │ │
│ │ (~168 MB) │ │
│ └────────┬────────┘ │
│ │ │
│ ┌────────▼────────┐ │
│ │ Inference │ │
│ │ CLI │ │
│ │ │ │
│ │ Base + Adapter │ │
│ │ 4-bit quantized │ │
│ └─────────────────┘ │
│ │
├─────────────────────────────────────────────────────────────────┤
│ Tech: Transformers · PEFT · Accelerate · bitsandbytes · W&B │
└─────────────────────────────────────────────────────────────────┘
| Skill | Implementation |
|---|---|
| Parameter-efficient fine-tuning | LoRA with rank-16 decomposition on all attention and MLP projections (q, k, v, o, gate, up, down) |
| Memory-efficient training | QLoRA 4-bit quantization via bitsandbytes NF4 |
| Training engineering | Gradient accumulation, cosine scheduling, warmup |
| Evaluation rigor | Held-out 5% split. The trainer logs eval loss and token accuracy, and evaluate.py computes perplexity and ROUGE-L on generated responses |
| Experiment tracking | Weights & Biases integration with artifact logging |
| Production inference | Adapter loaded on the 4-bit base model, with a streaming CLI |
| Reproducibility | YAML configs, fixed seed (42), minimum dependency versions in requirements.txt |
# Clone and install
git clone https://github.com/DevDizzle/lora-finetune-lab.git
cd lora-finetune-lab
pip install -r requirements.txt
# Prepare data
python src/data_prep.py --dataset tatsu-lab/alpaca --output data/
# Fine-tune (requires a GPU, see GUIDE.md)
python -m notebooks.finetune
# Evaluate
python -m notebooks.evaluate
# Run inference
python src/inference.py --adapter outputs/lora-adapter --prompt "Explain LoRA in one paragraph."lora-finetune-lab/
├── configs/
│ ├── lora_config.yaml # LoRA hyperparameters
│ └── training_config.yaml # Training loop config
├── notebooks/
│ ├── finetune.py # Fine-tuning notebook (percent script)
│ └── evaluate.py # Evaluation notebook (percent script)
├── src/
│ ├── data_prep.py # Dataset preprocessing
│ └── inference.py # Inference CLI
├── requirements.txt
├── GUIDE.md # Step-by-step execution guide
└── README.md
Run Date: February 12, 2026 Hardware: NVIDIA A100 Training Time: ~3 hours (2h 59m 54s) Tracking: Weights & Biases (private project). The adapter is logged as a W&B artifact.
| Metric | Value | Notes |
|---|---|---|
| Eval Loss | 1.187 | On the held-out 5% split. The gap to the final train loss (0.890) suggests some overfitting by epoch 3 |
| Perplexity | 3.28 | exp(eval loss) on the same split |
| Eval Entropy | 0.888 | Measure of prediction uncertainty |
| Token Accuracy | 70.78% | Percentage of tokens predicted correctly |
| Train Loss | 0.890 | Final training loss after 3 epochs |
Example Inference:
Instruction: Write a python function to reverse a list.
Response:
def reverse_list(lst): return lst[::-1]
Training Details:
- Base model:
mistralai/Mistral-7B-v0.3 - Trainable parameters: 41.9M of 7.29B (0.58%), from rank 16 on 7 projections x 32 layers
- Epochs: 3
- Speed: 13.73 samples/sec (train), 61.35 samples/sec (eval)
MIT