Model Training, Fine-Tuning & Evaluation — 2026 Guide

A practical map of the model training and fine-tuning landscape — the techniques, tools, and trade-offs I reach for, minus the hype.

01 — The Training Landscape

Model training in 2026 is a stack of stages, each with its own technique, data requirement, and compute profile. Understanding where each stage sits prevents the most common mistake: applying the wrong technique to the wrong problem. Training is one layer of a larger system, and the AI architecture field guide is where I work out when to reach for it at all rather than prompting or RAG.

The model training stack (stages run top → bottom):

Where you fit in: Most practitioners work at the SFT + alignment layers. Pre-training and continued pre-training are frontier-lab territory (Anthropic, OpenAI, Meta). Your job as an AI architect is to know which layer solves your problem and which technique to apply at that layer.

02 — Fine-Tuning Techniques

The central question: how many parameters do you actually need to change? The answer determines your compute cost, data requirement, and quality ceiling.

The 2026 default: QLoRA SFT is the starting point for almost every fine-tuning project. LoRA when you have GPU headroom and need maximum throughput. Full fine-tuning only when adapters demonstrably can’t close the quality gap (rare). Unsloth’s optimized kernels make QLoRA only slightly slower than LoRA while using about 4x less VRAM.

How LoRA works (the key insight)

• Instead of updating a huge weight matrix W (billions of params), LoRA freezes W and trains two small matrices A and B such that the update is ΔW = A × B.
• Rank r controls adapter size: r=8 is common, r=16–64 for harder tasks. Higher rank = more capacity but more compute.
• At inference, merge the adapter into the base model (zero overhead) or keep separate (swap adapters per task).
• QLoRA adds 4-bit NF4 quantization on the frozen base — gradients flow through dequantized weights during training, but the base never leaves 4-bit in memory.

LoRA vs QLoRA vs full fine-tuning — VRAM for a 70B model:

03 — Alignment & Preference Tuning

After SFT teaches the model what to do, alignment teaches it how well to do it — ranking outputs by human preference.

The practical default: For most fine-tuning projects, SFT alone is enough. Add DPO/ORPO if you have preference pairs and want to refine quality/safety. Reach for GRPO for critic-free RL post-training — most commonly reasoning tasks with verifiable rewards (RLVR), though it also works with a learned reward model. RLHF is the frontier lab’s tool — most teams don’t need it.

Alignment technique decision tree:

04 — Training Frameworks & Tools

Starter recommendation: Unsloth + TRL for single-GPU QLoRA fine-tuning (fastest, easiest). Axolotl for multi-GPU or when you want YAML configs. DeepSpeed/FSDP when models don’t fit on a single GPU. torchtitan if you want pure PyTorch without HF abstractions.

Supporting tools

05 — Data Preparation & Curation

Data quality is the single biggest determinant of fine-tuning success. 5K well-curated examples consistently outperform 50K noisy ones.

Data formats by training type

Data quality checklist

• Deduplication: remove exact and near-duplicate examples — duplicates cause memorization, not generalization.
• Contamination check: ensure evaluation data is not in the training set. This is the #1 cause of inflated metrics.
• Quality filtering: remove low-quality, off-topic, or contradictory examples. Use an LLM judge to score quality if manual review is impractical.
• Mix base data (5–20%): include general instruction data during domain SFT to prevent catastrophic forgetting — the model losing its general capabilities.
• Balance: ensure class/topic distribution matches what you want the model to learn, not just what’s easiest to collect.
• Privacy: PII redaction before training. Differential privacy if fine-tuning on user data.

Synthetic data (the 2026 shortcut)

Use a frontier model (Claude Opus 5.5, GPT-6 Astra) to generate training data — instruction-response pairs, preference comparisons, or domain examples. Standard practice in 2026, but requires evaluation: always validate synthetic data quality before training on it.

Common mistake: Fine-tuning on synthetic data generated by the same model you’re fine-tuning → model collapse (the model learns to imitate its own biases). Use a stronger model to generate training data for a weaker model (distillation pattern).

06 — Distributed Training

When a model doesn’t fit on one GPU, or you need faster training, you distribute across multiple GPUs/nodes.

Key training optimizations

• Mixed precision (bf16/fp16): train in half precision — 2x throughput, half memory, minimal quality loss. bf16 preferred (no loss scaling needed).
• Gradient checkpointing: recompute activations during backward pass instead of storing them — saves a large share of activation memory at roughly a 20% speed cost.
• Flash Attention (2/3/4): memory-efficient attention computation — essential for long sequences. Built into most frameworks. FlashAttention-2 is still the stable default; FlashAttention-3 (Hopper, FP8 forward pass) and FlashAttention-4 (Hopper and Blackwell) are the newer kernels, both still in beta.
• Gradient accumulation: simulate larger batch sizes by accumulating gradients over multiple micro-batches before updating.
• Checkpointing: save model state regularly during training — critical when using spot/preemptible GPUs that can be interrupted.

07 — Evaluation Harnesses & Benchmarks

The eval harness must exist before training starts. Without it, you cannot tell if a checkpoint is better than the last. This is the most under-invested area in most fine-tuning projects.

Evaluation frameworks

The evaluation stack for fine-tuning: Before training: lm-evaluation-harness baseline on the base model. After each checkpoint: task-specific metrics (exact match, JSON validity, tool-call accuracy) + LLM-as-judge (faithfulness, instruction following, tone). Before production: human eval on a sample + Promptfoo red-teaming. In production: RAGAS for RAG quality + DeepEval as CI gate.

Key benchmarks to know

Eval traps to avoid:

• Evaluating only on the task you fine-tuned for — check for catastrophic forgetting on general benchmarks too.
• Training data contaminating eval set — always hold out eval data before any training begins.
• Trusting a single metric — use task metrics + LLM judge + human eval together.
• Evaluating once — evaluate at every checkpoint and after every data change.

08 — Model Optimization (Post-Training)

After training, make the model smaller, faster, and cheaper for inference.

Trade-off — quantization level: INT8: nearly lossless, 2x savings — the safe default. INT4 (GPTQ/AWQ): 4x savings with slight quality loss — production-viable for most tasks. Aggressive (2-3 bit): large quality drop — only for latency-critical, accuracy-tolerant use cases. Always evaluate your specific task after quantization — generic benchmarks don’t predict domain-specific impact.

09 — Open-Weight Ecosystem

The base models you’ll fine-tune. “Open weights” means you download the weights and run/fine-tune them yourself.

How to choose a base model for fine-tuning: Start with the smallest model that does well on your task with good prompting. Fine-tuning a 7B model is 10x cheaper and faster than fine-tuning a 70B. If 7B SFT isn’t good enough, try 14B before jumping to 70B. Benchmark the base model with prompting first — if it’s already 90% there, fine-tuning the last 10% is much cheaper than starting from a weak base.

10 — The Complete Training Workflow

End-to-end fine-tuning workflow:

1. Define task — set success metrics
2. Build eval harness — before training
3. Curate data — 500–5K examples
4. Baseline — evaluate the base model
5. SFT (QLoRA) — train + checkpoint
6. Evaluate — vs baseline
7. Alignment? — DPO if preference data
8. Evaluate again — plus catastrophic forgetting check
9. Merge adapter — and quantize (INT4/8)
10. Deploy — vLLM / cloud endpoint
11. Monitor — drift + quality → retrain as needed

Steps 10 and 11 are where this guide hands off: serving, drift detection, and the retraining loop are the subject of MLOps and AI production operations.

The golden rule: Steps 2 and 6 (evaluation) are the most important. Training without evaluation is like flying without instruments — you have no idea if you’re improving or crashing. Build the eval harness before you write a single line of training code.

11 — Cloud Training Services

Trade-off — managed fine-tuning API vs self-managed: Managed API (Together, Fireworks, Gemini tuning) = upload data, get model, zero infra — but limited control over hyperparameters, training loop, and model architecture. (OpenAI is winding its fine-tuning platform down: closed to new organizations since May 2026, no new jobs for anyone from January 2027.) Self-managed (Unsloth on cloud GPU) = full control, cheaper per hour — but you own the entire training stack. Start with managed for validation; switch to self-managed for production optimization.

12 — Trade-off Master Reference

Built from current frameworks, research papers, and production practice. Independent reference — not affiliated with any vendor. Framework versions and model availability change rapidly; verify before starting a training run.