Riddam Jain — Playbooks

Model Training, Fine-Tuning & Evaluation — 2026 Guide

Sat, 11 Jul 2026 00:00:00 GMT

A practical map of the model training and fine-tuning landscape — the techniques, tools, and trade-offs I reach for, minus the hype.

01 — The Training Landscape

Model training in 2026 is a stack of stages, each with its own technique, data requirement, and compute profile. Understanding where each stage sits prevents the most common mistake: applying the wrong technique to the wrong problem. Training is one layer of a larger system, and the AI architecture field guide is where I work out when to reach for it at all rather than prompting or RAG.

The model training stack (stages run top → bottom):

Stage	Description	Cost & scale
Pre-training	Train from scratch on massive text corpus (trillions of tokens). Creates the base model. Only done by frontier labs.	$M+ · months · 1000s GPUs
Continued pre-training	Extend the base model with domain-specific corpus (medical, legal, code). Rare — needs large domain data.	$10K–100K+ · weeks
Supervised fine-tuning (SFT)	Train on (instruction, response) pairs to teach specific behavior, format, tone. The most common fine-tuning.	$100–5K · hours–days
Preference alignment	DPO / ORPO / RLHF on (chosen, rejected) pairs to refine quality, safety, style. Usually after SFT.	$100–5K · hours–days
Post-training optimization	Quantization, distillation, pruning to make the model smaller/faster/cheaper for inference.	$0–100 · minutes–hours

Where you fit in: Most practitioners work at the SFT + alignment layers. Pre-training and continued pre-training are frontier-lab territory (Anthropic, OpenAI, Meta). Your job as an AI architect is to know which layer solves your problem and which technique to apply at that layer.

02 — Fine-Tuning Techniques

The central question: how many parameters do you actually need to change? The answer determines your compute cost, data requirement, and quality ceiling.

Technique	What it changes	VRAM	Data needed	When to use
Full fine-tuning	All model weights	Very high (~16 bytes/param ≈ 8× fp16 model size)	10K+ examples	Deep domain adaptation; rarely needed for most tasks
LoRA	Small low-rank adapter matrices (0.1–1% of params)	Moderate (model + adapters)	500–5K examples	Default for 2026 — best quality/cost ratio
QLoRA	Same as LoRA but base model stays 4-bit quantized	Low (~50–70% less than LoRA)	500–5K examples	Memory-constrained (single GPU, consumer hardware)
DoRA	LoRA + weight decomposition (magnitude + direction)	Slightly more than LoRA	500–5K examples	When LoRA quality isn't quite enough; marginal improvement

The 2026 default: QLoRA SFT is the starting point for almost every fine-tuning project. LoRA when you have GPU headroom and need maximum throughput. Full fine-tuning only when adapters demonstrably can't close the quality gap (rare). Unsloth's optimized kernels brought QLoRA within ~10–20% of LoRA throughput.

How LoRA works (the key insight)

Instead of updating a huge weight matrix W (billions of params), LoRA freezes W and trains two small matrices A and B such that the update is ΔW = A × B.
Rank r controls adapter size: r=8 is common, r=16–64 for harder tasks. Higher rank = more capacity but more compute.
At inference, merge the adapter into the base model (zero overhead) or keep separate (swap adapters per task).
QLoRA adds 4-bit NF4 quantization on the frozen base — gradients flow through dequantized weights during training, but the base never leaves 4-bit in memory.

LoRA vs QLoRA vs full fine-tuning — VRAM for a 70B model:

Method	VRAM	Hardware
Full fine-tune	~1.1 TB VRAM (mixed-precision Adam ≈ 16 bytes/param — 2 weights + 2 grads + 8 optimizer + 4 fp32 master ≈ 8× fp16 size); ~16+ H100s, or ~400–600 GB with ZeRO-3/offload	Multi-GPU required
LoRA (16-bit base)	~140 GB (base in 16-bit + small adapters + optimizer for adapters only)	2–4 GPUs
QLoRA (4-bit base)	~48 GB (base in 4-bit + adapters in 16-bit)	Single H100 (80GB)

03 — Alignment & Preference Tuning

After SFT teaches the model what to do, alignment teaches it how well to do it — ranking outputs by human preference.

Technique	How it works	Data format	When to use
RLHF	Train a reward model from preference pairs → optimize LLM with PPO against the reward	(prompt, chosen, rejected) + reward model	Frontier alignment; most complex but most expressive. Labs only.
DPO	Direct Preference Optimization — skips the reward model; optimizes directly on preference pairs	(prompt, chosen, rejected)	2026 default for preference tuning. Simpler than RLHF, competitive quality.
ORPO	Merges SFT and DPO into one stage — simplest pipeline	(prompt, chosen, rejected)	When you want the simplest possible pipeline and have preference data
KTO	Kahneman-Tversky Optimization — works with just thumbs-up/thumbs-down (no paired comparisons needed)	(prompt, response, good/bad label)	When you have binary feedback, not pairwise preferences
GRPO	Group Relative Policy Optimization — critic-free RL (introduced in DeepSeekMath, the method behind DeepSeek R1) using group-relative advantages; works with learned reward models or verifiable functions	(prompt, group of responses, reward)	General RL post-training; reasoning tasks. RLVR (verifiable rewards — math, code, logic) is one common application, not the only one
SimPO / IPO	Variants of DPO addressing edge cases (length bias, BT-model assumption)	Same as DPO	When vanilla DPO produces length-biased or unstable results

The practical default: For most fine-tuning projects, SFT alone is enough. Add DPO/ORPO if you have preference pairs and want to refine quality/safety. Reach for GRPO for critic-free RL post-training — most commonly reasoning tasks with verifiable rewards (RLVR), though it also works with a learned reward model. RLHF is the frontier lab's tool — most teams don't need it.

Alignment technique decision tree:

Question	Answer
Do you have paired preference data (chosen vs rejected)?	→ DPO (default) or ORPO (simpler)
Only binary feedback (thumbs up/down), no pairs?	→ KTO
Want critic-free RL — a verifiable reward function (math, code tests) or a learned reward model?	→ GRPO
Need maximum alignment control, budget for reward model training?	→ RLHF (PPO)
No preference data at all?	→ SFT only (generate synthetic preferences if needed)

04 — Training Frameworks & Tools

Framework	What it is	Best for	2026 status
HF TRL (v1.0+)	Hugging Face's unified post-training library: SFTTrainer, DPOTrainer, KTOTrainer, ORPOTrainer, GRPOTrainer, RewardTrainer	Full control over the training loop; the "standard library" for fine-tuning	v1.0 (April 2026) — the default
Unsloth	Optimized kernels for LoRA/QLoRA — 2–5x faster, ~50–70% less VRAM than naive HF + bitsandbytes	Single-GPU fine-tuning; the fastest path to a working fine-tune	Integrated into TRL v1.0
Axolotl	Config-driven (YAML) fine-tuning framework — supports LoRA, QLoRA, full, DPO, GRPO, ORPO, reward modeling	Multi-GPU pipelines; teams wanting declarative configs over code	v0.29 (Feb 2026); active community
torchtune	PyTorch-native fine-tuning library from the PyTorch team	Teams wanting no HF dependency; pure PyTorch ecosystem	Growing; good for custom training loops
MLX-LM	Apple Silicon native — LoRA, QLoRA, DoRA on Mac	Local fine-tuning on Mac (32 GB → 7–8B; 96 GB → 70B QLoRA)	Mature for Apple Silicon users
DeepSpeed	Microsoft's distributed training library — ZeRO stages 1/2/3 for memory optimization	Multi-GPU/multi-node training; large models that don't fit on one GPU	Essential for full fine-tuning at scale
FSDP (Fully Sharded Data Parallel)	PyTorch's native distributed training — shards model across GPUs	Multi-GPU training without DeepSpeed dependency	Built into PyTorch; simpler than DeepSpeed

Starter recommendation: Unsloth + TRL for single-GPU QLoRA fine-tuning (fastest, easiest). Axolotl for multi-GPU or when you want YAML configs. DeepSpeed/FSDP when models don't fit on a single GPU. torchtune if you want pure PyTorch without HF abstractions.

Supporting tools

Tool	Function
Hugging Face Transformers	Model loading, tokenization, inference — the foundation everything else builds on
Hugging Face PEFT	LoRA/QLoRA/DoRA adapter management (save, load, merge adapters)
bitsandbytes	4-bit/8-bit quantization for loading large models on limited VRAM
Weights & Biases (W&B)	Experiment tracking, hyperparameter sweeps, model comparison dashboards
MLflow	Experiment tracking + model registry (open-source alternative to W&B)
Hugging Face Hub	Model hosting, sharing, versioning — the "GitHub for models"

05 — Data Preparation & Curation

Data quality is the single biggest determinant of fine-tuning success. 5K well-curated examples consistently outperform 50K noisy ones.

Data formats by training type

Training type	Data format	Example
SFT (instruction tuning)	`{"instruction": "...", "response": "..."}` or chat format with system/user/assistant turns	`{"instruction": "Summarize this email", "response": "The email discusses..."}`
DPO / ORPO	`{"prompt": "...", "chosen": "...", "rejected": "..."}`	Two responses to the same prompt, one preferred
GRPO	`{"prompt": "...", "response": "...", "reward": 0.85}`	Response + verifiable reward score
Continued pre-training	Raw text documents (no instruction format)	Domain-specific corpus (medical literature, legal filings)

Data quality checklist

Deduplication: remove exact and near-duplicate examples — duplicates cause memorization, not generalization.
Contamination check: ensure evaluation data is not in the training set. This is the #1 cause of inflated metrics.
Quality filtering: remove low-quality, off-topic, or contradictory examples. Use an LLM judge to score quality if manual review is impractical.
Mix base data (5–20%): include general instruction data during domain SFT to prevent catastrophic forgetting — the model losing its general capabilities.
Balance: ensure class/topic distribution matches what you want the model to learn, not just what's easiest to collect.
Privacy: PII redaction before training. Differential privacy if fine-tuning on user data.

Synthetic data (the 2026 shortcut)

Use a frontier model (Claude Opus 5, GPT-5) to generate training data — instruction-response pairs, preference comparisons, or domain examples. Standard practice in 2026, but requires evaluation: always validate synthetic data quality before training on it.

Common mistake: Fine-tuning on synthetic data generated by the same model you're fine-tuning → model collapse (the model learns to imitate its own biases). Use a stronger model to generate training data for a weaker model (distillation pattern).

06 — Distributed Training

When a model doesn't fit on one GPU, or you need faster training, you distribute across multiple GPUs/nodes.

Strategy	What it does	Use when
Data Parallel (DDP)	Each GPU has a full model copy; data is split across GPUs. Gradients averaged.	Model fits on one GPU but you want faster training
FSDP (Fully Sharded)	Shards model weights, gradients, and optimizer states across GPUs	Model doesn't fit on one GPU; PyTorch native
DeepSpeed ZeRO-1/2/3	Progressive memory optimization — ZeRO-3 shards everything	Very large models; maximum memory efficiency
Tensor Parallelism	Splits individual layers across GPUs (within a node)	Very large layers that don't fit on one GPU even sharded
Pipeline Parallelism	Splits model layers sequentially across GPUs	Very deep models; used alongside tensor parallelism

Key training optimizations

Mixed precision (bf16/fp16): train in half precision — 2x throughput, half memory, minimal quality loss. bf16 preferred (no loss scaling needed).
Gradient checkpointing: recompute activations during backward pass instead of storing them — saves ~60% memory at ~30% speed cost.
Flash Attention (2/3): memory-efficient attention computation — essential for long sequences. Built into most frameworks. In 2026, Flash Attention 3 is the default on Hopper (H100) with FP8 support, and FA4 is emerging for next-gen hardware.
Gradient accumulation: simulate larger batch sizes by accumulating gradients over multiple micro-batches before updating.
Checkpointing: save model state regularly during training — critical when using spot/preemptible GPUs that can be interrupted.

07 — Evaluation Harnesses & Benchmarks

The eval harness must exist before training starts. Without it, you cannot tell if a checkpoint is better than the last. This is the most under-invested area in most fine-tuning projects.

Evaluation frameworks

Framework	What it does	Best for
lm-evaluation-harness (EleutherAI)	Standardized benchmark suite: MMLU, GSM8K, HumanEval, HellaSwag, 400+ tasks	Base model benchmarking; academic comparison. No substitute.
DeepEval	pytest-native LLM eval — 14+ metrics: hallucination, bias, toxicity, RAG faithfulness	CI/CD integration; broad application eval; quality gates that block deploys
RAGAS	RAG-specific evaluation: context precision, recall, faithfulness, answer relevance	RAG pipeline evaluation; the standard for retrieval quality
Promptfoo	YAML-driven multi-model comparison + red-teaming (500+ adversarial vectors)	Prompt/model selection; security testing; CLI-first workflow
LLM-as-judge	Use a stronger model to evaluate a weaker model's output against rubrics	Scalable proxy for human eval; runs on 5–10% of production traces
Human evaluation	Domain experts rate outputs on rubrics (accuracy, helpfulness, safety)	Gold standard; expensive; use for final validation

The evaluation stack for fine-tuning: Before training: lm-evaluation-harness baseline on the base model. After each checkpoint: task-specific metrics (exact match, JSON validity, tool-call accuracy) + LLM-as-judge (faithfulness, instruction following, tone). Before production: human eval on a sample + Promptfoo red-teaming. In production: RAGAS for RAG quality + DeepEval as CI gate.

Key benchmarks to know

Benchmark	What it measures
MMLU	Broad knowledge across 57 subjects. Historically the headline general-capability metric, but largely saturated by 2026 — prefer MMLU-Pro and GPQA as more discriminating successors
GSM8K	Grade-school math reasoning
HumanEval / MBPP	Code generation and programming ability
MTEB	Embedding model quality (the benchmark for RAG embedding models)
MT-Bench	Multi-turn conversation quality (LLM-as-judge scored)
IFEval	Instruction following accuracy (does the model do exactly what you asked?)
TruthfulQA	Hallucination resistance — does the model say "I don't know" when it should?

Eval traps to avoid:

Evaluating only on the task you fine-tuned for — check for catastrophic forgetting on general benchmarks too.

Training data contaminating eval set — always hold out eval data before any training begins.

Trusting a single metric — use task metrics + LLM judge + human eval together.

Evaluating once — evaluate at every checkpoint and after every data change.

08 — Model Optimization (Post-Training)

After training, make the model smaller, faster, and cheaper for inference.

Technique	What it does	Speed-up	Quality impact
Quantization (INT8)	Reduce weight precision from 16-bit to 8-bit	~2x throughput, ~50% memory	Minimal (< 1% accuracy loss typically)
Quantization (INT4 / GPTQ / AWQ)	Reduce to 4-bit with calibration	~4x throughput, ~75% memory	Slight (~1–3% loss); good enough for most production
GGUF (llama.cpp format)	Optimized quantized format for CPU/local inference	Runs on CPU/Mac	Varies by quant level (Q4_K_M is the sweet spot)
Distillation	Train a small model to mimic a large model's outputs	10–100x smaller/faster	Depends on task; can retain 90%+ of quality on narrow domains
Pruning	Remove unnecessary weights/neurons	1.5–3x faster	Moderate; less common than quantization in 2026
Speculative decoding	Small model drafts tokens, large model verifies — net faster generation	2–3x generation speed	Identical output (verification ensures quality)

Trade-off — quantization level: INT8: nearly lossless, 2x savings — the safe default. INT4 (GPTQ/AWQ): 4x savings with slight quality loss — production-viable for most tasks. Aggressive (2-3 bit): large quality drop — only for latency-critical, accuracy-tolerant use cases. Always evaluate your specific task after quantization — generic benchmarks don't predict domain-specific impact.

09 — Open-Weight Ecosystem

The base models you'll fine-tune. "Open weights" means you download the weights and run/fine-tune them yourself.

Family	Provider	Sizes	Strengths	License
Llama 4	Meta	Scout (109B MoE), Maverick (400B+ MoE)	Best open-weight general capability; huge ecosystem	Llama Community
Llama 3.3	Meta	70B	Dense model; well-tested; huge fine-tuning ecosystem	Llama Community
Qwen 3	Alibaba	0.6B–235B (MoE + dense)	Strong multilingual; competitive embeddings; thinking mode	Apache 2.0
Gemma 3	Google	1B–27B	Efficient; multimodal (vision); strong for size	Gemma (permissive)
Mistral	Mistral AI	Small/Medium/Large	EU-hosted; multilingual; fast inference	Apache 2.0 (some)
DeepSeek-R1 / V3	DeepSeek	R1 (671B MoE), V3 (671B MoE)	Reasoning; cost-efficient; MoE architecture	MIT
Phi-4	Microsoft	14B	Small but punches above weight; good for edge	MIT

How to choose a base model for fine-tuning: Start with the smallest model that does well on your task with good prompting. Fine-tuning a 7B model is 10x cheaper and faster than fine-tuning a 70B. If 7B SFT isn't good enough, try 14B before jumping to 70B. Benchmark the base model with prompting first — if it's already 90% there, fine-tuning the last 10% is much cheaper than starting from a weak base.

10 — The Complete Training Workflow

End-to-end fine-tuning workflow:

Define task — set success metrics
Build eval harness — before training
Curate data — 500–5K examples
Baseline — evaluate the base model
SFT (QLoRA) — train + checkpoint
Evaluate — vs baseline
Alignment? — DPO if preference data
Evaluate again — plus catastrophic forgetting check
Merge adapter — and quantize (INT4/8)
Deploy — vLLM / cloud endpoint
Monitor — drift + quality → retrain as needed

Steps 10 and 11 are where this guide hands off: serving, drift detection, and the retraining loop are the subject of MLOps and AI production operations.

The golden rule: Steps 2 and 6 (evaluation) are the most important. Training without evaluation is like flying without instruments — you have no idea if you're improving or crashing. Build the eval harness before you write a single line of training code.

11 — Cloud Training Services

Service	Provider	Best for
SageMaker Training	AWS	Managed training jobs + HyperPod for fault-tolerant large runs; deepest GPU selection
Vertex AI Training	GCP	Managed training + TPU access; tight BigQuery integration
Azure ML Compute	Azure	Managed training; ND-series GPUs; Enterprise Agreement pricing
Lambda Cloud / RunPod / Vast.ai	GPU clouds	Cheapest GPU hourly rates; no managed MLOps (you manage everything)
Google Colab Pro	Google	Quick experiments; T4/A100 access for prototyping; not for production training
Managed fine-tuning APIs	Together, Fireworks, Anyscale	Upload data → get fine-tuned model endpoint; zero infra management

Trade-off — managed fine-tuning API vs self-managed: Managed API (Together, Fireworks, OpenAI fine-tuning) = upload data, get model, zero infra — but limited control over hyperparameters, training loop, and model architecture. Self-managed (Unsloth on cloud GPU) = full control, cheaper per hour — but you own the entire training stack. Start with managed for validation; switch to self-managed for production optimization.

12 — Trade-off Master Reference

Decision	Option A	Option B	Default
Fine-tuning method	QLoRA (memory-efficient)	LoRA (higher throughput)	QLoRA unless you have GPU headroom
Full fine-tune vs adapter	Full (all params)	LoRA/QLoRA (adapters only)	Adapters unless proven insufficient
Alignment technique	DPO (pairs needed)	SFT only (no pairs)	SFT alone; add DPO if you have preference data
Training framework	Unsloth + TRL (easiest)	Axolotl (YAML, multi-GPU)	Unsloth for single-GPU; Axolotl for multi-GPU
Data quality vs quantity	5K curated examples	50K noisy examples	Quality wins — always curate first
Base model size	7B (cheapest to fine-tune)	70B (most capable)	Smallest model that works on your task
Quantization level	INT8 (nearly lossless)	INT4 (more savings)	INT8 default; INT4 if memory/cost critical
Synthetic vs real data	Synthetic (frontier-generated)	Real (human-created)	Mix both; always validate synthetic quality
Eval approach	Automated (LLM judge)	Human evaluation	Automated for iteration; human for final validation
Distributed strategy	FSDP (PyTorch native)	DeepSpeed ZeRO	FSDP for simplicity; DeepSpeed for maximum memory efficiency
Open weights vs API fine-tuning	Self-hosted (full control)	API (Together/Fireworks/OpenAI)	API for validation; self-hosted for production control
Training hardware	Cloud GPU (H100)	Cloud TPU (v5e)	GPU for flexibility; TPU for large-scale cost efficiency

Built from current frameworks, research papers, and production practice. Independent reference — not affiliated with any vendor. Framework versions and model availability change rapidly; verify before starting a training run.

AI Architecture: A Practitioner's Field Guide (2026)

Sat, 11 Jul 2026 00:00:00 GMT

This is the guide I wish I'd had when I first started designing AI systems: every layer, pattern, and trade-off in one place. I keep it current and reach for it whenever I'm helping someone architect something new.

01 — The AI Architecture Landscape

AI architecture in 2026 is no longer "pick a model and prompt it." It's a layered system with distinct concerns at each level. Understanding where each decision lives prevents the most common mistake: solving a knowledge problem with fine-tuning, or solving a behavior problem with RAG.

Fig 01.1 — The AI system stack

Layer	Concerns
Application layer	chatbots, copilots, autonomous agents, data pipelines, content generation
Orchestration layer	single-agent loops, multi-agent coordination, human-in-the-loop, workflow engines
Intelligence layer	prompt engineering, RAG (retrieval), fine-tuning, tool use / MCP
Model layer	foundation models (GPT, Claude, Gemini, Llama), embedding models, specialized models
Infrastructure layer	GPUs / TPUs, model serving, vector databases, model gateways, observability

The architect's job: At each layer, you're making trade-off decisions: which model, how to customize it, how to orchestrate it, what guardrails to apply, and what infrastructure to run it on. The rest of this guide walks through each layer's decisions systematically.

02 — Foundation Models & Selection

The major model families (2026)

Provider	Frontier	Workhorse	Fast / cheap	Strengths
Anthropic	Claude Fable 5 (most capable) / Opus 4.8	Claude Sonnet 5	Claude Haiku 4.5	Safety, long context (1M; Haiku 200K), coding, structured output
OpenAI	GPT-5 / GPT-5.1	GPT-5 mini	GPT-5 nano	Ecosystem breadth, multimodal, reasoning
Google	Gemini 3 Pro (also 2.5 Pro)	Gemini 3 Pro	Gemini Flash	Multimodal native, huge context (1M+), Google integration
Meta	Llama 4 Maverick (Behemoth was previewed but never shipped)	Llama 4 Maverick	Llama 4 Scout	Open weights, self-hostable, fine-tunable, no API dependency
Mistral	Mistral Large	Mistral Medium	Mistral Small	EU-hosted, multilingual, open weights for some
DeepSeek	DeepSeek-R1	DeepSeek-V3.1 / V3.2	—	Open weights, reasoning, cost-efficient, China-based

Model selection decision framework

Fig 02.1 — How to choose a model

Need maximum reasoning / complex tasks (low volume, high stakes)? → Frontier (Fable 5 / Opus 4.8 / GPT-5 / Gemini 3 Pro)
Need balanced cost + capability for most production workloads? → Workhorse (Sonnet 5 / GPT-5 mini / Gemini Flash)
Need fast, cheap for classification / routing / high-volume simple tasks? → Fast (Haiku 4.5 / GPT-5 nano / Gemini Flash)
Need data sovereignty / no vendor lock-in / custom fine-tuning? → Open weights (Llama / Mistral / DeepSeek)

Trade-off — closed API vs open weights: Closed API (Claude, GPT) = highest capability, zero infra, pay-per-token, vendor lock-in. Open weights (Llama, Mistral) = self-hostable, fine-tunable, no per-token API cost — but you own GPU infra, ops, and security. Most production systems use a mix: closed API for the hard reasoning, open weights for high-volume/sensitive tasks.

Key principle — right-size the model: Don't use a frontier model for everything. In a multi-agent system, the router uses a fast/cheap model (Haiku 4.5), the main agent uses the workhorse (Sonnet 5), and only the hardest reasoning steps escalate to frontier (Opus 4.8 / Fable 5). This can cut costs 60–80% versus using frontier everywhere.

03 — The Customization Ladder

The single most important strategic decision: how do you make a general model perform well on your specific task? Three approaches, tried in this order.

Fig 03.1 — The customization ladder (try from bottom → top)

Rung	What it changes	Tag
Fine-tuning	Change the model's behavior by training on your data — tone, format, domain reasoning. Most expensive, slowest to iterate.	behavior
RAG	Give the model knowledge at runtime by retrieving relevant docs. Doesn't change the model, changes what it sees.	knowledge
Prompt engineering	Change the instructions — system prompt, examples, context structure. Cheapest, fastest to iterate.	instructions

The golden rule: Start with prompt engineering. Add RAG when you need knowledge the model doesn't have. Fine-tune only when behavior still isn't right after the first two. Most teams overshoot — they fine-tune when RAG would've solved it, or build RAG when better prompting was enough. The practical default for most production systems in 2026 is prompt engineering + RAG.

	Prompt engineering	RAG	Fine-tuning
Changes	Instructions to the model	Knowledge the model sees	Model weights / behavior
Setup time	Hours to days	Days to weeks	Weeks to months
Cost	~$0 (prompt design)	$70–1000+/mo (infra)	$1000+ (training) + higher inference
Data freshness	Static (in prompt)	Live (retrieves at runtime)	Frozen at training time
Best for	Tone, format, simple tasks	Knowledge grounding, docs, enterprise data	Domain style, specialized reasoning, latency
Iteration speed	Minutes	Hours (re-index)	Days (re-train)

The decision flowchart

Fig 03.2 — Which customization approach?

Can prompt engineering alone reach acceptable accuracy? → Yes → Ship it. Iterate prompts.
Does the model need knowledge it doesn't have (your docs, live data, domain facts)? → Yes → Add RAG
Is the model's behavior / tone / format still wrong after prompt + RAG? → Yes → Fine-tune
Do you need maximum control + data sovereignty + lowest per-token cost at scale? → Yes → Fine-tune open weights + self-host

04 — Prompt Engineering (Production Patterns)

Core techniques

Technique	What it does	Use when
System prompt design	Define persona, role, constraints, output format	Every production system — the foundation of all customization
Few-shot examples	Show 2–5 input/output pairs to anchor behavior	Inconsistent output; need specific tone/format/length
XML/structured tags	Wrap context in tags (`<document>`, `<instructions>`)	Complex prompts with multiple context sources
Chain-of-thought	Ask model to reason step-by-step before answering	Complex reasoning, math, multi-step logic
Prefilled response (legacy)	Start the assistant's response to steer its format	Older models only — most 2026 frontier models reject assistant prefill; use structured outputs instead
Self-critique / reflection	Model evaluates its own output, then revises	High-quality generation; catch hallucinations

The PRECISE framework (community mnemonic)

Persona · Role · Explicit instructions · Context · Instructions · Steps · Examples — a community mnemonic (not an official Anthropic framework) for structuring system prompts. It lines up well with Anthropic's actual published guidance: give clear and direct instructions, use multishot examples, wrap context in XML tags, let the model reason with chain-of-thought, and constrain the output with a schema. (Prefilling the assistant's reply to force a format was a common older trick, but most current frontier models reject a prefilled turn — reach for structured outputs instead.) The Examples component is the single most impactful lever: without examples, the model guesses format and tone from pretraining.

Structured output patterns

Tool-use extraction: define a "tool" whose input schema matches your desired JSON structure, force Claude to "call" it → schema-validated output. Production-grade.
Schema-constrained output: point the model at a full JSON Schema the response must satisfy (a response_format / output_config.format field). Guarantees parseable, schema-valid output — the modern replacement for prefilling a { to coax JSON. Older "JSON mode" only guarantees valid-JSON-shaped text, not your schema.
Validation-retry loop: parse output → validate against schema → if invalid, send error back to model with the original input → retry (cap at 2–3).

Trade-off — strict schema vs free-form: Structured (tool_use / JSON Schema): deterministic, parseable, production-safe — but constrains the model. Free-form text: more natural, flexible — but requires post-processing and can't be reliably parsed. Default to structured for any machine-consumed output.

05 — RAG Architectures

RAG (Retrieval-Augmented Generation) is the dominant enterprise AI pattern in 2026 — grounding LLM responses in your own data. But "RAG" is no longer one thing; there are 5+ distinct architectures with very different complexity/accuracy trade-offs.

The RAG maturity ladder

Fig 05.1 — RAG architectures (simpler → more capable)

Architecture	How it works	Tag
1. Naive RAG	Embed query → vector search → stuff top-K chunks into prompt → generate. Baseline; works for simple Q&A.	start here
2. Hybrid + rerank	Combine vector search + keyword (BM25) search → reranker scores results → top chunks to LLM. Much better precision.	production baseline
3. Self-RAG	Model decides whether to retrieve, evaluates relevance of chunks, may re-retrieve. Reduces noise.	adaptive
4. Graph RAG	Knowledge graph + vector search. Enables multi-hop reasoning over connected entities (contracts, case law, org charts).	relationships
5. Agentic RAG	Agent decomposes query into sub-queries, decides which retrieval tools to call, iterates until it has enough context. Most capable.	complex reasoning

The 60/30/10 rule: In a typical enterprise system, ~60% of queries are simple enough for Naive or Hybrid RAG. ~30% benefit from reranking or Self-RAG. Only ~10% need Graph or Agentic RAG. Adaptive RAG routes each query to the cheapest architecture that can handle it — this is the cost-efficient production pattern.

Fig 05.2 — Hybrid RAG pipeline (production baseline)

flowchart LR
    A["User query"] --> B["Embed query<br/>(embedding model)"]
    B --> C["Vector search<br/>(dense retrieval)"]
    A --> D["Keyword search<br/>(BM25 / sparse)"]
    C --> E["Reranker<br/>(cross-encoder)"]
    D --> E
    E --> F["Top-K chunks"]
    F --> G["LLM generates<br/>(prompt + context)"]
    G --> H["Answer + citations"]

Fig 05.3 — Agentic RAG (most capable)

flowchart TD
    A["Complex query"] --> B["Agent decomposes<br/>into sub-queries"]
    B --> C["Retrieves from<br/>multiple sources"]
    C --> D{"Enough context?"}
    D -->|"gaps remain"| C
    D -->|"yes"| E["Synthesizes<br/>grounded answer"]

Fig 05.4 — Graph RAG (relationship reasoning)

flowchart TD
    A["Query about<br/>connected entities"] --> B["Knowledge graph<br/>(entity + relation lookup)"]
    A --> C["Vector search<br/>(supporting docs)"]
    B --> D["Multi-hop reasoning<br/>(traverse relationships)"]
    C --> D
    D --> E["Answer with<br/>relationship chain"]

RAG pipeline components

Fig 05.5 — Document ingestion pipeline (offline / indexing)

flowchart LR
    A["Source docs<br/>(PDF, HTML, DB)"] --> B["Parse / extract<br/>(text + tables + images)"]
    B --> C["Chunk<br/>(semantic / recursive)"]
    C --> D["Embed<br/>(embedding model)"]
    D --> E["Store<br/>(vector DB + metadata)"]

Component	Options (2026)	Decision
Embedding model	Voyage 3.5, Cohere Embed v4, OpenAI text-embedding-3-large, Qwen3-Embedding	Domain-specific fine-tuned embeddings add 5–15% accuracy for specialized corpora
Vector database	Qdrant, Pinecone, Weaviate, Chroma, pgvector	Qdrant for greenfield; pgvector if already on Postgres; Pinecone for managed simplicity
Chunking strategy	Fixed-size, semantic, recursive, document-structure-aware	Structure-aware (respect headings, paragraphs) > fixed-size. Chunk size 256–512 tokens typical.
Reranker	Cohere Rerank, Voyage Rerank, cross-encoder models	Always add a reranker for production — cheap uplift in relevance
Retrieval method	Dense (vector), Sparse (BM25), Hybrid (both)	Hybrid is the 2026 production baseline — catches what pure vector misses

Trade-off — chunk size: Small chunks (128–256 tokens): more precise retrieval, but lose surrounding context. Large chunks (512–1024): more context per result, but may retrieve irrelevant noise. In practice, 256–512 with overlap is the sweet spot. Some systems use "parent-child" chunking — retrieve the small chunk, then expand to include its parent context.

06 — Fine-Tuning

Fine-tuning changes the model's weights — it's the most powerful customization but the most expensive and slowest to iterate. Use it only when prompt engineering + RAG can't achieve the behavior you need. (This section is the architectural summary; I go through the methods, data requirements, and evaluation in detail in model training, fine-tuning and evaluation.)

When to fine-tune (and when not to)

Fine-tune when:

You need a specific tone, style, or format consistently
Domain-specific reasoning patterns that prompting can't reliably produce
You need to reduce latency (smaller fine-tuned model > larger general model)
You have large volume and need to cut per-token cost by using a smaller model
You need data sovereignty and will self-host open weights

Don't fine-tune when:

You need knowledge — use RAG instead (fine-tuning "bakes in" data that goes stale)
You're still exploring the product — iteration speed matters more than optimization
You have less than 100 high-quality examples — insufficient for meaningful training
The issue is prompt quality, not model capability — fix the prompt first
The model already does it well with good prompting — fine-tuning adds complexity for no gain

Fine-tuning techniques

Technique	What it changes	Data needed	Use when
Full fine-tuning	All model weights	10K+ examples	Deep domain adaptation; requires significant compute
LoRA / QLoRA	Low-rank adapter layers only	100–1000 examples	Most practical for production; fast training, small adapters
RLHF / DPO	Alignment / preference	Preference pairs	Safety alignment, output quality improvement
Distillation	Trains small model to mimic large model	Large model outputs	Cost reduction: train Haiku-class to act like Opus-class on your domain

The hybrid pattern (2026 default): The best-performing production systems combine all three: fine-tuned model (behavior + domain tone) → RAG (current knowledge at runtime) → prompt engineering (per-request orchestration and guardrails). Each layer addresses a different failure mode.

Fig 06.1 — The hybrid production pattern

flowchart LR
    A["User request"] --> B["Prompt engineering<br/>(system prompt + guardrails)"]
    B --> C["RAG retrieval<br/>(current knowledge)"]
    C --> D["Fine-tuned model<br/>(domain behavior)"]
    D --> E["Response<br/>(grounded + on-brand)"]

07 — Agentic AI Patterns

An agent is an LLM that operates in a loop — observe, think, act, repeat — rather than answering one question and stopping. It can use tools, make decisions, and take multi-step actions toward a goal.

When to use agentic vs simpler patterns

Fig 07.1 — Complexity spectrum

Single question → single answer, no tools needed? → Simple prompt (no agent)
Fixed multi-step pipeline with no dynamic decisions? → Workflow / chain (deterministic)
Dynamic decisions + tool use + variable steps? → Single agent with tools
Multiple specialized capabilities + coordination? → Multi-agent system

Trade-off — agent vs workflow: Workflow (chain): deterministic, predictable, easy to debug, lower cost — but can't handle dynamic/unknown paths. Agent: flexible, handles ambiguity, can reason about next steps — but unpredictable, harder to debug, more expensive. Default to the simplest pattern that handles the task. Don't build an agent when a chain would do.

The agentic loop

1. Send request (prompt + tools + history)
2. Model responds
3. Check stop_reason
4. Execute tool(s)
5. Append results
⤴ Loop

stop_reason === "tool_use" → execute tools → loop back.
stop_reason === "end_turn" → terminate. This is the only reliable termination signal.
Every iteration: append the assistant response and tool results to conversation history, so the model can reason about the cumulative state.

What's new in agent loops (2026)

That hand-written loop still describes what happens under the hood, but in 2026 you rarely write it yourself. Four shifts changed how loops are built:

Managed loop runners. SDKs now ship a "tool runner" that owns the observe → act → loop cycle: you supply the tool functions, it calls the model, executes tools, feeds results back, and stops on end_turn. You keep per-iteration hooks — approval gates, logging, result rewriting — without hand-rolling control flow.
Budget-aware loops. Instead of an unbounded loop, hand the agent a token budget for the whole task. The model sees a running countdown and paces itself — prioritizing, then wrapping up gracefully — rather than being cut off mid-thought. This is distinct from a hard per-response cap the model can't see.
Context management inside the loop. Long loops overflow the context window. Two complementary tools keep them alive: context editing prunes stale tool results and reasoning from the transcript, and compaction summarizes earlier turns into a compact block. Together they let an agent run for hours without hitting the limit.
Server-managed agents. The newest option removes the loop from your code entirely: the provider runs the agent loop and hosts a sandboxed container where the tools execute (bash, files, code). You define the agent once, stream events, and answer any custom-tool calls — the right fit for long-running, scheduled, or stateful agents.

One loop mechanic worth knowing regardless: when a server-side tool (web search, code execution) runs many internal iterations, a turn can stop with a pause signal instead of finishing. That is a checkpoint, not a termination — you resume by sending the conversation back unchanged.

Rule of thumb: reach for the managed runner first; add budgets and context management when loops get long; graduate to a server-managed agent only when you need hosted execution or persistence.

Agent frameworks (2026)

Framework	Best for	Provider
Claude Agent SDK	Production agents on Claude; hooks, subagents, session management	Anthropic
LangGraph	Complex stateful workflows with conditional branching, checkpointing	LangChain
AutoGen	Multi-agent conversation, research-oriented architectures	Microsoft
CrewAI	Quick multi-agent prototyping, role-based agents	Community
OpenAI Agents SDK	Handoff-based agent routing, OpenAI ecosystem (the older Assistants API is being sunset in favor of the Responses API / Agents SDK)	OpenAI

08 — Multi-Agent Systems

When a single agent isn't enough, you coordinate multiple specialized agents. Here are the 6 proven production patterns.

Pattern	How it works	Use when
1. Router (classifier)	Lightweight classifier directs each request to a specialized agent	Many distinct intents; each handled by a different specialist
2. Planner-executor	Planner decomposes goal into steps; executor agents run steps in sequence/parallel	Complex tasks requiring dynamic decomposition
3. Tool-use agent	Single agent with a toolbox; LLM decides which tool to call when	Moderate complexity; the agent can reason about tool selection
4. Generator-critic	Primary agent generates output; critic agent verifies; ships only if critic passes	High-quality output; code review, compliance checking
5. Supervisor-workers	Manager owns the goal; workers own subtasks; manager delegates and synthesizes	Enterprise workflows, regulated environments
6. Parallel + judge	Multiple agents work the same problem in parallel; a judge picks the best output	Creative tasks, diverse approaches, consensus needed

Critical design principles

Fig 08.1 — Router pattern (classifier → specialists)

flowchart TD
    A["User request"] --> B["Router / classifier<br/>(Haiku 4.5 — fast, cheap)"]
    B --> C["Billing agent"]
    B --> D["Tech support agent"]
    B --> E["Returns agent"]

Fig 08.2 — Planner-executor pattern

flowchart LR
    A["Goal"] --> B["Planner agent<br/>(decomposes into steps)"]
    B --> C["Step 1 executor"]
    C --> D["Step 2 executor"]
    D --> E["Step N executor"]
    E --> F["Planner synthesizes"]

Fig 08.3 — Generator-critic (quality gate)

flowchart TD
    A["Task"] --> B["Generator agent<br/>(produces output)"]
    B --> C["Critic agent<br/>(evaluates / verifies)"]
    C -->|"pass"| D["Ship"]
    C -->|"fail"| E["Revise"]
    E --> B

Fig 08.4 — Supervisor-workers (enterprise)

flowchart TD
    S["Supervisor<br/>(owns goal + synthesizes)"] --> A["Worker A<br/>(subtask 1)"]
    S --> B["Worker B<br/>(subtask 2)"]
    S --> C["Worker C<br/>(subtask 3)"]

Context isolation: each agent gets only the context it needs — never share the coordinator's full history (prevents context pollution).
Structured handoffs: pass JSON between agents, not raw text — preserves attribution and structure.
Minimal footprint: each agent scoped to a single responsibility and returns a focused result.
Right-size models: router = Haiku 4.5; workers = Sonnet 5; hardest reasoning = Opus 4.8 / Fable 5.
Observability: log every agent call, tool use, and decision at the coordinator level. You cannot debug what you cannot see.

Common mistake: Building a multi-agent system when a single agent with tools would suffice. Multi-agent adds latency, cost, and debugging complexity. Start with the simplest architecture that works, and only add agents when you hit a clear capability ceiling.

09 — Tool Use & MCP

Tool use — what it is

Tool use lets the model call functions you define. The model doesn't execute the function — it outputs a structured request (function name + arguments), your code executes it, and you send the result back. This gives the model capabilities beyond text generation: search, calculation, API calls, database queries.

MCP (Model Context Protocol)

An open standard for connecting AI models to external tools and data sources — "USB-C for AI." Instead of each app building custom integrations, MCP provides a standard protocol.

MCP primitive	What it does	Direction
Tools	Functions the model can call (actions)	Model → server (write/execute)
Resources	Data the model can read (files, records, URIs)	Server → model (read-only)
Prompts	Reusable prompt templates exposed by the server	Server → model (templates)

Tool description quality matters most: The model uses the tool's description to decide which tool to call. Vague descriptions ("handles data") lead to misrouting. Write descriptions with distinct verbs, narrow scope, and clear use cases. Test with production queries and log selection accuracy.

10 — LLMOps & Evaluation

LLMOps is the discipline of deploying, monitoring, and improving LLM-based systems in production. It's MLOps adapted for the non-deterministic, prompt-driven world — which I cover as its own subject in MLOps and AI production operations.

The LLMOps lifecycle

Develop (prompts, RAG, agents) → Evaluate (test sets, benchmarks)
→ Deploy (gateway, versioning) → Monitor (logs, quality, cost)
→ Improve (feedback loops) → ⤴ Iterate

Evaluation (the hardest part)

Eval type	How it works	When
Human evaluation	Domain experts rate outputs on rubrics (accuracy, helpfulness, safety)	Gold standard; expensive; use for benchmarking
LLM-as-judge	A stronger model evaluates a weaker model's output against criteria	Scalable proxy for human eval; useful for regression testing
Automated metrics	BLEU, ROUGE, exact match, retrieval precision/recall	Coarse signal; best for RAG retrieval quality, not generation quality
A/B testing	Route % of traffic to new version, measure business outcomes	Production; the only eval that matters is real-user impact

Model gateway pattern

A centralized proxy between your application and LLM providers. It handles: rate limiting, token budget tracking, multi-provider failover, prompt/response logging, cost attribution, and policy enforcement. Every production system should have one.

Cost management levers: Prompt caching (90% cheaper cached reads) · Batch API (50% cheaper, async) · Model right-sizing (Haiku 4.5 for routing) · Token budgets (per-user, per-team) · Progressive summarization (reduce context growth). A well-run system can cut LLM costs 40–70% vs naive implementation.

11 — AI Safety & Guardrails

Defense in depth — layered guardrails

Fig 11.1 — Guardrail layers

Layer	Controls
Input guardrails (before the model)	prompt injection detection, PII scrubbing, topic filtering, rate limiting
Model-level controls	system prompt constraints, operator permissions, tool restrictions, response prefilling
Output guardrails (after the model)	content classification, schema validation, PII detection on output, citation verification
Action guardrails (before execution)	tool-call validation (hooks), human-in-the-loop gates, irreversible-action blocks, spending limits

Trade-off — programmatic vs prompt-based guardrails: Programmatic (hooks, code): deterministic, can't be jailbroken, auditable — use for hard safety (PII, compliance, permissions). Prompt-based: flexible, natural-language, covers nuance — but can be circumvented by adversarial input. Layer both: prompts handle the soft cases, code handles the hard constraints.

Key safety concepts

Prompt injection: adversarial input that overrides system instructions. Defense: input validation + output verification + model-level resistance + separation of trusted/untrusted content.
Hallucination mitigation: RAG grounding + citation requirements + self-evaluation + nullable fields in schemas (model can say "I don't know").
Human-in-the-loop: mandatory approval gate before irreversible or high-value actions (financial transactions, data deletion, external communications). Design the escalation criteria explicitly.
Red teaming: adversarial testing by humans or automated tools to find failure modes before deployment. Run regularly, not just once.

12 — Enterprise AI Governance

Data governance: RAG is only as good as the data it retrieves. Enforce access controls on retrieval (document-level and row-level ACLs). Don't let the model see data the user shouldn't see.
Compliance: regulated industries (healthcare, finance, government) need audit trails of every prompt, response, tool call, and decision. Log everything through the model gateway.
Data residency: know where your prompts are processed. Closed APIs send data to the provider's region. For sovereignty, self-host open-weight models or use provider regions that match your jurisdiction.
Cost governance: set token budgets per team/project; alert on anomalies; review cost-per-outcome, not just cost-per-token.
Model risk management: treat LLMs as you'd treat any critical vendor: track model versions, test for regression on updates, have a rollback plan, maintain a "model bill of materials."

13 — Infrastructure & Serving

Inference infrastructure

Approach	Use when	Trade-off
API providers (Anthropic, OpenAI, Google)	Default. Zero infra, pay per token, highest capability	Vendor dependency, data leaves your control
Self-hosted (vLLM, TGI, Ollama)	Data sovereignty, custom models, predictable cost at scale	You own GPU infra, ops, scaling, security
Managed serving (AWS Bedrock, GCP Vertex, Azure AI)	Cloud-provider ecosystem, compliance, hybrid of API + control	Cloud lock-in, sometimes behind on latest models

GPU/TPU landscape

NVIDIA H100/H200: the workhorse for LLM training and inference; most available, broadest framework support.
NVIDIA B100/B200 (Blackwell): next-gen, higher throughput; becoming available 2025–2026.
Google TPUs (v5e/v6e): competitive for training large models on GCP; Jax/TensorFlow-centric.
AMD MI300X: growing alternative; good cost/performance for inference.

Trade-off — API vs self-host: API wins when: you're under ~$50K/mo in inference, you want the latest models immediately, you don't have GPU ops expertise. Self-host wins when: you're spending $100K+/mo (amortized GPU cost is lower), you need data sovereignty, or you've fine-tuned open weights. Most enterprises use both — API for frontier reasoning, self-hosted for high-volume / sensitive tasks.

14 — Multimodal AI

Modern AI systems aren't text-only. Multimodal models process images, audio, video, and code alongside text.

Modality	Models (2026)	Use cases
Vision (image → text)	Claude Sonnet 5 / Opus 4.8, GPT-5, Gemini 3 Pro	Document understanding, UI analysis, image description, OCR replacement
Image generation	GPT-image-1, Midjourney, Stable Diffusion 3, Imagen 4	Creative assets, product mockups, data visualization
Audio / speech	Whisper (STT), GPT-5 (native audio), ElevenLabs (TTS)	Transcription, voice agents, real-time conversation
Video	Gemini (native video), Sora 2, Runway Gen-4, Google Veo 3	Video understanding, generation, editing
Code	Claude (Opus 4.8 / Sonnet 5), GPT-5, Codex, DeepSeek-Coder	Code generation, review, debugging, migration

Multimodal RAG

Standard RAG embeds text. Multimodal RAG also embeds images, tables, and diagrams from documents (using vision models or multimodal embeddings like CLIP/Jina CLIP). This is critical for document-heavy industries (legal, medical, manufacturing) where important information lives in charts, forms, and images — not just text.

15 — Cloud AI Platforms — GCP / AWS / Azure

Knowing the patterns (§03–§08) is half the job. The other half is knowing which cloud services implement them for a customer deployment. This section maps every AI architecture pattern to its concrete service on each cloud, plus the architect-level trade-offs for platform selection.

The three platforms at a glance

	GCP	AWS	Azure
Primary AI platform	Vertex AI / Gemini Enterprise Agent Platform	SageMaker + Bedrock	Azure AI Foundry + Azure ML
Foundation model access	Gemini (native), Llama, Gemma via Model Garden	Claude, Llama, Mistral, Cohere via Bedrock; GPT via custom	GPT-5 via Azure OpenAI (exclusive); Llama, Mistral via Model Catalog
Unique hardware	TPUs (v5e/v6e) — best $/FLOP for large training	Trainium/Inferentia — purpose-built inference chips	Maia AI accelerators (emerging)
Best for	Data in BigQuery, Gemini models, TPU training, opinionated managed workflows	Deepest flexibility, broadest ecosystem, Bedrock for Claude/multi-model	Microsoft-centric orgs, Azure OpenAI (GPT), compliance/hybrid
Sweet spot	Research, startups, warehouse-native ML	Expert teams, fine-grained control, large inference	Enterprises on Microsoft stack, regulated industries

Service mapping — pattern → cloud service

AI architecture pattern	GCP	AWS	Azure
Model training (custom)	Vertex AI Training + TPUs	SageMaker Training + p5/Trainium	Azure ML Compute + ND-series GPUs
Fine-tuning (managed)	Vertex AI Model Tuning	SageMaker JumpStart fine-tuning / Bedrock custom models	Azure AI Foundry fine-tuning / Azure OpenAI fine-tuning
Model serving / inference	Vertex AI Endpoints (dedicated + serverless)	SageMaker Endpoints (real-time, serverless, async, multi-model)	Azure ML Managed Endpoints (online + batch)
Foundation model API (hosted)	Vertex AI Model Garden / Gemini API	Amazon Bedrock	Azure OpenAI Service / Azure AI Model Catalog
RAG — vector search	Vertex AI Vector Search / AlloyDB pgvector	Amazon OpenSearch / Bedrock Knowledge Bases / Aurora pgvector	Azure AI Search (vector mode) / Cosmos DB vCore
RAG — embedding	Vertex AI Embeddings (Gemini) / text-embedding	Bedrock Embeddings (Titan / Cohere)	Azure OpenAI Embeddings (ada-002/3-large)
RAG — orchestration	Vertex AI Agent Builder	Bedrock Knowledge Bases + Agents	Azure AI Foundry Prompt Flow
MLOps pipeline	Vertex AI Pipelines (Kubeflow)	SageMaker Pipelines	Azure ML Pipelines
Experiment tracking	Vertex AI Experiments	SageMaker Experiments / MLflow on SageMaker	Azure ML + MLflow
Model registry	Vertex AI Model Registry	SageMaker Model Registry	Azure ML Model Registry
Feature store	Vertex AI Feature Store	SageMaker Feature Store	Azure ML Managed Feature Store
Data labeling	Vertex AI Data Labeling	SageMaker Ground Truth	Azure ML Data Labeling
AutoML	Vertex AI AutoML	SageMaker Autopilot	Azure AutoML
AI safety / guardrails	Model Armor + VPC-SC	Bedrock Guardrails	Azure AI Content Safety
Agent framework	Agent Builder / Vertex AI Agent	Bedrock Agents	Azure AI Agent Service
Notebooks	Vertex AI Workbench (managed JupyterLab)	SageMaker Studio	Azure ML Notebooks

Fig 15.1 — Cloud AI training & inference pipeline (generic, all clouds)

flowchart LR
    A["Data<br/>(lake / warehouse)"] --> B["Feature store<br/>(prep + versioning)"]
    B --> C["Training job<br/>(GPU/TPU cluster)"]
    C --> D["Model registry<br/>(versioning + metadata)"]
    D --> E["Serving endpoint<br/>(real-time / batch)"]
    E --> F["Monitoring<br/>(drift + performance)"]

Platform selection decision tree

Fig 15.2 — Which cloud for your AI workload?

Data gravity: where does most of your data already live? → S3 → AWS · BigQuery → GCP · Azure Data Lake → Azure
Need Azure OpenAI (GPT-5)? → Azure (exclusive enterprise GPT access)
Need Claude via managed service? → AWS Bedrock (deepest Claude integration)
Need Gemini or TPU training? → GCP Vertex AI
Microsoft-centric org (Entra, M365, Power BI)? → Azure ML
Need maximum flexibility + deepest service catalog? → AWS SageMaker
Greenfield team wanting opinionated, fast-to-start pipelines? → GCP Vertex AI

The data gravity rule: The single strongest predictor of platform choice is where the data already lives. Moving petabytes between clouds is expensive and slow. If your data is in BigQuery, build on Vertex AI. If it's in S3, build on SageMaker. If it's in Azure Data Lake or SQL Server, build on Azure ML. Everything else is secondary.

Training infrastructure trade-offs

Factor	GCP	AWS	Azure
Best GPU value	TPU v5p ~$4.20/hr (best $/FLOP for large training)	H100 via p5 ~$8.60–10.80/hr; Trainium for AWS-optimized workloads	H100 via ND-series; best through EA negotiated pricing
Spot / preemptible	Spot VMs up to ~91% off (good for training)	Spot Instances (~60–90% off); SageMaker Managed Spot Training	Spot VMs available but less GPU spot capacity
Distributed training	TPU pods (scale natively); GPU with NCCL	SageMaker HyperPod (auto fault recovery, 99.9% uptime in 6-week runs)	Azure ML distributed training with DeepSpeed/FSDP
Cost savings	Sustained Use Discounts (auto); CUDs	SageMaker Savings Plans (up to 64%)	Enterprise Agreement negotiated rates

Inference infrastructure trade-offs

Pattern	GCP	AWS	Azure
Real-time (always-on)	Vertex AI Dedicated Endpoints (~$7/day min)	SageMaker Real-time Endpoints	Azure ML Managed Online Endpoints
Serverless (scale-to-zero)	Vertex AI Serverless Prediction (limited model types)	SageMaker Serverless Inference (scales to zero — dev/low-traffic)	Azure ML Serverless (newer)
Batch	Vertex AI Batch Prediction	SageMaker Batch Transform	Azure ML Batch Endpoints
Multi-model hosting	Custom containers (manual routing)	SageMaker Multi-Model Endpoints + Inference Components (up to 80% savings)	Custom containers
Async inference	Custom (Pub/Sub + Cloud Run)	SageMaker Async Inference (native, built-in)	Custom (Service Bus + Azure Functions)
Foundation model API	Vertex AI / Gemini API (pay per token)	Bedrock (pay per token; 50% batch discount)	Azure OpenAI (pay per token; provisioned throughput for steady load)

Trade-off — managed platform vs self-hosted on cloud: Managed platform (SageMaker/Vertex/Azure ML) = faster to ship, less ops, built-in MLOps — but 20–40% more expensive than raw compute (e.g., SageMaker ml.* instances cost more than equivalent EC2). Self-hosted on raw VMs (EC2 + your own serving stack like vLLM) = cheaper at scale, full control — but you own infrastructure, scaling, monitoring. Use managed for most workloads; go self-hosted only if you have a dedicated MLOps team and the cost savings justify it.

Multi-cloud AI patterns

Train on one cloud, infer on another: train where GPUs/TPUs are cheapest (GCP TPUs for large models), deploy inference where your app already runs (AWS/Azure). Transfer the trained model weights, not the training data.
AI Gateway pattern: a proxy layer that routes LLM API calls to the cheapest/fastest/most-available provider (Claude on Bedrock, GPT on Azure, Gemini on Vertex) with automatic failover. Emerging standard for multi-cloud LLM ops.
Portable MLOps: standardize on vendor-neutral tools (MLflow for tracking, KubeFlow for pipelines, ONNX for model format) to reduce lock-in. Managed services are faster but create deeper coupling.

Fig 15.3 — Multi-cloud AI gateway pattern

flowchart TD
    A["Application"] --> G["AI Gateway<br/>(routing, failover, budget)"]
    G --> B["AWS Bedrock<br/>(Claude, Llama)"]
    G --> C["GCP Vertex AI<br/>(Gemini, Gemma)"]
    G --> D["Azure OpenAI<br/>(GPT-5)"]

16 — Trade-off Master Reference

Every architectural decision an AI architect makes, in one table.

Decision	Option A	Option B	Default
Customization approach	Prompt engineering	RAG → Fine-tuning	Start with prompts, escalate
Model tier	Frontier (most capable)	Fast/cheap (cost-efficient)	Right-size per task
Model source	Closed API	Open weights (self-host)	API unless sovereignty/cost forces self-host
RAG vs fine-tune	RAG (knowledge at runtime)	Fine-tune (behavior change)	RAG for knowledge, fine-tune for behavior only
Agent vs workflow	Deterministic workflow/chain	Dynamic agent loop	Simplest that works
Single vs multi-agent	One agent + tools	Multi-agent coordination	Single until you hit capability ceiling
Sequential vs parallel	Sequential (steps depend on prior)	Parallel (independent subtasks)	Sequential for dependent; parallel for independent
Guardrails	Prompt-based (soft)	Programmatic hooks (hard)	Both — prompts for nuance, code for safety
Structured vs free-form output	Schema-validated JSON	Natural language text	Structured for machine consumption
Chunk size (RAG)	Small (128–256 tokens)	Large (512–1024 tokens)	256–512 with overlap
Retrieval method	Vector (dense) only	Hybrid (vector + keyword)	Hybrid is 2026 production baseline
Evaluation	Automated metrics	Human evaluation	LLM-as-judge for scale, human for gold standard
Batch vs real-time	Batch API (50% cheaper)	Real-time Messages API	Batch for bulk ETL; real-time for user-facing
Context strategy	Full history (precise)	Progressive summarization (efficient)	Summarize when context exceeds 50% of window
Caching	Prompt cache (90% cheaper reads)	No cache	Cache anything repeated across turns
Cost control	Per-token optimization	Per-outcome optimization	Measure cost per successful outcome, not just tokens
Cloud AI platform	Managed (SageMaker/Vertex/Azure ML)	Self-hosted on raw compute	Managed unless dedicated MLOps team + cost savings justify self-hosting
Training hardware	GPUs (H100, universal)	TPUs (GCP) / Trainium (AWS)	GPUs for flexibility; TPU/Trainium for cost at large scale
Foundation model source	Cloud-managed API (Bedrock/Vertex/Azure OpenAI)	Direct vendor API (Anthropic/OpenAI)	Cloud-managed for governance + VPC; direct for latest models + flexibility
Multi-cloud vs single	Single cloud (simpler ops)	Multi-cloud (avoid lock-in)	Single unless regulatory/cost/model-access forces multi-cloud

The architect's mental model: Every decision above is a dial, not a switch. The expert architect doesn't pick "always A" or "always B" — they tune each dial per use case, per constraint, and per organizational context. The table gives you the defaults; the skill is knowing when to deviate.

17 — Books & Learning Path

Organized as a reading path — start at the foundation and work up. Each book is mapped to the guide sections it reinforces. The goal is depth and coherence, not breadth — these are the ones practitioners actually reference.

Tier 1 — Foundations (start here)

Book	Author(s)	Covers	Maps to
Build a Large Language Model (From Scratch)	Sebastian Raschka · Manning, 2025	Attention, tokenization, pretraining, fine-tuning, LoRA — builds a GPT-like model step by step. Understand what's inside the black box.	§02, §06
Hands-On Large Language Models	Jay Alammar & Maarten Grootendorst · O'Reilly, 2024	Embeddings → RAG → fine-tuning in one accessible flow. Great visual explanations of transformer internals.	§02, §05, §06
Designing Machine Learning Systems	Chip Huyen · O'Reilly, 2022	The ML lifecycle: data, modeling, deployment, monitoring, iteration. The principles transfer directly to LLM systems.	§10, §12

Tier 2 — Production AI engineering

Book	Author(s)	Covers	Maps to
AI Engineering	Chip Huyen · O'Reilly, 2025	The full stack: evaluation, prompt design, RAG, agent architectures, deployment trade-offs. Arguably the single best production LLM book.	§03–§10
The LLM Engineer's Handbook	Paul Iusztin & Maxime Labonne · Packt, 2024	End-to-end project: RAG, fine-tuning, vector DBs, evaluation, deployment, observability, cost optimization.	§05, §06, §10, §13
Building LLMs for Production	Louis-François Bouchard & Louie Peters · Towards AI, 2024	Practical RAG techniques (vanilla through GraphRAG), prompt engineering, agents, fine-tuning, deployment. Code-heavy.	§04, §05, §06, §07

Tier 3 — Agents & multi-agent systems

Book	Author(s)	Covers	Maps to
Designing Multi-Agent Systems	Victor Dibia · independently published, 2025	Principles, patterns, and implementation for AI agents — orchestration patterns, MCP/A2A, evaluation, failure modes, case studies. Framework-agnostic, build from scratch.	§07, §08, §09, §11
Agentic Architectural Patterns for Building Multi-Agent Systems	Ali Arsanjani & Juan Pablo Bustos · Packt, 2026	Hierarchical multi-agent architecture, coordination, explainability, fault tolerance, human-agent interaction. Enterprise-focused.	§08, §11, §12
Building LLM-Powered Applications	Valentina Alto · Packt, 2024	LangChain, agent memory, tool integration, multi-agent architectures, failure handling. Prototype-to-production.	§07, §08, §09

Tier 4 — Specialized depth

Book	Author(s)	Covers	Maps to
Prompt Engineering for LLMs	John Berryman & Albert Ziegler · O'Reilly, 2025	Context as "packets of knowledge"; flexible, scalable prompt systems. Written by a core GitHub Copilot engineer.	§04
Prompt Engineering for Generative AI	James Phoenix & Mike Taylor · O'Reilly, 2024	CoT, ReAct, planning loops, agent behavioral architecture, prompt debugging. Strong on why agents fail.	§04, §07
Building Reliable AI Systems	Rush Shahani · Manning, 2026	Reduce hallucinations, improve performance, manage bias. From prototype to production reliability.	§10, §11
Generative AI Design Patterns	Valliappa Lakshmanan & Hannes Hapke · O'Reilly	32 patterns including RAG, reasoning, generation, evaluation. Pattern catalog for architects.	§04–§08
Machine Learning System Design Interview	Ali Aminian & Alex Xu	ML design problems, feature engineering, scalability, monitoring. System-design thinking for ML/AI.	§10, §12, §13

Tier 5 — Data & infrastructure foundations

Book	Author(s)	Covers	Maps to
Fundamentals of Data Engineering	Joe Reis & Matt Housley · O'Reilly, 2022	Data lifecycle: ingestion, storage, transformation, orchestration, serving. The data layer beneath every AI system.	§05 (data for RAG), §12
Designing Data-Intensive Applications	Martin Kleppmann · O'Reilly, 2017	Distributed systems, consistency, replication, partitioning. Still the GOAT for understanding the infrastructure AI runs on.	§13

Tier 6 — Cloud AI platforms & MLOps

Book	Author(s)	Covers	Maps to
Practical MLOps	Noah Gift & Alfredo Deza · O'Reilly, 2021	MLOps on AWS, Azure, GCP — the cross-cloud operational playbook. AutoML, containers, edge, monitoring. Practical case studies.	§10, §13, §15
MLOps Engineering at Scale	Carl Osipov · Manning, 2022	Serverless ML pipelines on AWS: PyTorch + SageMaker + Lambda + Step Functions. Infrastructure-as-code for ML.	§10, §13, §15 (AWS)
Google Cloud Platform (GCP) MLOps Engineer Handbook	Dilip Kumar Mondal · 2026	Vertex AI, BigQuery, Cloud Build, Pipelines. End-to-end GCP-native ML platform design with drift detection and monitoring.	§10, §15 (GCP)
Hands-On MLOps on Azure	Banibrata De · Packt	Azure ML CLI, GitHub integration, LLMOps, secure and scalable ML workflows on Azure. Enterprise governance focus.	§10, §12, §15 (Azure)
The Machine Learning Solutions Architect Handbook	David Ping · Packt, 2nd ed. 2024	ML lifecycle, system design, MLOps, generative AI from a solutions architect perspective. Cross-cloud strategies and best practices.	§10–§15
AI Systems Performance Engineering	Chris Fregly · O'Reilly, 2025	Optimizing model training and inference with GPUs, CUDA, PyTorch. Hardware-level understanding for architects who need to spec infrastructure.	§13, §15

Free & open resources

Resource	What it is	Maps to
Anthropic Academy (anthropic.skilljar.com)	19 free courses: Claude API, MCP, Claude Code, AI Fluency, Agentic Architecture. Official Anthropic.	§04, §07, §09
Anthropic Cookbook (github.com/anthropics/anthropic-cookbook)	43K+ stars. Production recipes: RAG, tool use, agents, structured output, prompt caching.	§04, §05, §07, §09
DeepLearning.AI short courses	Free courses with Andrew Ng: LangChain, LlamaIndex, RAG, agents, fine-tuning, prompt engineering.	§03–§08
Full Stack AI Engineering (Towards AI Academy)	Louis-François Bouchard's practical course: RAG, agents, fine-tuning, deployment.	§05–§10
LLM Visualization (bbycroft.net/llm)	Interactive 3D visualization of transformer internals. Best single resource for building intuition on how LLMs work.	§02
MCP Specification (modelcontextprotocol.io)	The open standard specification for tool/resource/prompt integration.	§09
Cloud ML Platforms (Coursera — Board Infinity)	Free course: deploy ML on AWS SageMaker, Azure ML, Vertex AI. Practical cross-cloud comparison.	§15
AWS ML Specialty exam guide + free training	AWS Skill Builder free courses covering SageMaker, Bedrock, MLOps on AWS.	§15 (AWS)
Google Cloud Skills Boost — ML Engineer path	Free labs + courses on Vertex AI, BigQuery ML, MLOps on GCP.	§15 (GCP)
Microsoft Learn — Azure AI Engineer path	Free modules on Azure ML, Azure OpenAI, Prompt Flow, responsible AI.	§15 (Azure)

CCAR-F: Claude Certified Architect Foundations — 2026 Blueprint

Sat, 04 Jul 2026 00:00:00 GMT

Everything I'd want a teammate to walk into the Claude Certified Architect exam knowing — the five domains, the scenarios that trip people up, and the reasoning behind each answer.

Questions	Duration	Fee	Pass mark	Scenarios	Proctored
60	120 min	$125 USD	720 / 1000	4 of 6	Yes, closed-book

Exam Format & Domains

Launched March 12, 2026, Claude Certified Architect — Foundations (CCAR-F) was Anthropic's first proctored technical certification. (Early coverage widely wrote it CCA-F; CCAR-F is the official code, and it now sits alongside an Associate, a Developer, and a Professional-tier Architect exam.) It validates that you can design and ship production-grade Claude applications at enterprise scale. Every question is scenario-based — a realistic production system with a problem, and you pick the architecturally correct fix among plausible alternatives. No simple recall questions.

Before you plan around this: the certification is offered through the Claude Partner Network — registration runs via the Anthropic Partner Academy, which validates partner credentials at login. Confirm your organization's eligibility before budgeting exam time.

Five domains & weighting

Domain	Weight	Core concepts
1. Agentic Architecture & Orchestration	27%	Agentic loop, stop_reason, hub-and-spoke, subagents, hooks, task decomposition, session state
2. Tool Design & MCP Integration	18%	MCP servers (resources/tools/prompts), JSON Schema, stdio vs SSE, tool descriptions, tool_choice
3. Claude Code Configuration & Workflows	20%	CLAUDE.md hierarchy, skills, slash commands, plan mode, path-specific rules, CI/CD (-p flag)
4. Prompt Engineering & Structured Output	20%	PRECISE (community mnemonic), few-shot, XML tags, JSON schemas, tool_use for extraction, validation-retry, Batch API
5. Context Management & Reliability	15%	Context window, prompt caching, token budgets, CALM (informal study aid), escalation, error propagation, human-in-the-loop

Scoring: Scaled 100–1000, pass at 720. Domain-weighted — you can't pass by acing one domain and ignoring others. 4 of 6 published scenarios are randomly selected per exam; each provides the context for ~15 questions.

The 6 Exam Scenarios

You get 4 of these 6 on exam day — randomly selected. Study all six so you're not caught off guard. Each is a realistic production system that spans multiple domains.

Scenario 1 — Customer Support Resolution Agent

What it tests: Agent SDK + MCP tools + escalation triggers + error propagation
Key decisions: When to escalate to a human, how to structure the agentic loop, how to handle policy gaps and ambiguity

Scenario 2 — Code Generation with Claude Code

What it tests: CLAUDE.md config + plan mode + slash commands + built-in tools
Key decisions: Autonomous PR review/generation, skills with context: fork, how to structure project instructions

Scenario 3 — Multi-Agent Research System

What it tests: Coordinator-subagent orchestration + context isolation + structured passing
Key decisions: Hub-and-spoke vs pipeline, parallel vs sequential, preventing context pollution

Scenario 4 — Developer Productivity with Claude

What it tests: Built-in tools + MCP servers + tool descriptions + tool distribution
Key decisions: When to use existing MCP servers vs build custom, IDE integration patterns

Scenario 5 — Claude Code for CI/CD

What it tests: Non-interactive mode (-p flag) + structured output + independent review
Key decisions: Headless pipeline design, --output-format json, when to use --json-schema

Scenario 6 — Structured Data Extraction

What it tests: JSON schemas + tool_use for extraction + validation-retry loops + batch processing
Key decisions: Nullable fields to reduce hallucination, single-pass vs multi-pass, Message Batches API

§1 — Agentic Architecture & Orchestration (27%)

The heaviest domain and the one most candidates find hardest. It covers the full lifecycle of the agentic loop, multi-agent coordination, hooks, task decomposition, and session management. What follows is scoped to what the exam asks; for the same patterns treated as design decisions rather than answers, see the AI architecture field guide.

The agentic loop — the single most important concept

The agentic loop lifecycle:

1. Send request (history + tools)
   → 2. Receive response
   → 3. Check stop_reason        ← the key step
   → 4. Execute tools (if tool_use)
   → 5. Append results (to history)
   → Loop back to 1

stop_reason === "tool_use" → Claude wants to call tools → execute them → append results → loop back.
stop_reason === "end_turn" → Claude is finished → terminate the loop.
stop_reason === "pause_turn" → a long server-side-tool turn (web search, code execution) checkpointed instead of finishing → re-send the conversation unchanged to resume. It's a checkpoint, not a termination — don't treat it as "done."
The stop_reason field is a structured API signal — the only reliable termination mechanism.

Anti-patterns (exam favorites):

Parsing natural language to detect "done" / "task complete" — non-deterministic, fails silently.

Arbitrary iteration cap as the primary stopping mechanism — fragile, not semantically meaningful.

Checking assistant text content like response.content[0].text.includes('complete') — brittle string matching.

Multi-agent orchestration — hub-and-spoke

The hub-and-spoke (coordinator-subagent) pattern:

            Coordinator (hub)
           /       |        \
  Web search   Doc analysis   Synthesis
   subagent      subagent      subagent

The coordinator manages all inter-subagent communication, error handling, and routing. Subagents never talk to each other directly.
Context isolation: each subagent gets only the context it needs — never the coordinator's full context (prevents context pollution).
Structured passing: when handing findings between subagents, pass structured data (JSON with source, date, content), not plain text blobs — preserves attribution.
Minimal footprint: subagents should be scoped to a single responsibility and return a focused result.

Anti-pattern: Sharing the coordinator's full conversation history with every subagent. This causes context pollution — subagents get confused by irrelevant context and waste tokens.

Agent SDK hooks

Hooks are programmatic enforcement points — Python/TypeScript functions invoked by the Agent SDK at specific loop points. They are not invoked by Claude (the model never sees them).

Hook type	When it fires	Use for
`Pre-tool`	Before a tool is executed	Validate inputs, check permissions, block dangerous actions
`Post-tool`	After a tool returns	Log results, sanitize output, enforce PII rules
`Pre-message`	Before Claude's response is sent to the user	Content filtering, compliance checks, redaction

Trade-off — hooks vs prompt-based guardrails: Hooks = programmatic, deterministic, enforced by code, can't be jailbroken → use for hard constraints (compliance, PII blocking, permission checks). Prompt instructions = flexible, natural-language guidance, can be circumvented → use for soft guidelines (tone, style, preference). Default to hooks for anything safety-critical.

Task decomposition & execution patterns

Pattern	When to use	Trade-off
Sequential	Steps depend on prior results; order matters	Slower but safer; each step informed by the last
Parallel	Independent subtasks with no dependencies	Faster but costs more tokens; harder to debug
Dynamic planning	Task scope unclear upfront; agent decides next steps	Most flexible but needs guardrails to prevent drift

Session management: Know: fork_session creates an isolated branch (subagent work doesn't pollute the main conversation). Sessions can be resumed with conversation history. State can be in-context (conversation history) or external (database/file); external is more durable for long-running agents.

§2 — Tool Design & MCP Integration (18%)

Tests your ability to design tool interfaces, write schemas Claude can reliably use, and build/integrate MCP servers.

MCP (Model Context Protocol) — three primitives

Primitive	What it is	Analog
`Tools`	Functions Claude can call — input schema, returns a result	POST / RPC
`Resources`	Data Claude can read — files, DB records, URIs	GET / read-only
`Prompts`	Reusable prompt templates exposed by the server	Templates

Tool schema design

JSON Schema defines tool inputs. Every tool needs name, description, and input_schema.
Descriptions are how Claude decides which tool to use. They must be precise, use distinct verbs, and have narrow scope. "Two good matches" and "no good match" are both failures of description quality.
Nullable fields (e.g. "type": ["string", "null"]) reduce hallucination — Claude can explicitly return null instead of inventing a value.

Anti-patterns:

Vague descriptions ("does stuff with data") — Claude can't select the right tool.

Overlapping tools with similar descriptions — Claude oscillates between them.

Missing required fields in the JSON Schema — Claude hallucinates the structure.

Giant monolithic tools that do many things — hard to select, hard to debug.

Transport: stdio vs SSE

stdio (standard I/O):

Subprocess on the same machine
Lower latency, simpler setup
Best for local development, CLI tools

SSE (Server-Sent Events):

HTTP-based, works across network boundaries
Supports real-time streaming
Best for remote/cloud servers, team sharing
Note: the standalone HTTP+SSE transport is deprecated in favor of Streamable HTTP, which is now the recommended remote transport. Treat "SSE" here as shorthand for HTTP-based remote transport.

Scenario trigger: "Stream large file contents across a network" → SSE. "Local subprocess, same machine" → stdio. "Share MCP server across a team" → SSE (or Streamable HTTP).

tool_choice parameter

Value	Behavior	Use when
`auto`	Claude decides whether/which tool to call	Default; most scenarios
`any`	Claude must call a tool (any one)	Force tool use; extraction pipelines
`tool` (specific)	Claude must call this exact tool	Deterministic extraction; structured output
`none`	Claude cannot use any tools	Force text-only response

Structured error responses

Transient errors (network timeout, rate limit) → retry with backoff.
Business errors (record not found, invalid input) → return structured error to Claude so it can reason about next steps.
Permission errors → escalate or inform the user; don't retry silently.

§3 — Claude Code Configuration & Workflows (20%)

Tests your ability to configure Claude Code (Anthropic's agentic CLI/IDE tool) for production projects and team workflows.

CLAUDE.md hierarchy & scoping

Level	Location	Scope
`Enterprise`	Set by admin (operator)	All users in the org — hard constraints
`User / global`	`~/.claude/CLAUDE.md`	Personal preferences across all projects
`Project`	`./CLAUDE.md` (repo root)	Shared team conventions, committed to VCS
`Path-specific rules`	`.claude/rules/*.md` with YAML frontmatter	Activated only when editing matching file paths

Higher levels override lower: Enterprise > User > Project.
An operator (enterprise admin) can set constraints that users cannot override — this is the trust hierarchy.
.claude/rules/ with path-scoped YAML frontmatter is the recommended approach over a monolithic CLAUDE.md — reduces irrelevant context.

Skills, commands & built-in tools

Concept	What it is
`Custom slash commands`	Defined in `.claude/commands/`; reusable workflows invoked by `/command-name`
`Skills (SKILL.md)`	Project- or user-scoped instructions for specific tasks (e.g. "how to create a docx"); `context: fork` in frontmatter runs the skill in an isolated sub-agent
`Built-in tools`	Read, Write, Edit, Bash, Search, List — Claude Code's native file/code tools
`MCP servers in Claude Code`	Configured in `.claude/mcp.json` (project) or `~/.claude/mcp.json` (user)

Scenario trigger: context: fork = the skill runs as an isolated sub-agent, preventing verbose output from polluting the main session. Use it for tasks that generate large output (code generation, data processing).

Execution modes

Mode	How it works	Use when
`Plan mode`	Claude plans the approach before executing; shows the plan for review	Complex tasks; want to review before committing
`Direct execution`	Claude executes immediately	Simple / well-understood tasks
`Non-interactive (-p flag)`	Headless mode, no user prompts; outputs to stdout	CI/CD pipelines; add `--output-format json` for structured output

CI/CD (scenario 5): For CI/CD: use claude -p "review this PR" --output-format json --json-schema schema.json. This gives deterministic, machine-parseable output. Remember: -p = non-interactive / print mode.

§4 — Prompt Engineering & Structured Output (20%)

The PRECISE mnemonic

A community mnemonic — not official Anthropic terminology — for a structured approach to designing system prompts. Know the acronym and what each component does:

Letter	Component	What it does
P	Persona	Who Claude is ("You are a senior code reviewer")
R	Role	The function it performs
E	Explicit instructions	Clear, direct guidance
C	Context	Background information
I	Instructions	Step-by-step task rules
S	Steps	Ordered process to follow
E	Examples	Few-shot demonstrations of ideal output

Key insight: The Examples component is the behavioral anchor — it shows Claude what a correct response looks like in terms of length, tone, and structure. Without examples, Claude interpolates from pre-training, producing variance. This is often the fastest fix for inconsistent output.

Core techniques

Few-shot prompting: 2–3 input/output examples to anchor behavior. Place them after the instructions, before the actual task.
XML tags for context structure: <document>, <instructions>, <example> — Claude respects these boundaries for parsing and retrieval.
Chain-of-thought / extended thinking: for complex reasoning, let Claude think step-by-step before producing the final answer.
Prefilled assistant responses: start the assistant turn with a partial response to steer format/structure. Note: most current frontier Claude models (Opus 4.6+, Sonnet 4.6+, Fable 5) reject a prefilled assistant turn — prefer schema-constrained structured outputs there.

Structured output via tool_use

The exam's preferred pattern for extracting structured data: define a tool whose input_schema matches your desired JSON structure, then set tool_choice to force Claude to call it. This gives you validated, schema-conforming output.

Trade-off — tool_use extraction vs raw JSON prompting: tool_use with JSON Schema = schema-validated, deterministic structure, nullable fields reduce hallucination → preferred for production. Prompting for JSON = simpler to set up, but output can drift from schema, requires post-processing validation → acceptable for prototyping only.

Validation-retry loops

After extraction, validate the output programmatically against the schema.
If validation fails, send the error back to Claude with the original input and ask it to fix the specific issue.
Cap retries (2–3) to avoid infinite loops on truly malformed input.

Batch processing — Message Batches API

For high-volume extraction (hundreds/thousands of documents), use the Message Batches API — asynchronous, 50% cheaper than real-time, 24-hour processing window.
Trade-off: latency (not real-time) for cost savings — ideal for overnight/batch ETL.
Per-item error isolation: if one item in a batch fails, the rest still succeed. Design for this.

§5 — Context Management & Reliability (15%)

The lightest domain by weight but consistently underestimated. These questions are often the easiest marks available once you know the patterns — and the easiest to lose if you don't.

Context window fundamentals

Context window = input tokens + output tokens. Know the model limits: Opus (4.6/4.7/4.8), Sonnet (4.6/5), and Fable 5 = 1M-token context; only Haiku 4.5 = 200K.
System prompt + conversation history + tool definitions + tool results all consume input tokens. They add up fast in agentic loops.
Progressive summarization: as conversation grows, periodically summarize older turns and replace the raw history → keeps context under budget without losing critical information.
Conversation compaction: similar idea, applied at the system level — compress older context to make room for new.

Prompt caching

Use cache_control breakpoints on large, static content blocks (system prompts, reference docs, tool definitions) that don't change between turns.
Cached reads are 90% cheaper than uncached — massive savings in multi-turn agentic loops where the system prompt is repeated every turn.
Cache has a 5-minute TTL by default, with a 1-hour TTL option also available — if the next request comes within the TTL window, you get the cached price.
Trade-off: write cost to populate cache is 25% more than a normal read, so caching only pays off if you're making multiple requests against the same prefix.

Trade-off — when caching pays off: Cache if: multi-turn conversation, repeated system prompt, agentic loop (many iterations). Don't cache if: one-shot request with unique content. The break-even is roughly 2+ requests with the same cached prefix within 5 minutes.

The CALM mnemonic

Context-Aware LLM Management — an informal study aid, not an official Anthropic concept — for managing what goes into the context window:

Curate: only include information Claude actually needs for this turn.
Arrange: put the most important information at the start and end of context (primacy/recency bias).
Limit: enforce token budgets; summarize or truncate when approaching limits.
Monitor: track token usage, cache hit rates, and quality over time.

Error handling & escalation

Error type	Pattern
Tool failure (transient)	Retry with exponential backoff; return structured error to Claude if retries exhausted
Reasoning failure	Self-evaluation / confidence scoring; if confidence below threshold → escalate
Environment failure	Graceful degradation; fall back to cached/stale data or inform user
Policy gap / ambiguity	Human-in-the-loop escalation — don't guess, ask

Anti-pattern: Using self-reported confidence scores as the sole escalation signal. Claude's self-assessed confidence is unreliable — it may be confidently wrong. Combine with programmatic validation (schema checks, business-rule assertions) for robust escalation.

Model Selection & API Essentials

Model	Strength	Use when
`Opus (largest)`	Highest reasoning, complex tasks, nuanced judgment	Hard reasoning, complex orchestration, low-volume high-stakes
`Sonnet (mid)`	Best balance of capability and cost; fast	Default for most production workloads, agentic loops, coding
`Haiku (smallest)`	Fastest, cheapest, good for simple tasks	Classification, routing, simple extraction, high-volume low-complexity

Trade-off — model selection: Bigger = better reasoning but slower and costlier. Right-size to the task: use Haiku for routing/classification, Sonnet for the main workload, Opus for the hardest reasoning steps. In multi-agent systems, subagents can use smaller models than the coordinator.

API concepts the exam assumes

Messages API: the core endpoint. Send a list of messages (system, user, assistant), receive a response with content blocks and stop_reason.
Streaming: stream: true for real-time token delivery. Use for user-facing responses.
Extended thinking: thinking blocks that let Claude reason before responding. On current models this is adaptive (thinking: {type: "adaptive"} plus an effort level) rather than a fixed token budget. Useful for complex tasks; costs extra tokens.
System / Operator / User hierarchy: operator instructions (set by the app developer) can restrict what user prompts can override. Users can't elevate their own permissions beyond what the operator allows.

Anti-Pattern Master List

The exam loves presenting anti-patterns as plausible answer choices. Memorize these — they're the wrong answers that look right.

Anti-pattern	Why it's wrong	Correct approach
Parse NL to detect loop end	Non-deterministic; Claude may paraphrase	Check `stop_reason === "end_turn"`
Arbitrary iteration cap as primary stop	Not semantically meaningful	Use `stop_reason`; iteration cap as safety net only
Share full coordinator context with subagents	Context pollution; wasted tokens	Pass only relevant, structured context
Vague / overlapping tool descriptions	Claude misroutes tool calls	Distinct verbs, narrow scope, precise descriptions
Self-reported confidence as sole escalation trigger	Claude can be confidently wrong	Combine with programmatic validation
Prompt-only guardrails for hard safety constraints	Can be jailbroken / circumvented	Use hooks for hard constraints
Monolithic CLAUDE.md with all rules	Irrelevant context for most edits	`.claude/rules/` with path-specific scoping
JSON via prompting alone in production	Output drifts from schema	`tool_use` with JSON Schema for structured extraction
Retry on all errors equally	Wastes tokens on permanent failures	Classify errors: transient → retry; business → return; permission → escalate
No token budget monitoring	Silent degradation as context grows	Track usage; progressive summarization; prompt caching

Trade-off Master List

Decisions the exam tests repeatedly. Know the trigger phrase → right answer.

Decision	Option A	Option B	Pick A when	Pick B when
Loop termination	`stop_reason`	Iteration cap	Always primary	Safety net only
Guardrails	`Hooks (code)`	Prompt instructions	Hard safety constraints	Soft style/tone guidance
Execution	`Sequential`	Parallel	Steps depend on prior results	Independent subtasks
Transport	`stdio`	SSE	Local, same machine	Remote, team sharing, streaming
Structured output	`tool_use + schema`	Prompt for JSON	Production extraction	Quick prototyping only
Model size	`Haiku`	Opus	Simple / routing / high-volume	Complex reasoning / high-stakes
Caching	`Cache (90% cheaper reads)`	No cache	Multi-turn / agentic loop	One-shot unique requests
Batch vs real-time	`Batches API (50% off)`	Real-time Messages	Bulk ETL, overnight jobs	User-facing, low-latency
Context strategy	`Progressive summarization`	Full history	Long conversations / budgets	Short conversations / precision
CLAUDE.md structure	`Path-scoped rules`	Monolithic file	Large repos, mixed stacks	Tiny projects
Execution mode	`Plan mode`	Direct execution	Complex / high-risk changes	Simple / well-understood tasks
Skill isolation	`context: fork`	Inline execution	Verbose output / risk of pollution	Simple, brief tasks

Exam-Day Tips

Study all 6 scenarios — you get 4 randomly. Map each to its primary domains: Customer Support → agentic loop + escalation; Code Gen → Claude Code config; Multi-Agent → hub-and-spoke + context isolation; CI/CD → -p flag + structured output; Data Extraction → tool_use + validation-retry.
stop_reason is the answer for any loop-termination question. If an option says "parse the text for 'done'" — eliminate it.
Two answer choices will "work" — pick the architecturally superior one. The exam rewards the production-grade, scalable, safe choice over the quick hack.
Hooks for hard constraints, prompts for soft guidance. This distinction appears across multiple scenarios.
Context isolation is a theme. Any time a subagent gets "too much" context, or a coordinator shares everything, it's the wrong answer.
Structured > unstructured. Passing structured JSON between agents beats plain text. tool_use extraction beats prompting for JSON. Schema-validated output beats unvalidated.
Right-size the model. Using Opus for classification or Haiku for complex reasoning are both wrong in scenario questions.
Prompt caching math: 90% cheaper reads, 25% more expensive writes, 5-min default TTL (1-hour option available), break-even at ~2 requests. Know this for cost-optimization questions.
2 min/question average. Don't overthink — recognize the pattern (it maps to one of the anti-patterns or trade-offs above), pick the answer, and move on. Flag and return if unsure.

Preparation resources (free): Anthropic Academy (anthropic.skilljar.com): 13+ free courses covering all domains. Key courses: Building Applications with the Claude API (8+ hrs), Claude Code in Action, Introduction to MCP, AI Fluency Framework. Also: the official Exam Guide PDF (12 sample questions with explanations) and the official 60-question practice exam on Skilljar. Score 850+ on the practice before booking the real exam. Note the three-way split as of mid-2026: the public Academy hosts the free training courses, registration goes through the Partner Academy (anthropic-partners.skilljar.com, partner login required), and the exam itself is scheduled and proctored via Pearson VUE (OnVUE). Retakes are capped at 4 attempts per rolling 12 months, with waiting periods that lengthen after each failure (14 days, then 30, then 90).

Claude Certified Architect — Foundations (CCAR-F), 2026 Blueprint. Built from the official exam guide, Anthropic Academy materials, and community sources. Independent study aid — not affiliated with or endorsed by Anthropic. Exam format, fees, and scheduling change; verify current details on Pearson VUE's Anthropic certification page before booking.

GCP Professional Cloud Architect — 2026 Blueprint

Sat, 20 Jun 2026 00:00:00 GMT

The review I'd hand an engineer the week before their GCP Professional Cloud Architect exam: all six domains, the service trade-offs that actually decide questions, and the reasoning I want them to carry in.

Questions	Duration	Fee	Validity	Case studies	Pass mark
50–60	2 hours	$200 USD	2 years	2 of 4 · 20–30%	Not published

⚠ 2026 naming change: the exam is transitioning from Vertex AI to Gemini Enterprise Agent Platform. Both names may appear on questions. Renamed services are flagged here with the old name in brackets, e.g. GKE Enterprise (Anthos).

00 — Exam Format & Domains

The PCA is Google Cloud's flagship architect credential. It rewards architectural judgment and case-study reasoning, not memorization. Roughly 20–30% of questions attach to a case study; the rest are standalone scenarios where you pick the best answer among several that all "work". The Well-Architected Framework is woven through every domain — treat its six pillars as your default tie-breaker.

Standard vs Renewal exam

Attribute	Standard exam	Renewal exam
Length	2 hours	1 hour
Fee	$200	$100
Questions	50–60 MC & multi-select	25 MC & multi-select
Case studies	2 (from a pool of 4), 20–30%	2 available, gen-AI focused, 90–100%
Who	First-timers / expired certs	Active cert, within renewal window
Languages	English, Japanese	English, Japanese

Six domains & weighting (2026 guide)

⚠ Weightings are unofficial estimates — Google does not publish per-domain weightings for this exam. The percentages below are community/author estimates to help you prioritize study time; treat all six domains as testable and don't rely on these numbers.

Domain	Weight (est.*)	Watch for
1. Designing & planning a cloud solution architecture	~25%*	Business→technical mapping, trade-offs, HA/DR, Gemini Cloud Assist
2. Managing & provisioning solution infrastructure	~17.5%*	Network topology, storage/compute config, Agent Platform ML workflows
3. Designing for security & compliance	~17.5%*	IAM hierarchy, KMS, VPC-SC, Securing AI (Model Armor)
4. Analyzing & optimizing processes	~15%*	SDLC, CI/CD, cost (CapEx/OpEx), stakeholder mgmt
5. Managing implementation	~12.5%*	Apigee, IaC/Terraform, `gcloud`/`gsutil`/`bq`, emulators
6. Ensuring solution and operations reliability	~12.5%*	Cloud Observability, SLO/alerting, chaos/load testing

* Estimated — Google doesn't publish official per-domain weightings.

Read this first: The case studies are published in advance. Read all four cold before exam day — know each company's existing tech, business goals, constraints and compliance needs so you don't burn time reading during the exam.

01 — The Four Case Studies (2026 pool)

The old pool (Mountkirk Games, TerramEarth, Helicopter Racing League) is retired. Several current cases explicitly use Google's generative-AI solutions. Learn each as challenge → key services → themes.

Altostrat Media (Media)

Core challenge: Global streaming/media platform; personalization & gen-AI content features, cost-efficient transcoding, global low-latency delivery.
Key services: Global External App LB · Cloud CDN/Media CDN · Transcoder API · Agent Platform (Vertex AI) · BigQuery
Themes: Global delivery, personalization, cost at scale, gen-AI features.

Cymbal Retail (Retail)

Core challenge: Omnichannel retailer; demand forecasting, recommendations/search, seasonal traffic spikes, unifying operational + analytical data.
Key services: GKE / Cloud Run · Spanner or Cloud SQL · BigQuery · Agent Builder (search/recs) · Pub/Sub
Themes: Elastic scale, recommendations, HTAP data, gen-AI search.

EHR Healthcare (Healthcare)

Core challenge: Multi-hospital EHR SaaS migrating off legacy on-prem; HIPAA, multi-region HA, hybrid connectivity, strict governance.
Key services: Cloud Healthcare API · Cloud SQL (HA) · Shared VPC · Cloud Armor · Cloud KMS (CMEK) · VPC-SC · Assured Workloads
Themes: Compliance, security, hybrid migration, minimal downtime.

KnightMotives Automotive (Automotive / IoT)

Core challenge: Connected-vehicle telemetry at massive scale; time-series ingest, predictive-maintenance ML, modern data platform.
Key services: Pub/Sub · Dataflow · Bigtable · BigQuery · Agent Platform (Vertex AI) predictive models
Themes: IoT pipeline, batch vs streaming, ML integration, data lifecycle.

Pattern to internalize: Healthcare → compliance + Assured Workloads/CMEK. IoT telemetry → Pub/Sub → Dataflow → Bigtable/BigQuery. Retail/media personalization → Agent Platform (Vertex AI) + BigQuery.

02 — Service Renames — Old → New

Questions may use either name. These are the renames and deprecations most likely to trip you up in 2026.

Current name	Old / former name	Note
Gemini Enterprise Agent Platform	Vertex AI	The headline change — unified ML + gen-AI + agent platform
GKE Enterprise	Anthos	Fleet mgmt, multicluster/multicloud, Config Mgmt, Service Mesh
Cloud Run functions	Cloud Functions	FaaS folded under the Cloud Run brand
Cloud Service Mesh	Traffic Director / Anthos Service Mesh	Managed Istio/Envoy mesh
Google Cloud Observability	Stackdriver	= Cloud Monitoring + Logging + Trace + Profiler
Sensitive Data Protection	Data Loss Prevention (DLP)	Discover/classify/de-identify PII
Artifact Registry	Container Registry (GCR)	GCR deprecated — use Artifact Registry
Infrastructure Manager	Deployment Manager	Terraform-based; Deployment Manager is being retired
Chrome Enterprise Premium	BeyondCorp Enterprise	Zero-trust context-aware access
Google Security Operations (SecOps)	Chronicle	SIEM/SOAR under Google SecOps
Spot VMs	Preemptible VMs	Spot = successor, no 24h cap; preemptible is legacy
Migrate to Virtual Machines	Migrate for Compute Engine	VM lift-and-shift
Migrate to Containers	Migrate for Anthos	VM → container modernization
Managed Service for Apache Spark	(new, complements Dataproc)	Serverless Spark option
Hyperdisk / Block Storage	Persistent Disk (still valid)	Hyperdisk = next-gen, decoupled IOPS/throughput
gcloud storage	gsutil	Newer, faster CLI for Cloud Storage
Dataplex Catalog	Data Catalog	Now part of Dataplex governance

Deprecated — don't pick these: Cloud Debugger (retired), Cloud IoT Core (retired — use Pub/Sub + partners), and Deployment Manager (superseded by Terraform / Infrastructure Manager). If an answer relies on a retired service, it's almost certainly a distractor.

03 — AI & ML — the 2026 Focus Area

AI now appears across design (§1.3), provisioning (§2.4/2.5), and security (§3.1). Almost everything routes through the Gemini Enterprise Agent Platform (Vertex AI). Learn the stack and how the pieces fit.

This is more than a rename: Google has stated that Vertex AI services and roadmap are delivered exclusively through the Agent Platform rather than as a standalone service. In current docs and console the platform is organised under five top-level areas — Studio, Agents, Models, Notebooks, and CodeMender — so expect either vocabulary on exam questions and don't be thrown when a stem says "Vertex AI" and the answer options say "Agent Platform". The exam wants the Google service names; if you want the vendor-neutral reasoning underneath them, that's the AI architecture field guide.

Fig 03.1 — Gemini Enterprise Agent Platform stack

Layer	Components
Assist & agents (business value)	Gemini Cloud Assist · Agent Builder / Agent Studio · Gemini Enterprise / AI Agents · NotebookLM · Code Assist
Agent operations (new with the platform)	Agent-to-agent orchestration · Agent Registry · Agent Identity · Agent Gateway · Agent Observability
Models & APIs (build with)	Gemini LLMs · Model Garden (200+ models) · Search · Conversation · Vision · Image · Video · Audio (Speech)
MLOps (operate)	Agent Platform Pipelines (Vertex Pipelines) · Feature Store · Model Registry · Endpoints / serving
Infrastructure (train & serve)	AI Hypercomputer · GPUs · TPUs · Cloud Run functions
Securing AI (§3.1)	Model Armor · Sensitive Data Protection (DLP) · Secure model deployment / VPC-SC

What each piece is for

Service	Choose when…	AWS analog
Gemini Enterprise Agent Platform (Vertex AI)	Any custom-model training, tuning, deployment, or end-to-end MLOps	SageMaker
Model Garden	Pick from 200+ first/third-party models (Gemini, Llama, etc.) without building from scratch	Bedrock catalog
Agent Builder	Build RAG / search / conversational agents grounded on your data, low-code	Bedrock Agents
Gemini Cloud Assist	AI help designing, deploying & troubleshooting the architecture itself (in-console)	Amazon Q Developer
AI Hypercomputer	Large-scale training/serving; integrates GPUs & TPUs, optimized consumption models	EC2 UltraClusters
Prebuilt AI APIs	Off-the-shelf Vision, Speech-to-Text, Text-to-Speech, Translation, Document AI	Rekognition / Transcribe
Conversational Agents (Dialogflow CX / CCAI)	Build virtual agents / contact-center bots (voice & chat)	Lex / Connect
Model Armor	Screen prompts/responses for safety, prompt injection, data leakage	Bedrock Guardrails

Trade-off — prebuilt API vs custom model: Prebuilt API / Model Garden model = fastest, cheapest, no ML expertise, but generic. Custom-trained model on Agent Platform = best domain accuracy, but needs labeled data, MLOps, GPU/TPU cost, maintenance. Default: reach for prebuilt or a foundation model + RAG unless the scenario says pretrained accuracy is insufficient.

Scenario triggers: "Chatbot grounded on our docs/catalog" → Agent Builder + RAG. "Generate architecture / debug a failing deployment" → Gemini Cloud Assist. "Protect an LLM app from prompt injection / PII leakage" → Model Armor + Sensitive Data Protection. "Off-the-shelf image labels / transcription" → prebuilt AI API.

04 — Compute

Service	Control	Choose when…	AWS
Compute Engine	Max (IaaS)	OS access, custom kernels, licensed software, lift-and-shift, GPUs	EC2
GKE Standard	High	Kubernetes with node control, GPUs, custom node configs	EKS
GKE Autopilot	Medium	Kubernetes without node ops, pay-per-pod, simplified ops	EKS+Fargate
GKE Enterprise (Anthos)	High	Fleet mgmt across clusters / on-prem / multicloud	EKS Anywhere
Cloud Run	Low	Stateless containers, pay-per-request, scale-to-zero, HTTP	Fargate
Cloud Run functions (Functions)	Minimal	Event-driven code (Pub/Sub, HTTP triggers), short execution	Lambda
App Engine	Minimal	Opinionated PaaS web apps (Standard = runtimes; Flex = containers)	Beanstalk
Batch	Medium	Managed batch/HPC job scheduling	AWS Batch

Fig 04.1 — Compute decision tree

Need specific OS / kernel / licensing / GPUs or lift-and-shift? → Compute Engine
Existing Kubernetes / need portability & fine control? → GKE (Autopilot if no node ops)
Stateless containerized HTTP service, want scale-to-zero? → Cloud Run
Small event-driven glue (Pub/Sub, Storage triggers)? → Cloud Run functions
Multi-cluster / hybrid / multicloud fleet? → GKE Enterprise

Compute Engine essentials

Spot VMs (Preemptible): up to ~91% cheaper, reclaimed with 30s notice — fault-tolerant/batch only. Spot has no 24-hour cap (old preemptible did).
MIG: autoscaling + autohealing + rolling/canary + regional multi-zone spread. Unmanaged = static, no autoscale.
CUDs for steady state; Sustained Use Discounts apply automatically; custom machine types right-size vCPU/RAM.
Sole-tenant nodes for compliance/licensing; Confidential VMs encrypt memory in-use.

Scenario triggers: "Reduce operational overhead" → serverless (Cloud Run / functions). "Fine-grained control / custom nodes" → Compute Engine or GKE Standard. "Event-driven" → Cloud Run functions. "Operational simplicity for K8s" → Autopilot.

Trade-off — Cloud Run vs GKE: Cloud Run: zero infra, scale-to-zero, per-request billing, fastest to ship — but HTTP/event-centric, less control. GKE: full K8s (DaemonSets, operators, mesh, stateful, GPUs), portability — but you own upgrades/scaling/node cost even when idle. Default Cloud Run unless you need K8s primitives.

GKE scaling (concept level): GKE scales at two layers: the Horizontal Pod Autoscaler (HPA) adds/removes pods on CPU/memory/custom metrics, while the Cluster Autoscaler and Node Auto-Provisioning add/remove nodes to fit those pods. (Autopilot manages both for you.) You don't need kubectl or manifests for PCA — just know GKE autoscales pods and nodes separately, and that node pools can run on Spot VMs to cut cost.

Cloud Run: services vs jobs: Cloud Run services handle requests (HTTP/events, scale-to-zero). Cloud Run jobs run to completion (batch/ETL/scheduled scripts) with no listening port — the answer for "containerized batch task" without standing up GKE.

05 — Storage

Service	Type	Choose when…	AWS
Cloud Storage	Object	Unstructured data, backups, data lake, static sites, ML files	S3
Persistent Disk / Hyperdisk	Block	VM boot & data disks; Hyperdisk decouples IOPS/throughput	EBS
Local SSD	Block (ephemeral)	Scratch/cache — highest IOPS, data lost on stop	Instance Store
Filestore	File (NFS)	Shared POSIX file storage for VMs/GKE	EFS
NetApp Volumes / Managed Lustre	File	Enterprise NFS/SMB; HPC parallel FS	FSx

Fig 05.1 — Cloud Storage classes (hotter → colder = cheaper storage, pricier retrieval)

Class	Access pattern	Retrieval cost
Standard	Frequent access · no min duration · hot data, serving	no retrieval fee
Nearline	< once / month · 30-day min · backups	low retrieval
Coldline	< once / quarter · 90-day min · DR	higher retrieval
Archive	< once / year · 365-day min · compliance	highest retrieval

Location: region (lowest latency/cost), dual-region, or multi-region (highest availability). Bucket location & name are immutable; names globally unique.
Object Lifecycle Management: auto-transition classes or delete by age — the go-to for "reduce storage cost over time".
Autoclass: automatically moves each object between classes based on its actual access pattern (no age rules to write) — pick this when access is unpredictable.
Bucket Lock + retention policy for WORM/compliance; versioning for accidental deletes.
Signed URLs give time-limited object access to users without Google accounts; signed policy documents constrain browser-based uploads.
Access control: prefer uniform bucket-level access (IAM only, recommended) over fine-grained (IAM + legacy per-object ACLs).

Trade-off — colder classes: Each colder class lowers storage $/GB but adds a retrieval fee + minimum duration. Frequently-read data in Coldline/Archive costs more. Match the class to true access frequency and let lifecycle rules tier automatically.

06 — Databases

The single most-tested topic — know the decision rule cold.

Service	Model	Choose when…	AWS
Cloud SQL	Relational OLTP	MySQL/PostgreSQL/SQL Server, single region, standard OLTP	RDS
AlloyDB	Relational (PG)	High-perf PostgreSQL, HTAP, AI-ready	Aurora
Spanner	Relational, global	Multi-region strong consistency, 99.999% SLA, horizontal scale, >1000 TPS	Aurora Global
Firestore	NoSQL document	Mobile/web, real-time sync, offline, flexible schema	DynamoDB
Bigtable	NoSQL wide-column	IoT/time-series, high-throughput low-latency, >1TB	DynamoDB / Timestream
Memorystore	In-memory	Redis/Memcached caching, sessions, leaderboards, sub-ms	ElastiCache
BigQuery	Analytical OLAP	Petabyte analytics, warehouse, serverless SQL, BI	Redshift

Fig 06.1 — Database decision tree

Analytics / warehousing / SQL over huge datasets? → BigQuery
Relational + global scale & strong consistency? → Spanner
Relational, regional OLTP, standard engines? → Cloud SQL (AlloyDB if high-perf PG/HTAP)
NoSQL, real-time mobile/web, offline sync? → Firestore
NoSQL, high write throughput, time-series/IoT, >1TB? → Bigtable
Sub-ms caching / session store? → Memorystore

Scenario triggers (memorize): "Financial transactions" + "global" in one sentence → Spanner. "Sensor data / time-series" → Bigtable. Plain "relational" with no scale requirement → Cloud SQL. "Real-time mobile sync" → Firestore. "Ad-hoc SQL / dashboards / ML on history" → BigQuery.

Trade-off — Cloud SQL vs Spanner: Cloud SQL: cheaper, familiar engines, easy migration — but vertical ceiling & regional (replicas scale reads only). Spanner: unlimited horizontal scale + global strong consistency + 99.999% — but expensive and needs key design to avoid hotspots. Choose Spanner only for global writes or scale beyond one big instance.

Trade-off — Firestore vs Bigtable: Firestore: document model, real-time listeners, strong consistency, app backends — modest write throughput. Bigtable: wide-column, single-digit-ms at millions of ops/sec, ideal for time-series/IoT — no cross-row transactions, no secondary indexes, min 1-node cost. Row-key design is everything.

07 — Networking

Service	Function	AWS
VPC	Global virtual network (spans all regions)	VPC (regional)
Subnet	Regional (spans all zones in region)	Subnet (per-AZ)
Shared VPC	Host project shares subnets with service projects — centralised network admin	VPC sharing (RAM)
VPC Peering	Private connectivity between VPCs — non-transitive	VPC Peering
Private Service Connect	Private access to services/APIs across VPCs/orgs	PrivateLink
Private Google Access	VMs without external IP reach Google APIs (per-subnet)	Gateway endpoints
Cloud Interconnect	Dedicated/Partner private link to on-prem	Direct Connect
Cloud VPN (HA VPN)	Encrypted IPsec over internet (99.99% w/ 2 tunnels)	Site-to-Site VPN
Cloud Router	Dynamic BGP for VPN & Interconnect	(TGW/DXGW)
Cloud Armor	WAF + DDoS at the global LB (IP/geo/OWASP)	WAF + Shield
Cloud NAT	Egress for private instances	NAT Gateway
Network Connectivity Center	Hub-and-spoke transit — connects many VPCs/sites (works around peering's non-transitivity)	Transit Gateway
Cloud DNS	Managed DNS — public & private zones, split-horizon	Route 53
Cloud Service Mesh (Traffic Director)	Managed service-to-service traffic, mTLS, canary routing (Istio/Envoy)	App Mesh
Cloud IDS	Managed intrusion detection (inspects traffic for threats)	GuardDuty (network)

GCP vs AWS — the key mental model

	GCP	AWS
VPC scope	Global — one VPC spans every region	Regional — one VPC per region
Subnet scope	Regional — spans all zones	Per-AZ
Global LB	Single anycast IP, one LB worldwide	CloudFront + regional LBs

Load balancer selection

Scenario	Use
HTTP/HTTPS, global, multiple regions	Global External Application LB
HTTP/HTTPS, must stay in one region (data residency)	Regional External Application LB
TCP/UDP, preserve client IP, non-HTTP	External Network LB (passthrough)
Internal microservices / private VPC traffic	Internal Application or Network LB

Fig 07.1 — Hybrid connectivity: choosing the on-prem link

Highest bandwidth, dedicated physical fibre, lowest latency (10/100 Gbps)? → Dedicated Interconnect
Cannot co-locate at a Google PoP / lower bandwidth (50 Mbps–50 Gbps)? → Partner Interconnect
Quick / cheap / encrypted over public internet (<3 Gbps)? → HA VPN (99.99%)
Reach Google APIs privately from on-prem/VMs? → Private Google Access / PSC

Scenario triggers: "Cannot co-locate at a Google facility" → Partner Interconnect. "Dedicated physical link, highest bandwidth" → Dedicated Interconnect. "Global users, one IP, HTTP" → Global External App LB. "Preserve source IP / non-HTTP" → Network (passthrough) LB. Remember VPC peering is non-transitive.

Network observability: For "diagnose connectivity / see traffic" answers: VPC Flow Logs (traffic records), Firewall Rules Logging / Insights (what rules allow/deny), and Network Intelligence Center (Connectivity Tests, Topology, Performance Dashboard) to troubleshoot reachability and latency.

IP / CIDR planning: Before any hybrid connectivity, ensure non-overlapping RFC 1918 ranges across on-prem and every VPC/subnet — overlapping CIDRs break routing and can't be corrected without re-addressing. Plan ranges with headroom; subnet ranges can expand but must never overlap.

08 — High Availability & Disaster Recovery

RTO vs RPO — know the difference cold

RTO — Recovery Time Objective

Max acceptable downtime after a failure.
"How long until we're back up?"
Lower RTO → hot standby, automated failover, MIGs.

RPO — Recovery Point Objective

Max acceptable data loss (measured in time).
"How much recent data can we lose?"
Lower RPO → synchronous replication, frequent snapshots.

Fig 08.1 — DR strategy ladder (cheaper/slower → pricier/faster)

Backup & Restore — snapshots/exports to Cloud Storage → high RTO/RPO · lowest cost
Cold standby (pilot light) — minimal core running, scale on failover → medium RTO
Warm standby — scaled-down full env, promote on failover → low RTO
Hot / multi-region active-active — no data loss → near-zero RTO/RPO · highest cost

HA patterns by service

Service	HA configuration
Compute Engine	Regional MIG across multiple zones + autoscaling + health checks
Cloud SQL	HA config with standby in a 2nd zone, automatic failover
Spanner	Multi-region config = 99.999%, synchronous replication
GKE	Regional cluster spreads nodes across 3 zones automatically
Cloud Storage	Multi-region / dual-region bucket = geo-redundant objects
BigQuery	Multi-region dataset; cross-region dataset replication for DR
Backup and DR Service	Centralized managed backup/restore for GCE, disks, databases — the managed backup answer

Scenario triggers: "Cannot tolerate data loss" (RPO≈0) → synchronous replication → Spanner multi-region or Cloud SQL HA. "Low-cost DR, some data loss acceptable" → async replication / cross-region GCS copy. "Survive a zone failure" → regional (multi-zone). "Survive a region failure" → multi-region.

Trade-off — availability tiers: Zonal = cheapest, single point of failure. Regional (multi-zone) = survives a zone outage, the usual production baseline. Multi-region = survives a region outage, highest cost and higher write latency. Buy only the tier the SLA demands.

09 — IAM & Security

Fig 09.1 — Resource hierarchy (policies inherit downward; org policies + deny rules override)

Organization → Folder(s) → Project(s) → Resources (VMs, buckets, DBs)

Roles: Basic (Owner/Editor/Viewer — too broad, never in prod) → Predefined (service-specific, recommended) → Custom (exact least-privilege permissions).
Service accounts: both identity and resource. Prefer attached SAs + short-lived tokens and impersonation (separation of duties) over exported keys. Workload Identity Federation = GitHub Actions / AWS / on-prem auth without keys.
IAM Conditions: attribute-based (time, resource, IP). Deny policies override allow.
Human identity: Cloud Identity is the user/group directory (IdaaS). Google Cloud Directory Sync (GCDS) one-way syncs on-prem AD/LDAP → Cloud Identity. Use SSO via SAML/OIDC with an external IdP (Okta, Entra ID). Workforce Identity Federation lets external-IdP employees use GCP with short-lived creds and no user provisioning — the human analog to Workload Identity Federation.
Org policies: preventive guardrails (restrict regions, block external IPs, enforce CMEK) applied hierarchy-wide.

Service	Function	AWS
Cloud KMS / HSM / EKM	Managed / hardware / external key management (CMEK)	KMS / CloudHSM
Secret Manager	Secrets storage + rotation + versioning + audit	Secrets Manager
VPC Service Controls	Perimeter to stop data exfiltration from managed APIs	(no direct equal)
Identity-Aware Proxy (IAP)	Zero-trust access to apps/VMs without VPN	Verified Access
Access Context Manager	Defines access levels (IP, device, region) that power IAP & VPC-SC	(condition keys)
Chrome Enterprise Premium (BeyondCorp)	Context-aware zero-trust access	—
Binary Authorization	Only trusted/signed container images deploy to GKE/Run	(Signer)
Security Command Center	Posture mgmt, vuln scanning, threat detection	Security Hub
Cloud Asset Inventory	Search/export/monitor all resources & IAM across the org (governance, audits)	Config / Resource Explorer
Google Security Operations (Chronicle)	SIEM/SOAR — threat hunting & correlation (behind SCC Enterprise)	Security Lake + Detective
Sensitive Data Protection (DLP)	Discover / classify / de-identify PII	Macie
Assured Workloads	Compliance-controlled envs (residency, personnel)	(compliance controls)
Model Armor	Guardrails for LLM prompts/responses	Bedrock Guardrails

Trade-off — key management tiers: Encryption at rest is always on. Google-managed keys = zero effort. CMEK (Cloud KMS) = you control rotation/disable for compliance. Cloud HSM = FIPS 140-2 L3 hardware. Cloud EKM = key held by an external/third-party manager (max control & separation, most overhead). Escalate only as compliance demands.

Scenario triggers: "Prevent data leaving a perimeter even with valid creds" → VPC Service Controls. "Access internal app, no VPN" → IAP. "External CI/CD auth without long-lived keys" → Workload Identity Federation. "Only signed images to GKE" → Binary Authorization. "Regulator requires keys off-cloud" → Cloud EKM. "Federate on-prem AD / external IdP for employees" → Cloud Identity + GCDS + SSO (or Workforce Identity Federation for no provisioning).

10 — Operations & Observability

All under Google Cloud Observability (Stackdriver).

Service	Function	AWS
Cloud Monitoring	Metrics, dashboards, uptime checks, SLOs, alerting	CloudWatch Metrics
Cloud Logging	Log ingest; Log Sinks export to BigQuery/GCS/Pub/Sub	CloudWatch Logs
Cloud Trace	Distributed latency tracing across microservices	X-Ray
Cloud Profiler	Continuous CPU/heap profiling	CodeGuru Profiler
Error Reporting	Aggregates + surfaces app errors in real time	—
Managed Service for Prometheus	Managed Prometheus for GKE/metrics	AMP
Cloud Audit Logs	Admin / Data Access / System / Policy logs	CloudTrail

Scenario triggers: "Centralise logs across all projects" → Log Sinks to a central project's logging/GCS bucket (aggregated sink at org/folder). "Long-term log retention/analytics" → sink to BigQuery/GCS. Admin Activity audit logs are always-on & immutable; Data Access logs are opt-in (high volume/cost).

SRE mindset (§6): Reliability answers favor SLIs/SLOs + error budgets, alerting on symptoms not causes, and validating resilience with chaos / load / penetration testing.

CI/CD & orchestration (§5 Managing implementation)

Service	Function	AWS
Cloud Build	CI — build, test, containerize from source	CodeBuild
Artifact Registry (GCR)	Store build artifacts / container images	ECR / CodeArtifact
Cloud Deploy	CD — managed progressive delivery to GKE/Run	CodeDeploy
Eventarc	Event routing (Cloud/Audit/Pub-Sub events → services)	EventBridge
Workflows	Serverless orchestration of API/service steps	Step Functions
Cloud Scheduler	Managed cron for jobs/HTTP/Pub-Sub	EventBridge Scheduler
Cloud Tasks	Async task queue with rate/retry control	SQS (task-style)

Scenario triggers: Pipeline chain: Cloud Build → Artifact Registry → Cloud Deploy. "Trigger a service from a GCS/Audit event" → Eventarc. "Coordinate multi-step API calls" → Workflows. "Run on a schedule" → Cloud Scheduler. "Rate-limited async work queue" → Cloud Tasks. Note Cloud Composer (Airflow) is for heavy data pipelines; Workflows is lighter service orchestration.

11 — Data & Analytics

Service	Function	AWS
Pub/Sub	Global async messaging — entry point for streaming pipelines	SNS+SQS / Kinesis
Dataflow	Managed Apache Beam — unified stream + batch ETL	Kinesis Data Analytics
Dataproc	Managed Hadoop/Spark — migrate existing jobs	EMR
Managed Service for Apache Spark	Serverless Spark	EMR Serverless
BigQuery	Serverless warehouse + BQML + BI Engine	Redshift
Cloud Composer	Managed Apache Airflow orchestration	MWAA
Cloud Data Fusion	Visual, code-free ETL/ELT	Glue (visual)
Dataplex	Governance, lakehouse, catalog	Lake Formation
Looker / Looker Studio	Governed BI & visualization on BigQuery	QuickSight
Datastream	Serverless change-data-capture (CDC) — replicate DB changes into BigQuery/GCS	DMS (CDC)
Cloud Healthcare API	Managed FHIR/HL7v2/DICOM store — the EHR-case ingestion/interop layer	HealthLake

Fig 11.1 — Canonical streaming pipeline (the IoT / telemetry answer)

Pub/Sub → Dataflow (transform) → Bigtable (low-latency serving) + BigQuery (analytics)

Trade-off — Dataflow vs Dataproc: Dataflow: serverless, autoscaling, one pipeline for batch and stream — preferred greenfield. Dataproc: when migrating existing Hadoop/Spark/Hive with minimal rewrite. Lift-and-shift Spark → Dataproc; net-new streaming → Dataflow.

12 — Migration Strategy (the 6 R's)

Strategy	Meaning	GCP tooling
Rehost (lift & shift)	Move as-is, fastest	Migrate to VMs, Migration Center
Replatform	Minor optimization (self-managed DB → Cloud SQL)	Database Migration Service
Refactor / modernize	Re-architect to cloud-native	Migrate to Containers, GKE, Cloud Run
Repurchase	Switch to SaaS	—
Retire	Decommission unused	—
Retain	Keep on-prem for now (hybrid)	Interconnect / GKE Enterprise

Assess first with Migration Center (discovery, TCO, dependency mapping) — always the step before choosing a strategy.
Data transfer sizing: Storage Transfer Service (online), Transfer Appliance (petabyte-scale offline), gcloud storage for smaller sets.

Trade-off — speed vs value: Rehost = fastest, least cloud value, keeps tech debt. Refactor = highest long-term value but most time/cost/risk. "Tight deadline / minimize change" → rehost; "reduce operational burden / scale elastically" → refactor to managed/serverless.

13 — Cost Optimization

Lever	Savings / trigger	Use when…
Committed Use Discounts (CUD)	up to ~57–70% for 1- or 3-yr commit	Steady-state, predictable baseline load
Spot VMs (Preemptible)	up to ~91% off; 30s reclaim notice	Batch, fault-tolerant, stateless workers
Sustained Use Discounts	automatic after >25% of the month	Always — no action needed (Compute Engine)
Rightsizing / Recommender	flags idle/underused VMs	Regular reviews; Active Assist
Storage Lifecycle policies	auto-tier to Nearline/Coldline/Archive	Aging data with dropping access frequency
Autoscaling / scale-to-zero	pay only for what runs	Spiky/variable traffic → Cloud Run, MIGs, Autopilot
BigQuery editions vs on-demand	reservations for heavy, on-demand for variable	Predictable heavy analytics → capacity/reservations
Budget alerts	catch overspend early	Always — set in Cloud Billing
Billing export to BigQuery	detailed spend analysis & dashboards	"Analyze / break down cloud spend"
Labels	tag resources for cost allocation & reporting	Chargeback / show-back by team/env

Scenario triggers: "Steady 24/7 workload, cut compute cost" → CUDs. "Batch jobs, cost-sensitive, interruptible" → Spot VMs. "Reduce storage cost over time" → lifecycle policies. "Right-size / find waste" → Recommender / Active Assist. "Prevent surprise bills" → budget alerts (they don't cap spend — they notify).

Trade-off — CUD vs Spot: CUD guarantees capacity + discount but locks you into a 1/3-yr spend commitment (best for baseline). Spot is far cheaper with no commitment but can vanish in 30s (best for fault-tolerant/batch). Many designs combine: CUD for the steady baseline + Spot for burst/batch.

14 — Business, Process & Governance

Domain §4 (~15%, unofficial estimate — Google doesn't publish weightings) is not about services — it tests architect judgment: stakeholders, process maturity, buy-vs-build, and how responsibility is shared. These questions have no product in the answer; they reward the option that reflects sound engineering-organization practice.

People & process

Stakeholder & change management: identify stakeholders early, communicate trade-offs in business terms, and plan training/change so a technically correct design actually gets adopted.
Success = business KPIs, not tech metrics. Tie the architecture to measurable outcomes (cost per transaction, time-to-market, latency SLA) and define them up front.
SDLC & DevOps maturity: favor CI/CD, IaC, automated testing, and small frequent releases. The DORA metrics — deployment frequency, lead time for changes, change-failure rate, MTTR — are the standard health signals.
Team skills & topology: assess current skills and plan upskilling; managed/serverless reduces the ops burden on a stretched team.

Buy vs build

Default heuristic: Prefer managed services / SaaS (repurchase) to remove undifferentiated heavy lifting; build custom only where it is a genuine competitive differentiator. "Reduce operational burden / focus engineers on the product" points to buy/managed.

Shared Responsibility & Shared Fate

Google secures ("of the cloud")	You secure ("in the cloud")
Hardware, network, hypervisor, physical DCs, managed-service internals	Your data, IAM & access, configuration, OS/patching (on IaaS), app code

The more managed the service, the more responsibility shifts to Google (IaaS → PaaS → serverless).
Shared fate goes further: Google actively helps you succeed via secure blueprints, Assured Workloads, and Security Command Center.

Compliance & data governance

Certifications (ISO 27001, SOC 1/2/3, PCI-DSS, HIPAA, FedRAMP) — find attestations in the Compliance Reports Manager.
Data residency / sovereignty: Assured Workloads enforces location + personnel controls; the org policy resource-locations constraint restricts where resources can be created.

Deployment strategies

Strategy	How it works	Trade-off
Recreate	Stop old, deploy new	Simplest; causes downtime
Rolling	Replace instances gradually	No downtime; mixed versions during rollout (MIG/GKE default)
Blue/Green	Full parallel env, switch traffic	Instant rollback; double the resources briefly
Canary	Route a small % to the new version first	Limits blast radius; needs good metrics/automation
A/B	Route by attribute to test variants	Experimentation, not just release safety

Scenario triggers: "Zero downtime, instant rollback" → Blue/Green. "Limit risk / test on real traffic first" → Canary. "No downtime, minimal extra cost" → Rolling. Cloud Deploy automates progressive (canary/rolling) delivery.

15 — Well-Architected Framework — 6 Pillars

Your default tie-breaker when two answers both "work". Explicitly referenced throughout the exam guide.

Operational Excellence — Observability, automation, incident response, deployment discipline.
Security, Privacy & Compliance — Least privilege, defense in depth, encryption, governance.
Reliability — HA/DR, redundancy, graceful degradation, SLOs & error budgets.
Performance Optimization — Right-sizing, autoscaling, caching, managed services.
Cost Optimization — CapEx→OpEx, commitment discounts, lifecycle tiering, cleanup.
Sustainability — Region carbon data, efficient utilization, right-sizing.

How to use it in questions: Stuck between two valid options? Pick the one serving the pillar the scenario stresses: "always available" → reliability; "sensitive data" → security; "reduce spend" → cost. Managed/serverless usually wins on operational excellence.

16 — Trade-off Master List

The decisions examiners most love to test — memorize the "choose X when" trigger.

Compute pricing models

Cheaper:

Spot VMs — up to ~91% off, interruptible
CUDs — steady, predictable load
Sustained Use — automatic, no commitment
Autopilot / serverless — pay for what runs

Pricier but safer:

On-demand VMs — no interruption, no commit
Reserved capacity — guaranteed availability
Over-provisioned nodes — headroom, waste

Consistency vs scale (databases)

Strong consistency:

Spanner — global + strong (premium)
Cloud SQL / AlloyDB — ACID, regional
Firestore — strong within region

Scale / throughput:

Bigtable — huge throughput, no cross-row txns
BigQuery — analytics scale, not OLTP
Read replicas — scale reads, eventual lag

Managed vs self-managed

Default heuristic: Managed / serverless usually beats self-managed — less ops = better operational excellence + often lower TCO. Choose self-managed (Compute Engine, self-hosted DB, Dataproc) only when you need OS control, a specific engine/version, licensing, or a low-effort lift-and-shift.

AI build vs buy

Escalation ladder: Prebuilt API → Foundation model from Model Garden → RAG / Agent Builder on your data → Fine-tune → Train custom on Agent Platform + AI Hypercomputer. Climb only when the cheaper rung can't meet accuracy/latency/data-control needs.

17 — Exam-Day Tips

Read the case studies days before. Know each company's goals, existing tech, and constraints so questions become fast lookups.
Every answer maps to a WAF pillar. When two options both work, pick the one serving the pillar the scenario emphasizes.
Least privilege & managed services are the safe defaults for "best practice" questions.
Watch the qualifier: "most cost-effective", "least operational overhead", "fastest to deploy", "minimize change", "global", "cannot tolerate data loss" — the qualifier decides between otherwise-valid answers.
Eliminate deprecated services (Cloud Debugger, IoT Core, Deployment Manager) — they're distractors.
AI questions: prefer prebuilt/RAG over custom training unless accuracy demands it; remember Model Armor + Sensitive Data Protection for securing AI.
Global VPC / regional subnets — a recurring networking trick; VPC peering is non-transitive.
Compliance flags (HIPAA, PCI, residency, sovereignty) → Assured Workloads + CMEK/EKM + VPC-SC.
Flag & move on. ~2 min/question; mark for review and return.

18 — Exam FAQs

How hard is the PCA, and how long should I prepare?

It's one of the harder GCP exams — 50–60 questions in 2 hours, heavily scenario-based. Most candidates with 2+ years of cloud experience pass in one attempt after 4–8 focused weeks, with the emphasis on case studies and service trade-offs rather than rote facts.

What are the current official case studies?

The 2026 pool is Altostrat Media, Cymbal Retail, EHR Healthcare, and KnightMotives Automotive — you get 2 of the 4 on your exam. The older Mountkirk Games / TerramEarth / Helicopter Racing League set has been retired. Read all four in advance and map their requirements to services.

Cloud SQL vs Spanner — when do I pick each?

Cloud SQL for standard OLTP in a single region (MySQL/PostgreSQL/SQL Server). Spanner when you need multi-region strong consistency, a 99.999% SLA, or horizontal scale beyond one instance (roughly >1,000 TPS or global writes). "Financial + global" in one sentence is almost always Spanner.

GKE Autopilot vs Standard — quick rule?

Autopilot when the scenario prioritizes reduced operational burden — Google manages nodes, scaling, and patching, billed per pod. Standard when you need specific machine types, GPUs/TPUs, or custom node configs. On the exam, "operational simplicity" almost always points to Autopilot.

Which storage class for archival / compliance data?

Archive for < once/year (regulatory records), Coldline for < once/quarter, Nearline for < once/month. Use Object Lifecycle Management to auto-transition so you never pay Standard rates for cold data. Colder = cheaper to store but pricier to retrieve.

What changed for AI in 2026?

Generative AI is now explicitly in scope and the exam is moving from the Vertex AI name to Gemini Enterprise Agent Platform. Expect questions on Model Garden, Agent Builder (RAG), Gemini Cloud Assist, AI Hypercomputer, and securing AI with Model Armor + Sensitive Data Protection. Both old and new product names may appear.

Standard vs renewal exam?

First-timers and expired certs take the standard exam (2 hrs, $200, 50–60 questions, 2 case studies). If you hold an active cert within the renewal window, you can take the shorter renewal exam (1 hr, $100, 25 questions, with 2 gen-AI case studies available and case-study-weighted questions making up 90–100% of it).

Appendix A1 — CLI Quick Reference

Recognition, not memorization: The PCA is a design exam — it won't ask you to type commands with exact flags (that's the Associate Cloud Engineer exam). §5.2 only expects you to know which tool does what and to recognize a command in an answer choice. Learn the tool-to-service mapping below; don't over-invest here.

The three CLIs — know the split

Tool	Scope	Example
`gcloud`	Everything except the two below — compute, IAM, networking, GKE, projects, config	`gcloud compute instances list`
`gcloud storage` (gsutil)	Cloud Storage objects & buckets	`gcloud storage cp file gs://bucket`
`bq`	BigQuery datasets, tables, queries	`bq query 'SELECT ...'`
`kubectl`	In-cluster GKE objects — after `get-credentials`	`kubectl get pods`

Commands worth recognizing

Area	Command	What it does
Config	`gcloud init` / `gcloud auth login`	Set up + authenticate the SDK
Config	`gcloud config set project PROJECT_ID`	Switch the active project
Projects	`gcloud projects create\|list`	Create / list projects in the hierarchy
Compute	`gcloud compute instances create\|list\|delete`	Manage individual VMs
Compute	`gcloud compute instance-templates create`	Define the VM blueprint for a MIG
Compute	`gcloud compute instance-groups managed create`	Create a MIG (autoscaling + autohealing)
GKE	`gcloud container clusters create --region ...`	Regional cluster = nodes across 3 zones (HA)
GKE	`gcloud container clusters get-credentials NAME`	Fetch kubeconfig → then use `kubectl`
IAM	`gcloud projects add-iam-policy-binding PROJECT --member=... --role=roles/...`	Grant a role to a principal (least privilege)
IAM	`gcloud iam service-accounts create NAME`	Create a service-account identity
Storage	`gcloud storage cp \| ls \| rm`	Copy / list / delete objects (old: `gsutil`)
BigQuery	`bq mk` · `bq load` · `bq query`	Make dataset · load data · run SQL
Pub/Sub	`gcloud pubsub topics create` / `subscriptions create`	Create streaming entry points
Network	`gcloud compute networks create --subnet-mode=custom`	Custom-mode VPC (global; subnets regional)
Network	`gcloud compute firewall-rules create`	Allow/deny VPC traffic

The one fact that actually earns points: It's the tool-to-service mapping, not flags: bq = BigQuery · gcloud storage/gsutil = Cloud Storage · kubectl (after get-credentials) = inside a GKE cluster. And for provisioning: prefer Terraform / Infrastructure Manager (IaC) over the deprecated Deployment Manager. Also know the emulators — Bigtable, Spanner, Pub/Sub, Firestore — exist for local testing without touching real resources.

Don't over-study this: If you find yourself memorizing exact flag names or output formats, stop — you're studying for the ACE exam, not PCA. Spend that time on case studies and service trade-offs instead.

Appendix A2 — Editions & Tiers

Why this matters: The exam tests tiers indirectly: a scenario names a capability (threat detection, global backbone routing, high IOPS) and the right answer depends on knowing which tier provides it. Learn what unlocks at each tier, not the prices.

Security Command Center

Tier	Unlocks	Scenario signal
Standard (free)	Basic posture — Security Health Analytics (misconfigs, exposed resources), Web Security Scanner custom scans. GCP only.	"Free dashboard / find misconfigurations"
Premium	+ Threat detection (Event / Container / VM), attack-path simulation, compliance monitoring (CIS, PCI-DSS, NIST, HIPAA, ISO). Org or project level.	"Detect active threats / monitor PCI-HIPAA / attack paths"
Enterprise	Multi-cloud CNAPP across AWS/Azure, integrated Google SecOps (SIEM/SOAR), CIEM. Org level only.	"One security view across GCP + AWS + Azure with SIEM"

Note: The SCC Enterprise tier is scheduled to shut down on May 21, 2027, with those orgs moving to Premium. If a question forces a choice, Premium is the safe "paid tier" default for threat detection + compliance.

Network Service Tiers

Premium (default):

Traffic rides Google's global backbone end-to-end
Supports global load balancing & anycast
Best performance & reliability

Standard:

Egresses to public internet near the region
Regional load balancing only
Cheaper — trade performance/global reach for cost

Trigger: "Minimize network egress cost, single-region app" → Standard tier. "Global users, best latency/reliability" → Premium tier.

Other capability-gated choices

Choice	Options (cheaper → pricier / more capable)	Pick on
Persistent Disk / Hyperdisk	pd-standard (HDD) → pd-balanced (default SSD) → pd-ssd → pd-extreme / Hyperdisk (tunable)	cost vs IOPS/latency
BigQuery compute	On-demand (per-TB scanned, spiky) vs Editions (Standard / Enterprise / Enterprise Plus — slot capacity + autoscale + commitments)	predictable heavy load → Editions
Firestore	Native mode (real-time sync + offline) vs Datastore mode (server-side, no real-time)	new apps → Native
GKE	Autopilot / Standard modes → GKE Enterprise (Anthos) for fleet/multicloud	hybrid/multicluster → Enterprise
Cloud VPN	Classic VPN (99.9%) → HA VPN (99.99%)	production → HA VPN

Built from the official Google Cloud exam guide and certification page. Verify current details at cloud.google.com/learn/certification/cloud-architect. Independent study aid, not affiliated with Google.

MLOps & AI Production Operations: The 2026 Guide

Thu, 04 Jun 2026 00:00:00 GMT

How I think about taking AI from a working notebook to something that runs reliably in production — the full path from commit to monitoring, and the decisions that matter along the way.

01 — DevOps vs MLOps: What's Different

MLOps extends DevOps principles to machine learning — but ML systems have fundamentally different properties that traditional CI/CD wasn't built for. Everything here assumes the architecture upstream of it is already settled; that groundwork is the AI architecture field guide.

	Traditional DevOps	MLOps
What you version	Code	Code + data + model weights + hyperparameters + environment
What you test	Correctness (unit/integration)	Correctness + accuracy + fairness + drift + latency
What degrades	Bugs (introduced by code changes)	Bugs + model drift (performance degrades even without code changes, because real-world data shifts)
Deployment artifact	Container / binary	Container + model weights + feature schema + serving config
Rollback	Revert to last good container	Revert model version + verify feature compatibility + validate data pipeline
Monitoring	Latency, errors, CPU, memory	All the above + prediction quality + data distribution + feature drift + business KPIs

The fundamental difference: In software, code is the complete specification — if the code hasn't changed, the behavior hasn't changed. In ML, the data is part of the specification — the model can degrade silently even when no code changes, because the real-world data distribution shifted. This is why MLOps adds continuous training and continuous monitoring to the DevOps loops.

02 — The MLOps Pipeline (End-to-End)

The production MLOps pipeline:

Data ingestion (sources → lake)
  → Data validation (schema + quality)
  → Feature engineering (feature store)
  → Model training (GPU/TPU cluster)
  → Evaluation (metrics + tests)
  → Model registry (versioning)
  → CI/CD gate (approve / reject)
  → Deployment (container + endpoint)
  → Monitoring (drift + quality)
  → ⤴ Retrain

What each stage does

Stage	What happens	Key tool(s)
Data ingestion	Collect data from sources (DBs, APIs, streams, files) into storage	Pub/Sub, Kinesis, Airflow, dbt
Data validation	Schema checks, null detection, distribution validation — reject bad data before it reaches training	Great Expectations, TFX Data Validation, Evidently
Feature engineering	Transform raw data into model features; store in feature store for consistency between training and serving	Feast, Vertex AI Feature Store, SageMaker Feature Store
Model training	Run training job on GPU/TPU cluster; track experiments, hyperparameters, metrics	PyTorch/TF, MLflow, W&B, cloud training services
Evaluation	Compare new model against baseline on held-out test set + fairness/bias checks	MLflow, Evidently, custom eval harnesses
Model registry	Version and store the trained model with metadata, lineage, and lifecycle stage (staging → production)	MLflow Model Registry, cloud registries
CI/CD gate	Automated tests pass → human approval (optional) → promote to production	GitHub Actions, GitLab CI, Jenkins, cloud pipelines
Deployment	Package model into container, deploy to serving endpoint (real-time/batch/serverless)	Docker, KServe, Seldon, cloud endpoints
Monitoring	Track prediction quality, data drift, latency, cost, business KPIs — trigger retraining when needed	Evidently, Arize, Langfuse, Prometheus+Grafana

The 20/80 rule: Training a model is ~20% of the effort. The other 80% is testing, packaging, versioning, deploying, monitoring, and maintaining it. Teams that treat ML like one-off experiments instead of production software always fail at scale.

03 — CI/CD for ML: The Three Loops

Traditional DevOps has one CI/CD loop (code change → test → deploy). ML has three:

Loop 1 — Continuous Integration (code): code change → lint + unit tests → integration tests → data validation tests → model training tests (small subset)

Loop 2 — Continuous Training (model): new data arrives or drift detected → feature pipeline runs → training job executes → evaluation against baseline → register new model version

Loop 3 — Continuous Deployment + Monitoring: promote model to staging → canary/shadow deploy → monitor quality + drift → auto-rollback if degraded → trigger retraining if drift exceeds threshold

Google's MLOps maturity levels

Level	Description	Automation
Level 0	Manual — data scientists train in notebooks, hand-off model as a file, manual deployment	None
Level 1	ML pipeline automation — automated training pipeline, but deployment is still manual or semi-auto	Training automated
Level 2	CI/CD for ML — automated testing, automated deployment, continuous training triggered by data/drift, monitoring closes the loop	Full automation

Target: Most teams start at Level 0. The goal is Level 2 — where new data automatically triggers retraining, evaluation gates ensure quality, and deployment is hands-free with auto-rollback. Level 1 (automated training, manual deploy) is a practical intermediate milestone.

04 — DevOps / MLOps Tool Catalog

Organized by function. Pick one tool per row — don't try to use all of them.

Core DevOps tools (you still need these)

Function	Tools	Notes
Version control	Git (GitHub, GitLab, Bitbucket)	Foundation for everything — code, configs, IaC
CI/CD runner	GitHub Actions, GitLab CI, Jenkins, CircleCI	GitHub Actions is the default for most teams in 2026
Containers	Docker, Podman	Containerize everything: training, serving, pipelines
Container orchestration	Kubernetes (GKE, EKS, AKS), Docker Compose	K8s for production scale; Compose for local dev
Container registry	Artifact Registry (GCP), ECR (AWS), ACR (Azure), Docker Hub	Store training + serving container images
Infrastructure as Code	Terraform, Pulumi, CloudFormation, Bicep	Terraform is cross-cloud default; use cloud-native for single-cloud
Secrets management	Vault, Secret Manager (GCP/AWS), Azure Key Vault	Never hardcode API keys, model weights paths, DB creds

ML-specific tools

Function	Tools	Choose when
Experiment tracking	MLflow, Weights & Biases, Comet, Neptune	MLflow (open-source, self-host) is the default; W&B for teams wanting managed + collaboration
Data versioning	DVC, LakeFS, Delta Lake	DVC for small teams; LakeFS/Delta for lake-scale versioning
Feature store	Feast, Tecton, Vertex AI/SageMaker built-in	Feast (open-source) or cloud-native for managed simplicity
Pipeline orchestration	Kubeflow Pipelines, Airflow, Prefect, Dagster, cloud pipelines	Airflow for general orchestration; Kubeflow for K8s-native ML pipelines
Model registry	MLflow Model Registry, cloud registries	MLflow for portability; cloud-native for tight integration
Model serving	vLLM, TGI, KServe, Seldon Core, BentoML, cloud endpoints	vLLM/TGI for LLM serving; KServe for K8s; cloud endpoints for managed
Data quality / validation	Great Expectations, Evidently, Pandera	Great Expectations for schema; Evidently for drift + quality reports
Model format	ONNX, SafeTensors, GGUF, TorchScript	ONNX for portability; SafeTensors for safe LLM weights; GGUF for local inference

LLM-specific tools

Function	Tools	Choose when
LLM observability	Langfuse, LangSmith, Arize Phoenix, Helicone	Langfuse (open-source, self-host); LangSmith (LangChain teams); Arize (RAG debugging)
LLM gateway / proxy	LiteLLM, Portkey, TrueFoundry Gateway, custom	Multi-provider routing, failover, cost tracking, rate limiting
Prompt management	Langfuse, PromptLayer, Humanloop, Agenta	Version prompts like code; A/B test prompt variants
LLM evaluation	DeepEval / Confident AI (its managed platform), RAGAS, custom judges	DeepEval / RAGAS for RAG eval; LLM-as-judge for generation quality
Vector database	Qdrant, Pinecone, Weaviate, Chroma, pgvector	Qdrant (greenfield); pgvector (existing Postgres); Pinecone (managed)

Starter stack (if you're choosing today): Git + GitHub Actions + Docker + Terraform + MLflow + Evidently + vLLM + Langfuse + Qdrant covers 90% of production needs for both traditional ML and LLM systems. Add Kubernetes and Feast as you scale. Avoid tool sprawl — one tool per function.

05 — Cloud Infrastructure Selection

Training infrastructure

Option	Best for	Trade-off
Cloud managed (SageMaker/Vertex/Azure ML)	Most teams — spin up training, tear down after	20–40% premium over raw VMs but zero cluster management
Raw GPU VMs + custom stack	Teams with MLOps expertise, steady high utilization	Cheapest per hour but you manage scaling, fault tolerance, storage
TPU pods (GCP only)	Large-scale training (>10B params), research	Best $/FLOP for large models; requires JAX/XLA expertise
Spot/preemptible instances	Fault-tolerant training (checkpoint frequently)	60–91% cheaper but can be interrupted; use with checkpointing

Inference infrastructure

Pattern	Latency	Cost	Use when
Real-time endpoint (always-on)	Low (ms)	Steady — pay even at zero traffic	User-facing, low-latency requirements
Serverless inference (scale-to-zero)	Higher (cold start)	Pay per request	Dev/staging, low-traffic, variable demand
Batch inference	Hours	Cheapest per prediction	Overnight scoring, recommendations, ETL
Multi-model endpoint	Low	Shared infra (up to 80% savings)	Many models with moderate traffic each
Edge inference	Lowest	Device cost	Offline, privacy-sensitive, real-time on-device

Infrastructure decision tree

Question	Answer
User-facing, <100ms latency required?	Real-time endpoint (GPU)
Variable traffic, can tolerate cold starts?	Serverless inference
Bulk scoring, overnight, not time-sensitive?	Batch inference (cheapest)
Many models, moderate traffic each?	Multi-model endpoint
Privacy / offline / on-device?	Edge (quantized model)

Trade-off — managed vs self-hosted serving: Managed endpoint (SageMaker/Vertex/Azure ML) = autoscaling, monitoring, blue-green deployment built in — but higher per-hour cost and less control. Self-hosted (vLLM on K8s) = cheapest at scale, full control over batching/quantization — but you build your own scaling, health checks, and rollback. Start managed; migrate to self-hosted when monthly inference spend exceeds ~$10K and you have dedicated ops capacity.

06 — Model Deployment Patterns

The model packaging and deployment pipeline:

Trained model (from registry)
  → Containerize (Docker + serving runtime)
  → Push to registry (ECR / Artifact Registry)
  → Deploy to endpoint (real-time / batch / serverless)
  → Health check (smoke test + canary)
  → Live traffic

Serving runtimes for LLMs

Runtime	Best for	Key feature
vLLM	Production LLM inference — the 2026 default	PagedAttention, continuous batching, OpenAI-compatible API, ~2–4x throughput vs naive
TGI (Text Generation Inference)	Hugging Face models, production serving	Tensor parallelism, watermarking, streaming
Ollama	Local development and testing	One-command local LLM serving; not for production
TensorRT-LLM	Maximum NVIDIA GPU throughput	NVIDIA-optimized; best throughput but NVIDIA-only
Cloud endpoints	Managed, zero-ops serving	SageMaker/Vertex/Azure ML handle everything

07 — Release Strategies for Models

Strategy	How it works	Use when
Shadow deploy	New model receives real traffic but its predictions aren't served to users — only logged for comparison	First deployment of a new model; validate on real data without risk
Canary release	Route 5–10% of traffic to new model; watch metrics; gradually increase	Model updates; limit blast radius
Blue/green	Two full environments; instant switch + instant rollback	Critical models where fast rollback is essential
A/B testing	Route traffic by user segment; measure business outcomes over weeks	Competing model architectures; product experiments
Multi-armed bandit	Dynamically shift traffic toward the better-performing model variant	Recommendation / ranking models where real-time optimization matters

ML-specific release difference: In software, a canary catches code bugs. In ML, a canary catches accuracy degradation under real traffic — which is much harder to detect because the model returns HTTP 200 even when its predictions are wrong. You need quality metrics (accuracy, latency percentiles, business KPIs) in your canary monitoring, not just error rates.

08 — Monitoring & Observability

ML monitoring has four layers — each catches different failure modes.

Layer 1 — Infrastructure monitoring (DevOps standard): CPU/GPU utilization, memory, latency (p50/p95/p99), error rate, throughput (QPS), disk/network

Layer 2 — Model performance monitoring (ML-specific): prediction accuracy, precision / recall / F1, confidence distribution, prediction vs actual (when labels arrive)

Layer 3 — Data / feature monitoring (drift detection): input feature distributions, data schema validation, missing value rates, statistical drift tests (PSI, KL, JS)

Layer 4 — Business KPI monitoring (the only one that really matters): conversion rate, revenue impact, customer satisfaction, false positive cost, escalation rate

Monitoring tool stack

Layer	Open-source	Managed
Infrastructure	Prometheus + Grafana	Datadog, New Relic, CloudWatch, Cloud Monitoring
Model performance	Evidently, MLflow	Arize, WhyLabs, SageMaker Model Monitor, Vertex AI Model Monitoring
Data/drift	Evidently, Great Expectations	Arize, WhyLabs, cloud monitoring
Business KPIs	Grafana dashboards, custom metrics	Looker, Power BI, Datadog Business Monitoring
LLM-specific	Langfuse, Arize Phoenix	LangSmith, Helicone, Confident AI

Common mistake: Teams monitor Layer 1 (infrastructure) and think they're covered. The model returns HTTP 200 even when predictions are garbage. Infrastructure can be perfectly healthy while the model is silently failing. You need all four layers.

09 — Model Drift & Retraining

Three types of drift

Type	What shifts	Example	Detection
Data drift (covariate)	Input feature distributions change	Customer demographics shift; new product categories	Statistical tests (PSI, KL divergence, JS distance) on input features
Concept drift	Relationship between inputs and outputs changes	What "spam" looks like evolves; customer preferences shift	Monitor prediction accuracy vs actual labels (delayed)
Prediction drift	Output distribution changes	Model suddenly predicts mostly one class	Monitor output distribution (class balance, score histogram)

Retraining strategies

Whichever trigger you pick, what actually runs is a training job — the techniques, data curation, and eval gates behind it are in model training, fine-tuning and evaluation.

Strategy	When to use
Scheduled (time-based)	Stable environments; retrain weekly/monthly regardless. Simple, predictable.
Drift-triggered	When drift metrics cross a threshold → automatically kick off retraining pipeline
Performance-triggered	When accuracy/business KPI drops below a threshold (requires ground truth labels)
Continuous training	Models retrain on every new data batch (streaming or micro-batch). Most mature but complex.

Trade-off — retrain frequency: More frequent = more current model, less drift — but more compute cost, more operational complexity, and more risk of training on noisy/insufficient data. Less frequent = cheaper, simpler — but model degrades between retrains. Match frequency to data volatility: fraud models retrain weekly, product recs daily, medical models with regulatory review quarterly.

10 — LLMOps: What's Different for LLMs

LLMs add new concerns that traditional MLOps doesn't cover well. Think of LLMOps as an extension layer.

Concern	Traditional MLOps	LLMOps (what's new)
What you version	Code + data + model weights	+ prompts (system prompt, few-shot examples, templates)
Testing	Accuracy metrics on test set	+ eval harnesses (faithfulness, hallucination, safety, tool-use accuracy)
Deployment artifact	Containerized model	Often just API calls to a provider + prompt config — no weights to deploy
Cost model	Compute hours for training + inference	+ per-token costs that scale with prompt length and output length
Failure mode	Accuracy drops silently	+ hallucination (confidently wrong), prompt injection, tool misuse
Monitoring	Feature drift, accuracy	+ semantic drift (embedding space shifts), prompt regression, retrieval quality

LLMOps pipeline additions

Prompt versioning (track + A/B test)
  → LLM evaluation (judge + metrics)
  → Prompt caching (90% cheaper reads)
  → Model gateway (route + failover)
  → Guardrails (input/output filters)
  → LLM observability (traces + quality scores)

LLM observability: what to monitor

Quality metrics: faithfulness (is the output grounded in context?), relevance (does it answer the question?), safety (no toxicity/PII/bias).
RAG metrics: retrieval precision, context relevance, groundedness — is the retrieval pipeline returning useful chunks?
Operational metrics: tokens per request, time to first token (TTFT), cache hit rate, cost per session, agent step count.
Drift metrics: embedding drift (semantic shift in queries/outputs), prompt regression (quality drops after prompt edit), model version regression.
Agent metrics: tool selection accuracy, planning quality, loop count, escalation rate.

The key insight: LLMs fail silently — a hallucinated answer returns HTTP 200. Infrastructure monitoring cannot catch this. You need quality scoring on every trace (or a sample). The best practice is to run an LLM-as-judge on 5–10% of production traces and alert on statistically significant quality drops.

11 — Cost Optimization

Lever	Savings	How
Model right-sizing	60–80%	Use Haiku for routing, Sonnet for main work, Opus only for hardest tasks
Prompt caching	90% on cached reads	Cache static system prompts, tool definitions, reference docs. Default TTL is 5 minutes; a longer option exists at a higher write premium, which pays off only for traffic bursty enough to otherwise let the cache expire
Batch API	50%	Async processing for non-real-time workloads (24-hr window)
Spot/preemptible GPUs	60–91%	For training jobs with checkpointing; not for serving
Quantization	2–4x throughput	INT8/INT4 quantization reduces model size; slight accuracy trade-off
Token budgets	Variable	Set per-user/per-team/per-project token limits; alert on anomalies
Progressive summarization	Variable	Compress older conversation turns; reduces per-request token count
Committed/reserved capacity	30–70%	SageMaker Savings Plans, GCP CUDs, Azure EA for steady-state workloads
Auto-scaling + scale-to-zero	Variable	Serverless inference for dev/staging; autoscale production endpoints

Measure cost per outcome, not cost per token: A cheaper model that gets the answer wrong 30% of the time costs more than an expensive model that's right 95% of the time — because you pay for retries, escalations, and lost customers. Track cost per successful resolution, not just token spend.

12 — Security & Governance

Version everything: code (Git), data (DVC/LakeFS), models (MLflow registry), prompts (Langfuse/version control), infrastructure (Terraform). If you can't reproduce it, you can't audit it.
Access control: RBAC on model registry (who can promote to production?), IAM on training resources, API auth on serving endpoints.
Data lineage: track which data trained which model, which model is serving which endpoint. Metadata stores (MLflow, cloud registries) maintain this.
Audit trail: log every training run, evaluation result, deployment, and rollback. Required for EU AI Act, HIPAA, SOC2.
Model cards: document each model's purpose, training data, known limitations, fairness evaluations, and intended use. This is both good practice and increasingly required by regulation.
Supply chain security: scan model weights for trojans (backdoors injected during training). Don't download untrusted models from the internet without verification. Use SafeTensors format, not pickle.
Prompt injection defense: input validation + output verification + separation of trusted/untrusted content + guardrail hooks.

The EU AI Act: High-risk AI systems (Annex III) must demonstrate: transparency, explainability, human oversight, data governance, accuracy/robustness testing, and a conformity assessment. Note the timeline has shifted — the May 2026 "Digital Omnibus" postponed the high-risk (Annex III) obligations from August 2, 2026 to December 2, 2027, so the near-term compliance deadline is no longer 2026. That's more runway, not a reprieve: building MLOps pipelines that version, test, monitor, and audit is how you demonstrate compliance, and it's far easier to bake in now than to retrofit later. If your pipeline can't reproduce a training run and explain a prediction, you won't be EU AI Act compliant when the obligations land.

13 — Trade-off Master Reference

Decision	Option A	Option B	Default
Managed platform vs self-hosted	SageMaker / Vertex / Azure ML	Raw VMs + custom stack	Managed unless >$10K/mo + dedicated MLOps team
Training on spot vs on-demand	Spot (60–91% off)	On-demand (guaranteed)	Spot with checkpointing for training; on-demand for serving
Real-time vs batch inference	Real-time endpoint	Batch transform	Real-time for user-facing; batch for scoring/ETL
Model serving: vLLM vs managed	Self-hosted vLLM on K8s	Cloud managed endpoint	Managed to start; self-hosted when scale justifies ops cost
Experiment tracking	MLflow (open-source)	W&B (managed)	MLflow for portability and cost; W&B for team collaboration
Pipeline orchestration	Airflow / Prefect	Cloud-native (Vertex/SageMaker Pipelines)	Cloud-native for single-cloud; Airflow for multi-cloud
Monitoring: open-source vs managed	Prometheus + Grafana + Evidently	Datadog + Arize	Open-source for cost; managed for faster setup
LLM observability	Langfuse (self-hosted)	LangSmith / Arize (managed)	Langfuse for data sovereignty; managed for speed
Retraining trigger	Scheduled (time-based)	Drift-triggered (automated)	Scheduled to start; drift-triggered at maturity (Level 2)
Model release strategy	Blue/green (instant rollback)	Canary (gradual rollout)	Shadow first → canary for ongoing updates → blue/green for critical
IaC tool	Terraform (multi-cloud)	CloudFormation / Bicep (cloud-native)	Terraform unless deeply single-cloud
Container orchestration	Kubernetes	Serverless (Cloud Run / Lambda)	K8s for complex serving; serverless for simple functions
CI/CD platform	GitHub Actions	GitLab CI / Jenkins	GitHub Actions is the 2026 default; Jenkins for legacy/complex
Data versioning	DVC (lightweight)	LakeFS / Delta (lake-scale)	DVC for small teams; LakeFS when data exceeds TB scale

Built from current industry practice and production patterns. Independent reference — not affiliated with any vendor. Tools and pricing change rapidly; verify before procurement decisions.

Riddam Jain — Playbooks

Model Training, Fine-Tuning & Evaluation — 2026 Guide

01 — The Training Landscape

02 — Fine-Tuning Techniques

How LoRA works (the key insight)

03 — Alignment & Preference Tuning

04 — Training Frameworks & Tools

Supporting tools

05 — Data Preparation & Curation

Data formats by training type

Data quality checklist

Synthetic data (the 2026 shortcut)

06 — Distributed Training

Key training optimizations

07 — Evaluation Harnesses & Benchmarks

Evaluation frameworks

Key benchmarks to know

08 — Model Optimization (Post-Training)

09 — Open-Weight Ecosystem

10 — The Complete Training Workflow

11 — Cloud Training Services

12 — Trade-off Master Reference

AI Architecture: A Practitioner's Field Guide (2026)

01 — The AI Architecture Landscape

02 — Foundation Models & Selection

The major model families (2026)

Model selection decision framework

03 — The Customization Ladder

The decision flowchart

04 — Prompt Engineering (Production Patterns)

Core techniques

The PRECISE framework (community mnemonic)

Structured output patterns

05 — RAG Architectures

The RAG maturity ladder

RAG pipeline components

06 — Fine-Tuning

When to fine-tune (and when not to)

Fine-tuning techniques

07 — Agentic AI Patterns

When to use agentic vs simpler patterns

The agentic loop

What's new in agent loops (2026)

Agent frameworks (2026)

08 — Multi-Agent Systems

Critical design principles

09 — Tool Use & MCP

Tool use — what it is

MCP (Model Context Protocol)

10 — LLMOps & Evaluation

The LLMOps lifecycle

Evaluation (the hardest part)

Model gateway pattern

11 — AI Safety & Guardrails

Defense in depth — layered guardrails

Key safety concepts

12 — Enterprise AI Governance

13 — Infrastructure & Serving

Inference infrastructure

GPU/TPU landscape

14 — Multimodal AI

Multimodal RAG

15 — Cloud AI Platforms — GCP / AWS / Azure

The three platforms at a glance

Service mapping — pattern → cloud service

Platform selection decision tree

Training infrastructure trade-offs

Inference infrastructure trade-offs

Multi-cloud AI patterns

16 — Trade-off Master Reference

17 — Books & Learning Path

Tier 1 — Foundations (start here)

Tier 2 — Production AI engineering

Tier 3 — Agents & multi-agent systems

Tier 4 — Specialized depth

Tier 5 — Data & infrastructure foundations

Tier 6 — Cloud AI platforms & MLOps

Free & open resources

Recommended reading order

CCAR-F: Claude Certified Architect Foundations — 2026 Blueprint