23. Final Revision Guide

Concise revision notes organized by topic, covering the entire guide.

23.1 Backend Foundations

Type hints + Pydantic validate data shape at runtime — the same mechanism used later to validate AI-generated output.
async/await is required for I/O-bound model/tool calls so one slow call doesn't block other requests.
Authentication (who you are) ≠ Authorization (what you can do) — both required, checked in code, not in prompts.
Indexes prevent full table scans; transactions keep multi-step writes consistent.

23.2 LLM Foundations

Tokenization drives cost/latency; tokens ≠ words.
Self-attention lets every token weigh every other token — the mechanism behind transformer scaling.
Pretraining → instruction tuning → preference optimization → (optional) task-specific fine-tuning.
Hallucination + poor calibration are structural, not bugs — ground answers with RAG and verify with evals.
Context windows have practical attention limits — retrieve fewer, better chunks rather than maximizing stuffed context.

23.3 Prompting & Structured Outputs

Build on raw SDKs before frameworks.
Keep trusted system instructions separate from untrusted user/retrieved content.
Never trust generated text as executable/structured data without explicit schema validation.
Version prompts like code; evaluate before release.

23.4 Embeddings & Vector Search

Embeddings place similar meanings close together in vector space; never mix vectors from different embedding models.
Cosine similarity is the standard metric for text embeddings.
ANN indexing trades a little recall for a lot of speed at scale.
Semantic search misses exact terms; lexical search misses paraphrases — hence hybrid search.
Start with PostgreSQL + pgvector.

23.5 RAG

Two pipelines: async ingestion (parse → chunk → embed → store) and online query (retrieve → fuse → rerank → generate → cite).
Chunking strategy determines retrieval precision; preserve source offsets for citations.
Hybrid search (dense + sparse via RRF) beats either alone.
Rerankers refine a cheap shortlist, not the whole collection.
Evaluate retrieval, reranking, generation, and end-to-end separately — most failures are retrieval failures.
Faithfulness (supported by evidence) ≠ correctness (matches real-world truth).

23.6 Tools, Workflows & Agents

Tools are narrow, typed, single-purpose; never expose raw SQL/shell access.
Authorization for tool calls is enforced in code, always.
Prefer deterministic workflows; use agents only where genuine step-to-step judgment is needed.
Agents need explicit stopping conditions, checkpoints, and human-in-the-loop approval for sensitive actions.
Treat all tool/document output as untrusted data, never as instructions.

23.7 MCP

Standardizes tool/resource/prompt discovery and invocation — not safety.
Every tool-design and security rule from Chapters 8/12 still applies to MCP servers.

23.8 Evaluation & Observability

Non-deterministic systems need repeatable regression evals, not manual spot-checks.
Every production failure becomes a new eval case.
LLM-as-judge must be calibrated against human agreement.
Trace every request end-to-end; redact sensitive data from logs.
No change ships without passing the regression suite.

23.9 Security

AI security is additive to traditional AppSec, not a replacement.
Prompt injection: direct (user message) vs indirect (hidden in retrieved/tool content).
Excessive agency and unbounded consumption are mitigated by least privilege, approval gates, and hard budgets.
Apply authorization filters before retrieval, not after generation.

23.10 Production & Cloud

Long-running ingestion belongs in a durable queue with separate workers.
Reliability patterns: timeouts, backoff+jitter, idempotency keys, circuit breakers.
Cost/latency engineering: route by difficulty, cache, control context size, measure cost per successful task.
Docker + managed container platform is enough for most roles; Kubernetes is optional early.

23.11 Advanced Topics

Fine-tuning is for behavior/style/domain specialization — not for missing facts (that's RAG).
LoRA/QLoRA make fine-tuning feasible on modest hardware.
Catastrophic forgetting requires evaluating general capability, not just the target task.
Benchmark any candidate model on your own eval set, not public leaderboards.

24. Cheat Sheet

24.1 Key Formulas

Concept	Formula / Rule
Cosine similarity	`cos(θ) = (A·B) / (‖A‖‖B‖)` — range −1 to 1, higher = more similar
Precision	True positives / (True positives + False positives)
Recall	True positives / (True positives + False negatives)
F1	Harmonic mean of precision and recall
Recall@k	Fraction of all relevant docs found within top k results
Exponential backoff	Wait time doubles each retry (+ random jitter): 1s, 2s, 4s, 8s…

24.2 Key Architectures at a Glance

Structured output: Input -> Prompt+Schema -> Model -> Pydantic validate -> Business rules -> DB RAG query: Question -> rewrite -> retrieve -> fuse -> rerank -> assemble context -> generate -> cite Tool call: Model proposes -> validate args -> authorize -> execute -> validate result -> audit log Agent loop: inspect state -> choose action -> execute -> observe -> update state -> repeat/stop Safe execution: auth -> authorize -> propose -> validate -> policy check -> approve if sensitive -> execute -> audit Ingestion job: upload -> object storage -> queue -> worker parses/chunks/embeds -> store -> status update

24.3 Key Differences (Rapid Recall)

A vs B	Core Distinction
Authentication vs Authorization	Who you are vs what you can do
Concurrency vs Parallelism	Interleaved progress (often 1 core) vs simultaneous execution (multiple cores)
Dense vs Sparse retrieval	Semantic/meaning match vs exact/keyword match (BM25)
Workflow vs Agent	Fixed step sequence (engineer-controlled) vs model chooses next action each step
Faithfulness vs Correctness	Supported by given evidence vs true in the real world
Direct vs Indirect prompt injection	From user's own message vs hidden in retrieved/tool content
Fine-tuning vs RAG	Bakes behavior/style into weights (slow to update) vs external, swappable, citable knowledge
Hosted vs Open-weight models	No infra, less control vs full control, more operational burden
LoRA vs full fine-tuning	Trains small adapter matrices vs updates all parameters

24.4 Important Rules

Never expose raw SQL/shell/filesystem tools to a model.
Never trust model-generated text as executable data without schema validation.
Apply permission filters before retrieval, not after generation.
Enforce tool authorization in code, never rely on prompt instructions alone.
Every production AI failure becomes a permanent regression eval case.
No prompt/model/retrieval change ships without passing the regression suite.
Redact sensitive data from observability logs.
Give every write tool an idempotency key or equivalent safeguard.

24.5 Common Interview Points (Rapid Fire)

Why RAG over fine-tuning for private docs → external, current, citable, cheaper to update.
Why hybrid search → dense and sparse retrieval have complementary blind spots.
Why deterministic workflows before agents → simpler to test, debug, secure.
Why evals are mandatory → LLM systems are probabilistic and can silently regress.
Why "cost per successful task" not "cost per call" → cheap-but-unreliable can cost more overall.

25. Glossary

Agent: A system that uses a model to choose actions/tools over multiple steps toward a goal.
ANN (Approximate Nearest Neighbor): Fast, index-based search for near-optimal similarity matches, trading some recall for speed.
API: Application Programming Interface.
Attention / Self-attention: Mechanism letting each token weigh the relevance of every other token when building its representation.
BM25: A classic keyword ranking method used in lexical search.
Cache: Temporary high-speed storage used to reduce repeated expensive operations.
Calibration: How well a model's expressed confidence matches its actual correctness rate.
Catastrophic forgetting: Degradation of a model's prior general capability caused by aggressive narrow fine-tuning.
Checkpoint: A saved snapshot of an agent/workflow's state enabling pause and resume.
Chunk: A portion of a source document stored/retrieved for RAG.
Circuit breaker: A pattern that stops calling a failing downstream service for a cooldown period before testing recovery.
Cosine similarity: A metric measuring the angle between two vectors, used to compare embeddings.
Context window: The amount of input/output context a model can process in one interaction.
Cross-encoder: A model that jointly scores a (query, document) pair for precise relevance — used in reranking.
Distribution shift: When production input differs from training/evaluation data, causing benchmark performance to not hold.
Embedding: A vector representation used to compare semantic similarity.
Eval: A repeatable test used to measure model/application behavior.
Excessive agency: Granting a model/agent more autonomy or tool access than its task requires.
Faithfulness: Whether a generated answer is actually supported by the retrieved evidence supplied to the model.
Fine-tuning: Further training a pretrained model on a narrower, task-specific dataset.
Function/tool calling: A mechanism for a model to request structured calls to application-provided tools.
Grounding: Constraining or supporting model output with external evidence/data.
Hallucination: Fluent, confident model output that is factually incorrect or unsupported by any real source.
Hybrid search: Combining semantic/vector and lexical/keyword retrieval.
Idempotency: Design property where repeating the same request does not create unintended duplicate side effects.
Indirect prompt injection: Prompt injection hidden inside retrieved documents, webpages, or tool output rather than the user's direct message.
LLMOps: Operational practices for deploying, monitoring, evaluating and governing LLM applications.
LLM-as-judge: Using a model to score/compare outputs against a rubric, in place of exhaustive human review.
LoRA: Low-Rank Adaptation — a parameter-efficient fine-tuning method that trains small adapter matrices instead of all model weights.
MCP: Model Context Protocol; an open standard for exposing tools/resources/workflows to AI applications.
Multi-tenant isolation: Ensuring one customer/organization's data cannot be accessed by another in a shared deployment.
Quantization: Representing model weights with lower numerical precision to reduce memory/compute requirements.
RAG: Retrieval-Augmented Generation; retrieve relevant evidence and provide it to a model before generation.
Reranker: A second-stage model/algorithm that reorders retrieved candidates by relevance.
RRF (Reciprocal Rank Fusion): A method for combining ranked result lists from different retrieval methods (e.g. dense + sparse) into one.
Structured output: Model output constrained to a schema such as JSON that can be validated programmatically.
Token: A sub-word unit of text that a model actually processes; the basis for cost and context-window accounting.
Tool gateway: Application layer that validates, authorizes and executes model-requested actions.
Tracing: Recording the structured, end-to-end path of everything that happened while handling a single request.
Vector database: A database/search engine optimized for vector similarity retrieval.

26. Final Practice Questions (with Answers)

26.1 Beginner / Foundation Questions

Q1: What is the difference between authentication and authorization?

Authentication verifies who a user is; authorization determines what actions that user is permitted to perform.

Q2: Why are model API calls typically written as async functions?

Because they are I/O-bound and can take seconds; async code frees the server to handle other requests while waiting.

Q3: What does "tokenization" mean, and why does it matter for cost?

It's the process of breaking text into sub-word units a model processes; providers bill and bound requests by token count, not words or characters.

Q4: What is an embedding?

A fixed-length numeric vector representing the meaning of a piece of content, positioned so similar meanings are geometrically close together.

Q5: What is the purpose of a database index?

It avoids full table scans by letting the database quickly locate rows matching a query, at the cost of extra storage and slower writes.

26.2 Intermediate Questions

Q6: Compare dense and sparse retrieval and explain why hybrid search combines them.

Dense (embedding-based) retrieval matches on semantic meaning even without shared words; sparse (BM25) retrieval matches exact/rare terms precisely. Hybrid search fuses both ranked lists (e.g. via Reciprocal Rank Fusion) to capture the strengths of each.

Q7: Why is "cost per successful task" a better metric than "cost per API call"?

A cheaper model with a lower success rate can cost more overall once failed attempts requiring rework are accounted for; cost per successful outcome reflects the true economics.

Q8: What is the difference between a deterministic workflow and an agent?

A workflow's step sequence is fixed by the engineer in advance; an agent uses the model to decide the next action at each step based on current state.

Q9: Explain why chunk overlap involves a trade-off.

More overlap reduces the risk of relevant content being split across chunk boundaries and missed by retrieval, but increases storage and duplicate content passed into generation context, raising cost.

Q10: What is the purpose of a circuit breaker pattern?

It stops sending requests to a failing downstream dependency after a failure threshold, failing fast and giving the dependency room to recover, instead of piling up retries.

26.3 Advanced Questions

Q11: Explain the two-stage retrieve-then-rerank pattern and why rerankers aren't applied to the entire collection.

A cheap first-stage search (hybrid dense+sparse) narrows millions of documents to a shortlist (e.g. top 50); a more computationally expensive cross-encoder reranker then precisely reorders just that shortlist, since scoring every document with a cross-encoder would be prohibitively slow.

Q12: What is catastrophic forgetting, and how do LoRA-style methods relate to it?

Catastrophic forgetting is the loss of a model's general capability after aggressive fine-tuning on a narrow task. Parameter-efficient methods like LoRA train small adapter matrices while freezing the original weights, which tends to preserve more of the base model's general capability than full fine-tuning.

Q13: Why must permission filters be applied during retrieval rather than after generation in a multi-tenant RAG system?

Filtering after generation means unauthorized content was already fetched and potentially processed/exposed; filters must constrain what can be retrieved in the first place to prevent any unauthorized data from entering the pipeline at all.

Q14: Distinguish faithfulness from correctness in RAG evaluation, and explain why an answer can be one without the other.

Correctness measures agreement with real-world truth; faithfulness measures whether the answer is actually derived from the supplied evidence. An answer can be correct by coincidence without being grounded in the given context (unfaithful), which signals unreliable grounding likely to fail on other queries.

26.4 Scenario-Based Questions

Q15: Your application has one product remaining and 20 users attempt to purchase it simultaneously. How would you prevent overselling?

Use a database transaction with row-level locking (or an atomic conditional update, e.g. "decrement stock where stock > 0") so only one of the 20 concurrent requests succeeds; for distributed systems spanning multiple services, a distributed lock or a serializable transaction/queue-based reservation system may be needed instead.

Q16: A support-ticket RAG chatbot starts giving confidently wrong answers after a new batch of documents was ingested. What's your debugging sequence?

Inspect the retrieved chunks for recent queries first — check whether the new documents were chunked/embedded correctly, whether retrieval surfaces them, and whether metadata/permissions are correct — before touching the generation prompt, since most such regressions originate in ingestion or retrieval.

Q17: An internal agent with a "send email" tool starts sending unintended emails after processing a batch of scraped web content. What happened, and how do you fix it?

This is very likely indirect prompt injection — the scraped content contained instructions the agent treated as commands. Fix by treating all such content as untrusted data, adding an approval gate for the send-email tool, and validating tool arguments (e.g. recipient allow-lists) independently of model output.

Q18: You need to process a 500-page PDF without blocking the upload request. Describe the architecture.

The upload endpoint saves the file to object storage, creates an ingestion job, and enqueues it, returning a job ID immediately. A separate worker process dequeues the job, parses/OCRs if needed, chunks, embeds, and persists the chunks with metadata, updating job status as it progresses; the client polls or subscribes to status updates.

26.5 Viva / Interview Questions (Short-Answer)

Q19: What is MCP, in one sentence?

An open standard that lets AI applications discover and invoke external tools, resources, and prompt templates without bespoke point-to-point integrations.

Q20: Name three AI-specific security threats not present in traditional web applications.

Prompt injection (direct/indirect), excessive agency (over-privileged model actions), and RAG/data poisoning (malicious content injected into a knowledge base to influence future answers).

Q21: When should you reach for an agent instead of a deterministic workflow?

Only when the task genuinely requires the model to decide, at runtime, what steps to take next based on evolving state — not for predictable, well-defined business processes, which are better served by deterministic workflows with the model used only for bounded sub-tasks.

Q22: What single practice most improves an eval suite over time?

Converting every real production failure into a permanent regression eval case.

End-of-Guide Summary

This guide has walked the full Applied AI Engineer path: from ordinary backend foundations, through LLM/transformer theory, prompting and structured outputs, embeddings and vector search, the complete RAG lifecycle, tool calling and safe workflows, agents and LangGraph, MCP, evaluation and observability, AI security, production/cloud engineering, and multimodal/fine-tuning/open-model topics — followed by the 24-week execution plan, portfolio projects, interview preparation, and this revision material.

The throughline across every chapter is the same: treat AI output as an untrusted, probabilistic component inside an otherwise ordinary, disciplined software system — validate it, evaluate it, secure it, and observe it exactly as rigorously as any other part of a production backend.