4. Phase 2 — AI, ML, Transformer & LLM Foundations

Estimated time: 2–3 weeks.

💡 Framing You need enough ML/math to reason correctly about AI systems. You do not need to finish a university ML curriculum before building LLM applications. This chapter is deliberately conceptual — deep mathematical derivations are deferred to Chapter 14 (Advanced/PyTorch) for students who choose the ML Engineer path later.

4.1 Machine Learning Basics

Definitions and Worked Explanations

Supervised vs unsupervised learning. In supervised learning, the model learns from labeled examples (input paired with a known correct output) — e.g. an email marked "spam" or "not spam." In unsupervised learning, the model finds structure in unlabeled data — e.g. clustering customers by purchase behavior with no predefined categories. Classification predicts a discrete category (spam/not spam); regression predicts a continuous number (predicted house price).

Training/validation/test sets and data leakage. Data is split so the model is fit on the training set, tuned (hyperparameters, model choice) on the validation set, and given a final, honest performance check on the test set, which it never touches during development. Data leakage occurs when information from validation/test data — or from the future — improperly influences training, producing performance numbers that look great in development but collapse in production. Example: including a "cancelled" flag that is only set after the outcome you're trying to predict already happened.

⚠️ Common Mistake Evaluating an LLM application only on the same handful of examples used while writing the prompt. This is the AI-application equivalent of data leakage — the prompt gets implicitly "tuned" to those examples, and the reported quality does not generalize. Chapter 11 (Evaluation) formalizes the fix: held-out eval sets and regression suites.

Overfitting vs underfitting. Overfitting: the model memorizes noise/specifics of the training data and performs poorly on new data. Underfitting: the model is too simple to capture the real pattern and performs poorly everywhere. Regularization techniques discourage overly complex models to reduce overfitting.

Loss functions and gradient descent. A loss function is a formula measuring how wrong the model's predictions currently are. Gradient descent is the optimization procedure that repeatedly adjusts model parameters in the direction that reduces the loss, using the gradient (the loss function's slope) as the guide. Intuition: imagine standing on a hill in fog and taking small steps downhill using only what you can feel under your feet — that is gradient descent.

Precision, recall, F1, ROC/AUC — and why accuracy can mislead.

MetricFormula (conceptual)Answers
PrecisionTrue positives / (True positives + False positives)Of everything flagged positive, how much was correct?
RecallTrue positives / (True positives + False negatives)Of everything actually positive, how much did we find?
F1Harmonic mean of precision and recallA single balance score between the two
ROC/AUCArea under true-positive-rate vs false-positive-rate curveHow well the model ranks positives above negatives across thresholds

Worked example. A fraud detector sees 1,000 transactions, 5 of which are fraudulent. A model that predicts "not fraud" every single time achieves 99.5% accuracy — while catching zero fraud. This is why accuracy alone is misleading on imbalanced data; precision/recall/F1 expose what accuracy hides.

Distribution shift. Production behavior can differ from benchmarks when the real-world input distribution differs from what the model was trained/evaluated on — e.g. a support-ticket classifier trained on 2023 tickets facing an influx of a brand-new product category in 2026. This concept directly motivates the eval-and-monitor discipline in Chapter 11: benchmark performance is a snapshot, not a guarantee.

4.2 Neural-Network Basics

Prerequisite Concept — Tensors

Definition. A tensor is a multi-dimensional array of numbers — a scalar is a 0-D tensor, a vector is 1-D, a matrix is 2-D, and higher-dimensional tensors (3-D+) are used for batches of images, sequences of token embeddings, and so on. Every input, weight, and intermediate value in a neural network is represented as a tensor.

Input tokens │ ▼ Embedding layer (token id -> vector) │ ▼ Hidden layers (weights + biases + activation functions) │ ▼ Output layer (raw scores, "logits") │ ▼ Loss function (compares output to correct answer, during training) │ ▼ Backpropagation (computes how each weight should change) │ ▼ Optimizer step (updates weights slightly)
Fig 4.1 — One training step: forward pass produces a prediction, backpropagation computes gradients, the optimizer nudges weights to reduce future loss.

Forward pass, backpropagation, optimizer. The forward pass pushes input data through the network's layers to produce a prediction. Backpropagation is the algorithm that computes, layer by layer working backward, how much each weight contributed to the error — using calculus (the chain rule) to compute gradients efficiently. The optimizer (e.g. Adam, SGD) then uses those gradients to update the weights. Inference is using a trained model to make a prediction with no weight updates (only a forward pass); training repeats forward pass + backprop + optimizer step across many examples.

Batching, epochs, learning rate. A batch is a group of training examples processed together for efficiency. An epoch is one full pass through the entire training dataset. The learning rate controls how large each optimizer step is — too high and training becomes unstable; too low and training is painfully slow.

4.3 Transformer / LLM Fundamentals

Tokenization

Definition. Tokenization breaks text into sub-word units ("tokens") the model actually processes — often not the same as words. "Unbelievable" might tokenize as ["Un", "believ", "able"]. Roughly, English text averages ~0.75 words per token.

Why it matters. Every dollar and every latency millisecond in an LLM API call is billed and bounded by token count, not character or word count — this is why Chapter 5 emphasizes "usage accounting" (input/output/cached tokens) as a first-class engineering concern, not an afterthought.

📌 Sticky Note — Remember Context window = total tokens (input + output) a model can process in one call.
Latency = how long one operation takes.
Throughput = how many operations complete over a period of time.
These are related but distinct — a model can have low latency per call and low throughput if it can't handle concurrent requests well, or vice versa.

Embeddings (Preview)

Definition. An embedding is a fixed-length vector of numbers that represents the meaning of a piece of text (or image, audio, etc.), positioned in a high-dimensional space such that semantically similar inputs produce vectors that are geometrically close together. This concept is used constantly from Chapter 6 onward, but originates architecturally inside the transformer itself (the embedding layer in Fig 4.1). A full treatment — including cosine similarity and vector databases — is given in Chapter 6, since those are prerequisite to RAG rather than to core LLM behavior.

Attention and Self-Attention

Simple analogy. Reading the sentence "The trophy didn't fit in the suitcase because it was too big" — to know what "it" refers to, you unconsciously weigh every other word in the sentence and decide "trophy" matters far more than "suitcase" for resolving "it." Self-attention is the mechanism that lets a transformer do exactly this, mathematically, for every token against every other token.

Technical definition. Self-attention computes, for each token, a weighted combination of all other tokens' representations, where the weights ("attention scores") are learned and depend on the content of the tokens themselves — allowing the model to dynamically decide which parts of the input are relevant to each other, regardless of their distance in the sequence.

Token: "The" "trophy" "didn't" "fit" "in" "the" "suitcase" "because" "it" "was" "too" "big" Attention from "it": low HIGH low low low low low low - low low low
Fig 4.2 — Self-attention assigns higher weight to "trophy" than "suitcase" when resolving the pronoun "it", based on learned patterns.

Transformer Blocks and Why Parallelism Mattered

Before transformers, sequence models (RNNs/LSTMs) processed tokens one at a time, in order — token 5 could not be processed until token 4 finished, which made training slow and hard to scale on GPUs. Transformers process all tokens in a sequence simultaneously using self-attention plus feed-forward layers stacked into repeating "blocks." This parallelism is what allowed transformer models to scale to the sizes and datasets that produced modern LLMs.

Pretraining, Instruction Tuning, Preference Optimization, Fine-Tuning

StageWhat HappensGoal
PretrainingTrain on massive, broad text/code corpora to predict the next tokenGeneral language competence and world knowledge
Instruction tuningFine-tune on (instruction, ideal response) pairsFollow instructions rather than just continue text
Preference optimization (e.g. RLHF/DPO)Train the model to prefer outputs humans rate as betterHelpfulness, safety, tone alignment
Fine-tuning (task-specific)Further training on a narrow labeled dataset for one company/taskSpecialization (see Chapter 14)

Context Windows and Long-Context Limits

Why more context is not always better. Even when a model technically accepts very large contexts, retrieval and reasoning quality across that context is not uniform — models often attend more reliably to information near the beginning and end of a long context than to information buried in the middle (a pattern sometimes called "lost in the middle"). Stuffing irrelevant text into context also increases cost, latency, and the chance of the model being distracted by noise. This is the conceptual foundation for Chapter 7's insistence on retrieving "fewer, better chunks" rather than maximizing context usage.

Inference, Sampling, Determinism, Reasoning Effort

Definition. Inference is the process of generating output from a trained model. Sampling controls how the next token is chosen from the model's predicted probability distribution — parameters like temperature control how random vs. predictable the choice is. Lower temperature (near 0) makes output more deterministic and repetitive; higher temperature increases variety and creativity, at the cost of consistency. Some models expose a "reasoning effort" setting that trades latency/cost for more internal deliberation before answering.

Hallucination, Calibration, Knowledge Cutoffs

Definition. A hallucination is a fluent, confident output that is factually incorrect or unsupported by any real source — the model is not "lying," it is generating the statistically plausible continuation of text, which is not the same as generating a verified fact. Calibration refers to how well a model's expressed confidence matches its actual correctness rate; LLMs are frequently poorly calibrated, sounding equally confident whether right or wrong. A knowledge cutoff is the date after which the model has no training data, so it cannot know about events after that point unless given external context (this is a core motivation for RAG, Chapter 7).

💡 Interview Tip If asked "Why do LLMs hallucinate?", a strong answer connects mechanism to consequence: "The model is trained to predict plausible next tokens, not to verify facts against a source of truth — so when it lacks reliable information, it still produces fluent text that can be wrong. This is why production systems ground answers in retrieved evidence (RAG) and evaluate for faithfulness rather than trusting the model's unaided knowledge."

Multimodal Capabilities and Hosted vs Open-Weight Models

Modern models increasingly accept and/or produce multiple modalities — text, vision (images), audio (speech), and combinations thereof — under one architecture family, rather than requiring entirely separate specialized models (detailed further in Chapter 14). Separately, engineers choose between hosted proprietary models (accessed via API, no infrastructure to manage, cannot see or modify weights) and open-weight models (downloadable weights, can self-host, fine-tune freely, but require infrastructure and expertise) — a decision revisited in Chapters 13–14.

4.4 Math to Learn Now — and What Can Wait

Learn NowUnderstandCan Wait
Vectors, matrices, dot product, cosine similarityEnough to understand embeddings and retrievalProof-heavy linear algebra
Probability, conditional probability, distributionsEnough to reason about uncertainty and metricsAdvanced probability theory
Mean, variance, correlation, samplingEnough for experiments/evalsGraduate statistics
Derivative/gradient intuitionEnough to understand optimizationAdvanced calculus derivations
🚧 Do Not Get Stuck in Prerequisite Hell You need enough ML/math to reason correctly about AI systems. You do not need to finish a university ML curriculum before building LLM applications. If you notice yourself reading a third linear-algebra textbook before writing a single line of application code, that is a signal to stop and start building.

Common Questions

Q: Why would we choose a smaller/faster model instead of the largest available model?
Task difficulty varies; routing simple tasks to smaller models reduces cost and latency with no meaningful quality loss, reserving the strongest model for tasks that actually need it (formalized as a cost/latency engineering pattern in Chapter 13).
Q: What happens if a prompt exceeds the context window?
The request is typically rejected or truncated by the API, which is why token accounting and context-size management are treated as production engineering concerns, not just quality concerns.
Q: What is the biggest limitation of LLMs that engineers must design around?
Hallucination combined with poor calibration — the model can be confidently wrong with no built-in signal to distinguish confident-correct from confident-wrong output, which is why grounding (RAG) and evaluation are treated as mandatory, not optional.

Interview / Viva Questions

Q: Explain self-attention in one or two sentences suitable for a non-ML interviewer.
Self-attention lets each token in a sequence look at every other token and learn how much to "weigh" each one when building its own representation, which is how transformers capture long-range relationships like pronoun resolution.
Q: Why did transformers scale better than RNNs?
RNNs process tokens sequentially, which prevents parallel computation across the sequence during training; transformers process all tokens simultaneously via self-attention, which is far more GPU-parallelizable and enabled training on much larger datasets.
Q: What is the difference between pretraining and fine-tuning?
Pretraining builds general language competence from massive broad data; fine-tuning further trains an already-pretrained model on a smaller, task- or domain-specific dataset to specialize its behavior.
Q: Why is accuracy sometimes a misleading metric?
On imbalanced datasets, a model can achieve high accuracy by always predicting the majority class while completely failing at the minority class that actually matters (e.g. fraud detection), which is why precision, recall, and F1 are used alongside accuracy.
Q: What is distribution shift, and why should an Applied AI Engineer care?
It's when real-world production input differs from the data a model was trained or evaluated on, causing benchmark performance to not hold in production — it's a direct argument for continuous evaluation and monitoring rather than a one-time launch check.

Chapter Summary