Estimated time: 2–3 weeks.
Supervised vs unsupervised learning. In supervised learning, the model learns from labeled examples (input paired with a known correct output) — e.g. an email marked "spam" or "not spam." In unsupervised learning, the model finds structure in unlabeled data — e.g. clustering customers by purchase behavior with no predefined categories. Classification predicts a discrete category (spam/not spam); regression predicts a continuous number (predicted house price).
Training/validation/test sets and data leakage. Data is split so the model is fit on the training set, tuned (hyperparameters, model choice) on the validation set, and given a final, honest performance check on the test set, which it never touches during development. Data leakage occurs when information from validation/test data — or from the future — improperly influences training, producing performance numbers that look great in development but collapse in production. Example: including a "cancelled" flag that is only set after the outcome you're trying to predict already happened.
Overfitting vs underfitting. Overfitting: the model memorizes noise/specifics of the training data and performs poorly on new data. Underfitting: the model is too simple to capture the real pattern and performs poorly everywhere. Regularization techniques discourage overly complex models to reduce overfitting.
Loss functions and gradient descent. A loss function is a formula measuring how wrong the model's predictions currently are. Gradient descent is the optimization procedure that repeatedly adjusts model parameters in the direction that reduces the loss, using the gradient (the loss function's slope) as the guide. Intuition: imagine standing on a hill in fog and taking small steps downhill using only what you can feel under your feet — that is gradient descent.
Precision, recall, F1, ROC/AUC — and why accuracy can mislead.
| Metric | Formula (conceptual) | Answers |
|---|---|---|
| Precision | True positives / (True positives + False positives) | Of everything flagged positive, how much was correct? |
| Recall | True positives / (True positives + False negatives) | Of everything actually positive, how much did we find? |
| F1 | Harmonic mean of precision and recall | A single balance score between the two |
| ROC/AUC | Area under true-positive-rate vs false-positive-rate curve | How well the model ranks positives above negatives across thresholds |
Worked example. A fraud detector sees 1,000 transactions, 5 of which are fraudulent. A model that predicts "not fraud" every single time achieves 99.5% accuracy — while catching zero fraud. This is why accuracy alone is misleading on imbalanced data; precision/recall/F1 expose what accuracy hides.
Distribution shift. Production behavior can differ from benchmarks when the real-world input distribution differs from what the model was trained/evaluated on — e.g. a support-ticket classifier trained on 2023 tickets facing an influx of a brand-new product category in 2026. This concept directly motivates the eval-and-monitor discipline in Chapter 11: benchmark performance is a snapshot, not a guarantee.
Definition. A tensor is a multi-dimensional array of numbers — a scalar is a 0-D tensor, a vector is 1-D, a matrix is 2-D, and higher-dimensional tensors (3-D+) are used for batches of images, sequences of token embeddings, and so on. Every input, weight, and intermediate value in a neural network is represented as a tensor.
Forward pass, backpropagation, optimizer. The forward pass pushes input data through the network's layers to produce a prediction. Backpropagation is the algorithm that computes, layer by layer working backward, how much each weight contributed to the error — using calculus (the chain rule) to compute gradients efficiently. The optimizer (e.g. Adam, SGD) then uses those gradients to update the weights. Inference is using a trained model to make a prediction with no weight updates (only a forward pass); training repeats forward pass + backprop + optimizer step across many examples.
Batching, epochs, learning rate. A batch is a group of training examples processed together for efficiency. An epoch is one full pass through the entire training dataset. The learning rate controls how large each optimizer step is — too high and training becomes unstable; too low and training is painfully slow.
Definition. Tokenization breaks text into sub-word units ("tokens") the model actually processes — often not the same as words. "Unbelievable" might tokenize as ["Un", "believ", "able"]. Roughly, English text averages ~0.75 words per token.
Why it matters. Every dollar and every latency millisecond in an LLM API call is billed and bounded by token count, not character or word count — this is why Chapter 5 emphasizes "usage accounting" (input/output/cached tokens) as a first-class engineering concern, not an afterthought.
Definition. An embedding is a fixed-length vector of numbers that represents the meaning of a piece of text (or image, audio, etc.), positioned in a high-dimensional space such that semantically similar inputs produce vectors that are geometrically close together. This concept is used constantly from Chapter 6 onward, but originates architecturally inside the transformer itself (the embedding layer in Fig 4.1). A full treatment — including cosine similarity and vector databases — is given in Chapter 6, since those are prerequisite to RAG rather than to core LLM behavior.
Simple analogy. Reading the sentence "The trophy didn't fit in the suitcase because it was too big" — to know what "it" refers to, you unconsciously weigh every other word in the sentence and decide "trophy" matters far more than "suitcase" for resolving "it." Self-attention is the mechanism that lets a transformer do exactly this, mathematically, for every token against every other token.
Technical definition. Self-attention computes, for each token, a weighted combination of all other tokens' representations, where the weights ("attention scores") are learned and depend on the content of the tokens themselves — allowing the model to dynamically decide which parts of the input are relevant to each other, regardless of their distance in the sequence.
Before transformers, sequence models (RNNs/LSTMs) processed tokens one at a time, in order — token 5 could not be processed until token 4 finished, which made training slow and hard to scale on GPUs. Transformers process all tokens in a sequence simultaneously using self-attention plus feed-forward layers stacked into repeating "blocks." This parallelism is what allowed transformer models to scale to the sizes and datasets that produced modern LLMs.
| Stage | What Happens | Goal |
|---|---|---|
| Pretraining | Train on massive, broad text/code corpora to predict the next token | General language competence and world knowledge |
| Instruction tuning | Fine-tune on (instruction, ideal response) pairs | Follow instructions rather than just continue text |
| Preference optimization (e.g. RLHF/DPO) | Train the model to prefer outputs humans rate as better | Helpfulness, safety, tone alignment |
| Fine-tuning (task-specific) | Further training on a narrow labeled dataset for one company/task | Specialization (see Chapter 14) |
Why more context is not always better. Even when a model technically accepts very large contexts, retrieval and reasoning quality across that context is not uniform — models often attend more reliably to information near the beginning and end of a long context than to information buried in the middle (a pattern sometimes called "lost in the middle"). Stuffing irrelevant text into context also increases cost, latency, and the chance of the model being distracted by noise. This is the conceptual foundation for Chapter 7's insistence on retrieving "fewer, better chunks" rather than maximizing context usage.
Definition. Inference is the process of generating output from a trained model. Sampling controls how the next token is chosen from the model's predicted probability distribution — parameters like temperature control how random vs. predictable the choice is. Lower temperature (near 0) makes output more deterministic and repetitive; higher temperature increases variety and creativity, at the cost of consistency. Some models expose a "reasoning effort" setting that trades latency/cost for more internal deliberation before answering.
Definition. A hallucination is a fluent, confident output that is factually incorrect or unsupported by any real source — the model is not "lying," it is generating the statistically plausible continuation of text, which is not the same as generating a verified fact. Calibration refers to how well a model's expressed confidence matches its actual correctness rate; LLMs are frequently poorly calibrated, sounding equally confident whether right or wrong. A knowledge cutoff is the date after which the model has no training data, so it cannot know about events after that point unless given external context (this is a core motivation for RAG, Chapter 7).
Modern models increasingly accept and/or produce multiple modalities — text, vision (images), audio (speech), and combinations thereof — under one architecture family, rather than requiring entirely separate specialized models (detailed further in Chapter 14). Separately, engineers choose between hosted proprietary models (accessed via API, no infrastructure to manage, cannot see or modify weights) and open-weight models (downloadable weights, can self-host, fine-tune freely, but require infrastructure and expertise) — a decision revisited in Chapters 13–14.
| Learn Now | Understand | Can Wait |
|---|---|---|
| Vectors, matrices, dot product, cosine similarity | Enough to understand embeddings and retrieval | Proof-heavy linear algebra |
| Probability, conditional probability, distributions | Enough to reason about uncertainty and metrics | Advanced probability theory |
| Mean, variance, correlation, sampling | Enough for experiments/evals | Graduate statistics |
| Derivative/gradient intuition | Enough to understand optimization | Advanced calculus derivations |