Transformer & DL Foundations
IC3IC4IC5

Sampling and Decoding: From Logits to a Token

Every token a model emits is a deliberate bet placed on a reshaped probability distribution — this lesson teaches you to control that bet.

15 min read · 15 sections
0

1. Quick anchor

A transformer's forward pass ends with one vector of vocab_size real numbers — the logits — for the next position. Everything you call "the model's personality," "creativity," "determinism," or "hallucination at high temperature" happens after that vector, in the decoding step that turns logits into a single chosen token. Decoding is not the model; it is a tiny, cheap, fully-controllable post-processor you bolt on top. The whole field is a sequence of two operations applied to logits — reshape the distribution (temperature, top-k, top-p, min-p, penalties) and then pick from it (argmax, sample, or search) — and almost every production quality complaint traces back to a bad choice in those two operations rather than to the weights. Master this and you can debug "the model is repetitive / unhinged / off-format" without touching a single parameter.

2. Why interviewers probe this

  • IC3 — "do you know the mechanism?" Can you go from a logit vector to a token? Do you know temperature divides logits before softmax, that greedy is argmax, and that top-p is adaptive while top-k is fixed? This filters people who only ever called openai.chat.completions.create(temperature=0.7) without knowing what 0.7 means.
  • IC4 — "can you debug and choose?" Given a symptom (repetition loops, JSON that won't parse, bland outputs, garbage at temp 1.5), can you name the cause and the lever? Do you understand the determinism/diversity tradeoff as an explicit dial rather than a vibe? Can you reason about why temperature=0 is still not bit-for-bit reproducible across hardware?
  • IC5 — "can you reason about the search objective itself?" Why is the most-likely sequence (beam search) often a worse sequence (the likelihood trap)? When does constrained/structured decoding belong in the sampler vs. a retry loop? How does speculative decoding preserve the exact target distribution while being faster? This is where staff candidates separate: they treat decoding as a search-over-sequences problem with a cost model, not a knob.

3. Concept build-up

Beginner explainerNew here? The words first

The words first.

  • Logit — one raw, unnormalized score the model emits per vocabulary token; can be any real number, positive or negative.
  • Softmax — the function that turns a vector of logits into probabilities that are all positive and sum to 1.
  • Greedy decoding — always pick the single highest-probability token; deterministic.
  • Sampling — roll a weighted die using the probabilities; introduces randomness.
  • Temperature — a knob that flattens (high) or sharpens (low) the distribution before you pick.
  • Top-k / top-p — keep only the best k tokens, or the smallest set whose probabilities sum to p, then renormalize and sample from those.
  • Repetition penalty — push down the probability of tokens you've already said so the model stops looping.
  • Beam search — keep several candidate sentences alive at once and pick the highest-scoring whole sentence at the end.

Step by step.

  1. Run the forward pass; get a logit vector of length vocab_size for the next position.
  2. Optionally divide every logit by the temperature.
  3. Optionally zero out (mask to negative infinity) tokens outside your top-k or top-p set, plus apply any penalties.
  4. Apply softmax to get a probability distribution over the surviving tokens.
  5. Either take the argmax (greedy) or draw one token from the distribution (sampling).
  6. Append that token, feed it back in, and repeat until you hit a stop token or length limit.

Remember this: decoding never changes the weights — it only reshapes one probability vector at a time and then makes a pick.

3.1 The softmax bridge: from logits to probabilities

The forward pass gives you logits z = [z_1, ..., z_V]. Softmax maps them to probabilities: p_i = exp(z_i) / sum_j exp(z_j). Two properties matter for decoding. First, softmax is shift-invariant — adding a constant to every logit changes nothing — which is why implementations subtract max(z) before exponentiating (numerical stability, no semantic effect). Second, softmax is scale-sensitive: multiply all logits by a constant and the distribution sharpens or flattens. That scaling is exactly what temperature exploits.

3.2 Temperature: the master sharpness dial

Temperature tau rescales logits before softmax: p_i ∝ exp(z_i / tau). Define tau as the divisor applied to every logit. Lower tau makes the gaps between logits larger in exponent space, so the top token dominates (sharper, more deterministic). Higher tau shrinks the gaps, flattening toward uniform (more random). tau = 1 is the identity. The limit tau -> 0 collapses to greedy (argmax), and in practice APIs alias temperature=0 to greedy because dividing by literal zero is undefined — they short-circuit to argmax.

Temperature — on real numbers

We have three candidate tokens with logits z = [2.0, 1.0, 0.0] (say tokens "cat", "dog", "fish").

  • tau = 1.0 (no change): exponentiate [e^2, e^1, e^0] = [7.39, 2.72, 1.00], sum = 11.11. Probabilities = [0.665, 0.245, 0.090]. "cat" 66%, "dog" 25%, "fish" 9%.
  • tau = 0.5 (sharper): divide logits first -> [4.0, 2.0, 0.0], exponentiate [54.6, 7.39, 1.0], sum = 62.99. Probabilities = [0.867, 0.117, 0.016]. "cat" jumps to 87%; "fish" nearly vanishes.
  • tau = 2.0 (flatter): divide -> [1.0, 0.5, 0.0], exponentiate [2.72, 1.65, 1.0], sum = 5.37. Probabilities = [0.506, 0.307, 0.186]. Now "fish" has a real 19% shot.

What it did: the same logits produced a near-deterministic pick at tau=0.5 and a genuinely diverse three-way race at tau=2.0. The model never changed — only the temperature divisor did.

3.3 Truncation: top-k, top-p (nucleus), and min-p

Temperature alone is dangerous at the high end: flattening the distribution gives every token — including the absurd long tail — non-trivial mass, so you eventually sample nonsense. Truncation methods solve this by discarding the tail before sampling.

Top-k keeps the k highest-probability tokens, masks the rest to probability zero, renormalizes, and samples. It is simple but rigidly fixed: when the model is supremely confident (one token at 0.99), k=40 still drags in 39 near-zero tokens; when the model is genuinely uncertain (a flat distribution over 200 plausible continuations), k=40 arbitrarily amputates the rest.

Top-p / nucleus (Holtzman et al., ICLR 2020) fixes the rigidity by being adaptive. Sort tokens by descending probability, accumulate until the cumulative sum first reaches p (e.g. 0.9), keep exactly that set (the "nucleus"), renormalize, sample. When the model is confident the nucleus might be 1-2 tokens; when uncertain it might be hundreds. This is the production default for open-ended generation and the single most important method in this lesson.

Min-p (Hewitt et al., 2024, "Turning Up the Heat") attacks a specific pathology: at high temperature, top-p can still admit very low-probability tokens because flattening inflates the tail's cumulative share. Min-p sets a relative floor: keep token i only if p_i >= min_p_ratio * p_max, where p_max is the top token's probability and min_p_ratio is typically 0.05–0.1. Because the threshold scales with the model's own confidence, min-p stays coherent even at tau = 2+, letting you crank temperature for creativity without the garbage. Define min_p_ratio as the fraction of the peak probability a token must clear to survive.

3.4 Repetition control

Autoregressive models fall into loops ("the the the", or restating a sentence verbatim) because a token, once emitted, raises its own conditional probability on the next step. Repetition penalty divides the logit of any already-seen token by a coefficient gamma > 1 (often gamma^count for frequency). Define gamma as the penalty divisor. Set it too high (>1.3) and the model avoids necessary repeated words ("the", "is") and degrades fluency. Newer methods — DRY (Don't Repeat Yourself) and XTC (Exclude Top Choices) — target phrase-level repetition and "boring" top choices respectively, which a per-token penalty misses. For structured output (JSON, code) repetition penalties are dangerous: braces, commas, and indentation must repeat.

3.5 Search: beam search and the likelihood trap

Everything above is local — it picks one token at a time. Beam search searches over sequences: keep the b highest-scoring partial sequences (by cumulative log-probability), expand each by all tokens, re-rank, keep the top b. Define b as the beam width. It beats greedy's myopia for tasks with a single correct answer (translation, ASR, constrained extraction) where the highest-likelihood sequence really is the best one.

But for open-ended generation, beam search is worse than sampling — the likelihood trap. The most probable sequence under a well-trained LM is often degenerate: empty, repetitive, or generic ("I don't know. I don't know. I don't know."), because real human text is not the mode of the distribution — it lives in the high-entropy bulk. Beam search also suffers beam collapse, where all b beams share a prefix and explore nothing; Diverse Beam Search adds an inter-beam penalty to counter this. The staff-level takeaway: choosing a decoder is choosing a search objective — maximize likelihood (beam) vs. sample from a calibrated distribution (nucleus) — and the right objective depends entirely on whether your task has one answer or many.

3.6 Constrained / structured decoding

When output must obey a grammar — valid JSON, a regex, a SQL dialect, a function-call schema — you don't sample and pray. You mask the logits at each step to only tokens that keep the output valid under a finite-state machine or context-free grammar compiled from the schema. Libraries like Outlines, guidance, and llama.cpp's GBNF do this; OpenAI's Structured Outputs and Anthropic's tool-use schemas enforce it server-side. The win is a guarantee: the output parses, every time, with zero retries. The cost is a per-step mask computation and the subtle risk that constraining the distribution can lower quality — if you force a token the model finds unlikely, you've pushed it off-manifold, sometimes producing valid-but-wrong content. Prefer schema-guided generation for format guarantees, but keep a semantic eval (see /evals) because "parses" is not "correct."

3.7 Speculative decoding (teaser)

Sampling is sequential and memory-bound: each token needs a full forward pass. Speculative decoding (Leviathan et al., 2022) uses a small cheap draft model to propose several tokens, then verifies them in a single batched forward pass of the big target model, accepting the longest correct prefix via a rejection-sampling step that provably preserves the exact target distribution — same outputs in expectation, fewer expensive passes. It is a pure latency optimization that lives in the decoding loop, which is why it belongs here conceptually but is taught in depth in /inference/speculative-decoding.

4. Minimal implementation

Here is a real, dependency-light decoding loop showing the full pipeline — temperature, top-k, top-p, min-p, and repetition penalty — applied to logits, then sampling. This is the same logic vLLM and TGI implement (vectorized over a batch); the structure is identical.

import torch
import torch.nn.functional as F
 
def decode_step(logits: torch.Tensor,
                generated: list[int],
                temperature: float = 1.0,
                top_k: int | None = None,
                top_p: float | None = None,
                min_p: float | None = None,
                rep_penalty: float = 1.0) -> int:
    """logits: shape (vocab_size,), raw model output for the next position.
       Returns a single sampled token id. Order of ops matters."""
    logits = logits.clone().float()
 
    # 1) Repetition penalty: push down logits of already-emitted tokens.
    if rep_penalty != 1.0 and generated:
        seen = torch.tensor(sorted(set(generated)))
        # Divide positive logits, multiply negative ones (HF convention).
        sel = logits[seen]
        logits[seen] = torch.where(sel > 0, sel / rep_penalty, sel * rep_penalty)
 
    # 2) Temperature: scale before softmax. temperature==0 means greedy.
    if temperature == 0.0:
        return int(logits.argmax())
    logits = logits / temperature
 
    # 3) Top-k: keep only the k largest logits.
    if top_k is not None:
        kth = torch.topk(logits, top_k).values[-1]
        logits[logits < kth] = float("-inf")
 
    probs = F.softmax(logits, dim=-1)
 
    # 4) Top-p (nucleus): keep smallest set whose cumprob >= p.
    if top_p is not None:
        sorted_p, sorted_idx = torch.sort(probs, descending=True)
        cum = torch.cumsum(sorted_p, dim=-1)
        # Mask everything strictly after the token that crosses p.
        keep = cum - sorted_p < top_p           # token that crosses p stays
        remove_idx = sorted_idx[~keep]
        probs[remove_idx] = 0.0
 
    # 5) Min-p: relative floor against the peak probability.
    if min_p is not None:
        thresh = min_p * probs.max()
        probs[probs < thresh] = 0.0
 
    probs = probs / probs.sum()                 # renormalize survivors
    return int(torch.multinomial(probs, num_samples=1))

A few things to internalize. The order is load-bearing: penalty and temperature reshape, truncation prunes, then you renormalize over survivors. Top-p's mask keeps the token that crosses the threshold (so a single 0.95 token still survives top_p=0.9). temperature=0 short-circuits to argmax rather than dividing by zero. And multinomial is the actual weighted die-roll — replace it with argmax and the whole stochastic apparatus collapses to greedy.

5. Production tradeoffs

Method Latency cost Determinism Quality / failure mode What changes at scale
Greedy (temp=0) Lowest High (not bit-exact) Bland, repetitive; deterministic for tests Cheapest; batch-friendly
Temperature Negligible Low (>0) Too high -> tail garbage; too low -> bland Per-request knob, no infra cost
Top-k One topk op Low Rigid set; bad when confidence varies Cheap; superseded by top-p
Top-p (nucleus) One sort Low The default; mild over-truncation if p low Sort cost trivial vs. forward pass
Min-p One max + compare Low Keeps coherence at high temp Cheap; pairs with high temp
Repetition penalty Gather/scatter Low Too high breaks the/is/JSON braces Needs history buffer per sequence
Beam search (b) ~bx memory + compute High Likelihood trap, beam collapse Expensive; rare in LLM serving
Constrained decoding Per-step mask (FSM/CFG) High Guarantees format, can lower semantic quality Grammar compile + per-step mask
Speculative decoding Saves passes Preserves target dist. Net latency win; draft-model overhead Big win at scale; see /inference

Prose. The dominant cost in serving is the forward pass, so every sampling method here is essentially free except beam search (which multiplies memory and compute by the beam width) and constrained decoding (which adds a per-step grammar mask). That is why production LLM serving almost never uses beam search: it costs bx for outputs that are often worse on open-ended tasks. The real scale concern is determinism: temperature=0 is algorithmically greedy but still not bit-for-bit reproducible across GPU types, batch sizes, or kernel versions, because floating-point reduction order changes which of two near-tied logits wins the argmax. If you need exact reproducibility for evals or compliance, pin the model version, seed, batch size, and hardware — and even then, treat it as best-effort. The dominant failure modes in the wild: (1) high temperature without min-p/top-p -> incoherent tail tokens; (2) repetition penalty applied to structured output -> broken JSON; (3) temp=0 on a creative task -> users complain it's "robotic"; (4) constrained decoding masking the model off-manifold -> valid JSON with hallucinated field values that pass parsing but fail eval.

6. How it's asked

[IC3] Explain what temperature does to a softmax distribution and why temperature 0 is a special case. Temperature tau divides every logit before softmax: p_i ∝ exp(z_i / tau). Below 1 it widens the relative gaps in exponent space, sharpening toward the top token; above 1 it shrinks them, flattening toward uniform. As tau approaches 0 the top logit's exponential dominates infinitely, so the distribution collapses onto argmax — pure greedy. Because dividing by literal zero is undefined, APIs short-circuit temperature=0 directly to argmax, which is why it's the deterministic special case.
[IC4] A user reports their high-temperature sampling produces garbage. Walk me through why, and how top-p or min-p fixes it. Raising temperature flattens the distribution, which inflates the cumulative mass of the long tail of implausible tokens; with no truncation you eventually sample one of those and the generation derails. Top-p caps this by keeping only the nucleus — the smallest set summing to p — and renormalizing, so the tail is gone before you sample. Min-p is often better at high temperature because it sets a relative floor (p_i >= min_p_ratio * p_max): the threshold scales with the model's confidence, so even at tau=2 you stay coherent while preserving real diversity. The general fix is "never raise temperature without a truncation method underneath it."
[IC4] Why might repetition penalty hurt a code-generation endpoint? Repetition penalty divides the logits of already-seen tokens, but code and structured formats require repeated tokens — braces, commas, indentation, common keywords. Penalizing them pushes the model away from syntactically necessary characters, producing malformed or oddly-spelled output. For structured generation you want constrained/grammar decoding for the format and, at most, phrase-level methods like DRY rather than a blunt per-token penalty.
[IC5] Why does beam search underperform plain sampling for open-ended generation but still win for translation? Connect this to the likelihood trap and constrained decoding. Beam search approximates the maximum-likelihood sequence. For translation or ASR there is essentially one correct output, so the mode of the distribution is the right answer and searching for it helps. For open-ended generation the mode is degenerate — the single most probable continuation tends to be empty, generic, or repetitive, because natural human text lives in the high-entropy bulk, not the peak; this is the likelihood trap, and sampling from a calibrated nucleus matches human text far better. The unifying lens is that decoding is choosing a search objective: maximize likelihood (beam, constrained decoding for single-answer/format-guaranteed tasks) versus sample from a calibrated distribution (nucleus/min-p for many-answer tasks) — and constrained decoding is just beam-style likelihood maximization restricted to a grammar, with the same caveat that forcing the mode can push the model off-manifold.
[IC5] How does speculative decoding speed up generation without changing the output distribution? A small draft model proposes several tokens; the large target model verifies all of them in one batched forward pass. A rejection-sampling acceptance criterion accepts the draft token with probability min(1, p_target / p_draft) and, on rejection, resamples from an adjusted residual distribution — which provably yields samples from the exact target distribution. So you amortize multiple cheap draft steps against one expensive verification pass, cutting wall-clock latency while remaining distributionally identical to sampling from the target alone. It's a decoding-loop optimization, covered in depth in /inference.

7. Pitfalls & flashcards

  • Stacking temperature and top-p blindly. They interact: high temperature + high top_p reopens the tail you thought you closed. If you crank temperature, pair it with min-p, not just top-p.
  • Assuming temp=0 is reproducible. It's deterministic in algorithm, not in bits — floating-point reduction order across hardware/batch can flip near-tied argmaxes. Pin everything if you need exact evals.
  • Repetition penalty on structured output. Breaks JSON/code by penalizing mandatory repeated tokens. Use grammar-constrained decoding instead.
  • Top-k as a default. It's the legacy knob; top-p adapts to confidence and is almost always the better default.
  • Trusting "it parses" as "it's correct." Constrained decoding guarantees format, never semantics. Always keep a content eval (/evals).
  • Beam search for chat. Wrong objective — the likelihood trap makes it blander, not better, and it costs bx.

Flashcard. Decoding = reshape then pick: temperature/penalties/truncation reshape one logit vector, then argmax (greedy), a weighted draw (sampling), or a sequence search (beam) picks — and for open-ended text, sampling from a calibrated nucleus beats maximizing likelihood.

8. Further reading

Next: /inference/speculative-decoding — make the decoding loop fast without changing what it emits.

Primary sources
← More in Transformer & DL Foundations