Context Engineering & Prompting
IC4IC5IC6

Long Context and Lost in the Middle

A 1M-token window is an attention budget, not a hard drive — and the model reads the middle of it like a tired commuter skims the middle of a long email.

15 min read · 14 sections
0

1. Quick anchor

A 1M-token context window is an attention budget, not a hard drive. Every token the model emits must, in principle, weigh its relationship against every token it can see — an n² problem — so as you pour in more tokens, the share of attention any single fact can claim shrinks. The empirical fingerprint of this is "lost in the middle" (Liu et al., 2023): accuracy on retrieval-style tasks traces a U-shaped curve — high when the needed fact sits at the very start or very end of the context, sagging 20–30% when it sits in the middle. This is not a bug in positional embeddings you can patch; it is a budget and recency effect that survives even on million-token windows. The senior move is therefore to treat context as something you engineer — retrieve the right facts, place them at the edges, rerank and compress — rather than something you fill. The model with the biggest window does not win; the team with the best context discipline does.

2. Why interviewers probe this

  • IC4 — Do you know the window is not free RAM? Can you explain that "it fits in 1M tokens" and "the model will reliably use it" are different claims, and name the U-curve? Do you reach for retrieval instead of stuffing by reflex?
  • IC5 — Can you derive the mechanism and act on it? Can you connect quadratic attention and recency bias to the U-curve, argue why placement and reranking work, and reason quantitatively about the 20–30% middle penalty? Can you say when long-context beats RAG and vice versa?
  • IC6 — Can you design the whole context policy under constraints? For a long-horizon agent at 800K tokens you must trade off retrieval recall, ordering, compaction, prompt caching, and latency — and know which knob to turn when accuracy drops at depth. Interviewers want to see you treat the window as a managed resource with an explicit eviction and placement strategy, backed by an eval (MRCR/GraphWalks-style), not vibes.

3. Concept build-up

Beginner explainerNew here? The words first

The words first.

  • Context window — the total span of tokens (prompt + history + retrieved docs + output) the model can attend to in one request; e.g. 1,000,000 for Opus 4.8.
  • Token — a sub-word chunk; roughly 0.75 English words. 1M tokens ≈ 750K words ≈ a long bookshelf.
  • Attention — the mechanism by which each output token weighs how much every input token matters; cost grows with the square of sequence length.
  • Lost in the middle — the empirical finding that models retrieve facts at the start/end of context far more reliably than facts buried in the middle.
  • U-shaped curve — plot accuracy (y) against the position of the needed fact (x): high at both ends, low in the middle, like a smile.
  • RAG (retrieval-augmented generation) — fetch only the most relevant chunks from a store and put those in the prompt, instead of the whole corpus.
  • Reranking — a second, more precise pass that reorders candidate documents by true relevance before they enter the prompt.
  • Context rot — the gradual, non-linear accuracy decay as the window fills up.

Step by step.

  1. You have a big pile of text and a question.
  2. Naive approach: dump the whole pile into the 1M window and ask.
  3. The model attends across everything, but its attention budget is finite and biased toward the edges.
  4. A fact sitting in the middle of the pile gets under-weighted — you lose ~20–30% accuracy versus the same fact at the edge.
  5. Fix it by retrieving only what matters (RAG), reranking so the best doc is most relevant, and placing the critical facts at the start or end.
  6. Compress the rest so the signal-to-noise ratio stays high.

Remember this: a bigger window lets you fit more, not attend to more — placement and retrieval beat stuffing.

3.1 The window is an attention budget, not storage

The transformer's core operation is scaled dot-product attention: for a sequence of n tokens, each output position computes a weighted sum over all n input positions, giving an n × n matrix of pairwise interactions. Two consequences follow directly. First, compute and memory scale as n² (FlashAttention and friends reduce the constant and the memory footprint, but the fundamental pairwise structure remains). Second, and more important for this lesson, the softmax that produces attention weights is a normalized distribution — the weights over all positions sum to 1. So attention is a zero-sum resource: every token competes for a fixed total of attention mass. Add 900K tokens of distractors around your one important sentence and, all else equal, that sentence's share of the budget collapses. This is the first-principles reason "more context" is not "more knowledge."

◐ InteractiveLost in the middle

The answer the model needs sits at position 10 of 20 — retrieval accuracy there is ≈56%. Drag it to the edges (recency + primacy) and accuracy climbs; bury it in the middle and the model often misses it. The dip deepens with longer context — which is why you rerank and put the best evidence first or last, not why you stuff everything in.

3.2 The empirical fingerprint: the U-curve (Liu et al., 2023)

Liu et al. ran a clean experiment: multi-document question answering with 20 documents, where exactly one document contains the answer and the rest are relevant-looking distractors. They swept the position of the gold document from 1 to 20 and measured accuracy across GPT-3.5, GPT-4, Claude, and open models. The result was strikingly consistent — a U-shaped curve:

  • Accuracy is highest when the answer is at position 1 (primacy) or position 20 (recency).
  • Accuracy drops ~20–30% when the answer sits at positions 10–15 (the middle).
  • The same shape appears in a synthetic key-value retrieval task, ruling out "the QA was just hard."
  • Critically, it persists on 100K+ token windows and is not fixed by simply having a longer context.
The U-curve — on real numbers

Name the symbols in plain words:

  • pos — where the gold document sits among 20 docs (1 = first, 20 = last).
  • acc(pos) — task accuracy when the gold doc is at that position.

Concrete (illustrative numbers in the spirit of Liu et al.'s shape):

  • acc(1) = 0.75 — gold at the front, model nails it (primacy).
  • acc(20) = 0.73 — gold at the back, almost as good (recency).
  • acc(12) = 0.50 — gold buried in the middle.

Compute the middle penalty: 0.75 - 0.50 = 0.25, i.e. a 25% absolute drop just from moving the same fact from the edge to the middle. Nothing about the fact changed — only its position. That is the entire phenomenon in one subtraction: position is a first-class lever on accuracy, so where you put a fact is part of your prompt's correctness, not cosmetics.

3.3 Why it's a budget/recency problem, not a positional-embedding bug

A tempting hypothesis: "the model's positional encoding (RoPE, ALiBi, etc.) just degrades in the middle." Liu et al. and the follow-up TACL analysis argue against this. The U-shape is asymmetric in a way pure position can't explain — recency (end) is favored partly by the causal attention mask and the training objective (next-token prediction makes recent tokens highly predictive), while primacy (start) is favored because the system prompt and earliest tokens are attended to by every downstream position and often anchored by training. The middle has neither advantage. So the mechanism is (a) a finite attention budget the middle must compete hardest for, plus (b) a recency bias baked into causal LM training, not a coordinate bug. This matters for interviews because it tells you the fix is placement and pruning, not "wait for better positional embeddings."

3.4 Context rot: degradation is a gradient, not a cliff

In June 2026, Opus 4.8, Sonnet 4.6, and Haiku 4.5 ship 1M-token windows (Sonnet 4.5 and earlier: 200K; max single-request output: 128K). But Anthropic's own framing is blunt: usable performance depends on context engineering — placement, compression, retrieval — not window size alone. As tokens accumulate, accuracy degrades non-linearly and gradually ("context rot") rather than failing at a hard limit. The practical reading: you should not plan to operate at 95% of the window and expect 95% of the quality. The window is a ceiling on what's possible, with a quality gradient underneath it.

3.5 Countermeasures, ranked by leverage

  1. Retrieve before stuffing. Use RAG to pull the top-k chunks instead of pre-loading the corpus. This raises signal-to-noise more than any placement trick because it removes distractors entirely. (See /rag.)
  2. Place critical info at the edges. Put the most-relevant retrieved documents (and the question itself) at the start and end of the assembled context; bury weak candidates in the middle where the penalty is least costly.
  3. Rerank. A listwise or cross-encoder reranker (jina-reranker-v3 can process 64+ docs jointly in a 131K context) reorders candidates so your strongest evidence lands in an edge slot. (See /rag/reranking.)
  4. Compress with preservation rules. Summarize or extract to cut token count while preserving code, decisions, and state — accepting that summarization is lossy for fact-dense content.
  5. Cache the stable prefix. Long, fixed system/tool blocks belong behind a prompt-cache breakpoint so the cost of a big context is paid once. (See /context-engineering/prompt-caching.)

3.6 Long context vs RAG — the honest framing

These are not rivals; they are layers. Long context is a capability (the model can hold 1M tokens); RAG is a discipline (you choose to hold only the right ~10K). Use raw long-context when the task genuinely needs global reasoning over a coherent body (one 300-page contract, a full codebase diff) where chunking would sever cross-references. Use RAG when the corpus is large, mostly irrelevant per query, and changes often. Most production systems are hybrid: retrieve to narrow the field, then let the long window hold the retrieved set plus working state — and measure on a long-context benchmark like MRCR v2 (which replaced the now-saturated Needle-in-a-Haystack) rather than trusting that "it fit."

4. Minimal implementation

A runnable demo that reproduces the U-curve locally: we build a context of n filler documents, slot a "needle" fact at a chosen position, ask the model to retrieve it, and sweep position to plot accuracy. This is the spine of the lost-in-middle demo and a real eval harness you could run in CI.

import os
from anthropic import Anthropic
 
client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
MODEL = "claude-opus-4-8"  # 1M-token window, June 2026
 
# A distinctive needle the model can't guess from priors.
NEEDLE = "The access code for vault Theta-9 is QUARTZ-4417."
QUESTION = "What is the access code for vault Theta-9? Answer with only the code."
 
# Cheap, semantically-neutral filler so we measure position, not content difficulty.
def filler(i: int) -> str:
    return f"Log entry {i}: routine telemetry nominal; no anomalies recorded; sensor {i} ok."
 
def build_context(n_docs: int, needle_pos: int) -> str:
    docs = [filler(i) for i in range(n_docs)]
    docs[needle_pos] = NEEDLE  # overwrite one slot with the needle
    return "\n".join(docs)
 
def ask(context: str) -> str:
    resp = client.messages.create(
        model=MODEL,
        max_tokens=32,
        # Stable instruction first -> cache-friendly + primacy slot for the task.
        system="You answer retrieval questions from the provided log. Be exact.",
        messages=[{
            "role": "user",
            "content": (
                f"{QUESTION}\n\n<logs>\n{context}\n</logs>\n\n{QUESTION}"
            ),  # question repeated at start AND end = both edge slots
        }],
    )
    return resp.content[0].text.strip()
 
def sweep(n_docs: int = 400, positions=(0, 0.25, 0.5, 0.75, 0.99), trials: int = 5):
    for frac in positions:
        pos = min(int(frac * n_docs), n_docs - 1)
        hits = 0
        for _ in range(trials):
            ctx = build_context(n_docs, pos)
            if "QUARTZ-4417" in ask(ctx):
                hits += 1
        print(f"needle at {frac:>4.0%} of context -> accuracy {hits/trials:.2f}")
 
if __name__ == "__main__":
    sweep()

What to notice, and why each line is load-bearing:

  • Filler is content-neutral. Identical, boring log lines isolate position as the only variable — if accuracy drops at 50%, it's the middle penalty, not a harder question.
  • The question is repeated at the start and end of the user turn. This deliberately exploits both edge slots; in a real product you'd place the retrieved evidence at the edges, not just the question.
  • The system prompt is stable and goes first — it doubles as a prompt-cache prefix and occupies the primacy position.
  • You sweep needle_pos and expect to see the U-curve: high at 0% and 99%, lowest near 50%. With trials > 1 you get a noisy but real accuracy estimate per bin — the same design MRCR v2 formalizes with token-based bins up to 1M.

Run it once at n_docs=400 (cheap) and once near the window limit; the gap between the two is your context-rot budget.

5. Production tradeoffs

Strategy Cost Latency Quality effect Primary failure mode
Stuff everything in 1M window High (you pay for every input token) High (prefill scales with tokens) Strong global reasoning; U-curve hurts mid-context facts Middle facts silently dropped; context rot at depth
RAG (retrieve top-k) Low (small prompt) Low High if retrieval recall is good A missed retrieval = the fact is simply absent
Rerank then place at edges + reranker call + tens of ms Recovers much of the U-curve penalty Reranker is itself a model with its own errors
Compression / summarization + a summarize call + one model pass Fits more turns; lossy Drops a fact the user later needs; precision < full context
Prompt-cache the stable prefix 0.1× on hits, 1.25–2× on writes Big prefill savings on hits Neutral on quality Cache miss if any earlier block changes (prefix invariant)

Prose on what changes at scale:

  • Cost is dominated by input prefill, not output. A 100K-token cached prompt can save ~92% on repeat calls (cache hits at 0.1× base; writes at 1.25× for 5-min TTL). So the economics of long context are really the economics of caching the stable part and retrieving the volatile part small. (See /context-engineering/prompt-caching.)
  • Latency is prefill-bound. Time-to-first-token grows with input length because the model must process the whole prefix before decoding. Reranking adds latency but removes tokens, so it often nets out faster end-to-end than stuffing.
  • The dominant failure mode flips with strategy. Stuffing fails by silently ignoring the middle (hard to detect — the answer looks plausible). RAG fails by omission (easier to detect — you can log retrieval recall). Senior teams prefer failure modes they can measure, which is an argument for RAG + reranking over blind stuffing.
  • At agent scale (long horizons), the window is a managed resource. You combine server-side compaction (auto-summarize near a threshold, default ~150K input tokens) with context editing (clear old tool results, keep the N=3 most recent; optionally preserve thinking blocks for reasoning continuity and cache hits). The hard rule: compaction is lossy and costs two model evaluations, so trigger it as late as your accuracy budget allows. (See /context-engineering and /harness.)
  • Benchmark or you're flying blind. MRCR v2 reported, e.g., Claude Opus 4.6 at 76% on the 1M / 2-needle bin — far from saturated. The lesson: do not assume million-token recall; gate it behind an eval that mirrors your placement and depth. (See /evals.)

6. How it's asked

[IC4] You have a 1M window and 200 docs that fit. Why might retrieving the top 10 still beat stuffing all 200? Fitting is not attending. Attention is a normalized, zero-sum budget, so 190 distractor docs dilute the attention mass available to the 10 that matter — and any fact that lands in the middle suffers the 20–30% lost-in-the-middle penalty. Retrieving the top 10 raises signal-to-noise, lets you place the best evidence at the edges, and is cheaper and lower-latency on prefill. Stuffing only wins when the task needs genuine global reasoning across all 200, which most lookup tasks don't.
[IC5] Derive why 'lost in the middle' is an attention-budget problem, not a positional-embedding bug — and what falls out of it. Softmax attention produces weights summing to 1 over all positions, so it's zero-sum: more tokens means less mass per token. The U-shape is asymmetric — recency is favored by the causal mask and next-token training objective, primacy by every downstream position attending to the earliest tokens — which a pure coordinate (RoPE/ALiBi) bug wouldn't produce. So the cause is finite budget plus recency bias, and the fix is placement and pruning: put critical facts at the start/end, rerank so strong evidence lands in an edge slot, and remove distractors via retrieval. Waiting for "better positional embeddings" would be solving the wrong problem.
[IC5] When do you reach for long-context over RAG, and how do you decide quantitatively? Long-context wins when chunking would sever cross-references the task depends on — a single 300-page contract, a full repo diff — where global coherence matters more than corpus size. RAG wins when the corpus is large, per-query mostly irrelevant, and changing often. I decide by measuring: build an MRCR-style eval at the depths and placements I'll actually run, and compare end-to-end accuracy, cost (input-prefill dollars, with caching), and p95 latency. If raw long-context recall at my target depth is, say, 76% but reranked RAG hits 92% cheaper, RAG wins — and I keep the long window for holding the retrieved set plus working state.
[IC6] Design retrieval + ordering + compression for a 50-turn agent that must stay accurate at 800K tokens on a latency budget. I'd layer it. (1) Retrieve, don't preload: keep lightweight identifiers and load chunks just-in-time via tools (glob/grep-style), so the live window holds only active evidence. (2) Rerank listwise and map highest-relevance to edge positions, burying weak candidates mid-context where the penalty is cheapest. (3) Manage the window as a resource: prompt-cache the stable system/tool prefix (0.1× on hits), use context editing to clear old tool results (keep last 3, preserve thinking for continuity and cache hits), and trigger compaction only near a ~150K-token threshold since it's lossy and costs two evaluations. (4) Isolate work in sub-agents that return 1–2K-token condensed summaries instead of dumping raw transcripts upstream. (5) Gate the whole thing on an eval at 800K depth — if accuracy sags, the first knob is retrieval recall and edge-placement, not a bigger window. The boundary: long-context holds the working set; RAG decides what's in it.
[IC6] Your agent's accuracy is fine at 100K tokens but collapses at 600K. Walk the debugging. First, confirm it's context rot and not a code bug: re-run the failing query with the same evidence at 100K — if it passes, depth is the cause. Then attack the gradient in order of leverage: check whether critical facts are landing mid-context (instrument placement and move them to edges), measure retrieval recall (a 600K context full of distractors is a retrieval failure, not a model failure), add or strengthen reranking, and introduce compaction/context-editing to shrink the live set. Throughout, I'm watching the measurable failure (missed retrievals, mid-context misses on a probe) rather than trusting plausible-looking but wrong answers, which is exactly how the middle penalty hides.

7. Pitfalls & flashcards

  • "It fits, so it'll work." Fitting in 1M tokens says nothing about whether the model will use a mid-context fact. Always validate at your real depth.
  • Placing the answer in the middle by accident. Concatenating retrieved docs in score order often puts rank-#1 first (good) but rank-#5 in the dead middle. Reorder so strong evidence occupies both edges.
  • Trusting Needle-in-a-Haystack. It's saturated (98%+). Use MRCR v2 / GraphWalks-style multi-needle, co-reference evals — they expose the real degradation.
  • Over-compressing. Summarization is lossy for fact-dense content; a dropped number or decision resurfaces three turns later as a wrong answer. Preserve code, decisions, and state explicitly.
  • Breaking the cache while fighting rot. Editing or compacting earlier blocks invalidates the prompt-cache prefix (the prefix invariant). Sequence edits so the stable prefix stays byte-identical; clear thinking before tool results when both are edited.
  • Assuming the U-curve is symmetric. Recency usually beats primacy slightly; if you can place only one copy of the key fact, the end is often the safer slot.

Flashcard. Lost in the middle = U-shaped accuracy vs. fact position (high at start/end, ~20–30% lower in the middle); it's an attention-budget + recency effect, so the cure is retrieve → rerank → place at the edges, not a bigger window.

8. Further reading

Next: /context-engineering/prompt-caching — how to make a big stable context nearly free, so retrieval and placement are the only costs left to optimize.

Primary sources
← More in Context Engineering & Prompting