Models used the middle of a long context worse than the ends

Two tasks built to test where information sits, not just how much

In July 2023, researchers examined how well language models use information at different positions inside a long input, rather than only how much input they can accept. The Lost in the Middle paper tested models on multi-document question answering, where one supporting document contains the answer among many distractor documents, and on a synthetic key-value retrieval task, varying the position of the relevant document or key-value pair within the input.

What the paper measured

The paper reports that performance is often highest when the relevant information sits at the very start or the very end of the context, and degrades, sometimes sharply, when the same information sits in the middle, producing a U-shaped accuracy curve across position. It states this pattern held even for models built and marketed for extended context windows, meaning a longer supported context length did not by itself guarantee even access across that length. The degradation was measured on the specific multi-document and key-value tasks the paper constructed, under the specific models and context lengths it tested.

What the finding does not establish

A later benchmark, RULER, submitted in April 2024, tested a broader set of long-context tasks, including multi-hop tracing and aggregation rather than single-needle retrieval, and reported that of seventeen models claiming context sizes of 32,000 tokens or more, only about half maintained what the paper judged satisfactory performance at that length, with large performance drops as length increased further. RULER's tasks differ from the original multi-document and key-value setup, so its numbers are a separate measurement, not a replication, though both point toward the same broader caution: a stated maximum context length describes what a model accepts as input, not what it reliably uses. Neither paper establishes that the middle-position effect is permanent or architecture-independent; both describe a set of models at a point in time.

• Does the task place the critical fact at a context position the model was actually tested at?
• Has the specific model version been evaluated on a positional or effective-length benchmark, or only on a maximum-length claim?
• Would reordering the input, or retrieving a smaller relevant set instead of the whole context, reduce the risk observed here?

The evidence supports a concrete design caution: put critical information near the edges of a long input, or verify positional performance directly, rather than trusting a stated context length as a guarantee of even access across it.