Bigger models scored worse on a 2021 truthfulness test

A benchmark built from questions people get wrong

Stephanie Lin, Jacob Hilton and Owain Evans submitted 'TruthfulQA: Measuring How Models Mimic Human Falsehoods' to arXiv on 8 September 2021. Rather than sampling questions at random, the authors deliberately selected 817 questions across 38 categories, including health, law, finance and politics, that some humans answer incorrectly because of a common false belief, superstition or misconception. That construction method is the point: a question bank built this way specifically tests whether a model reproduces the same popular errors a person might, rather than testing general knowledge recall, which is what most contemporary benchmarks measured instead.

How the test is scored

The benchmark's reference implementation supports two scoring approaches. The generation task has a model produce a free-text answer, judged for truthfulness and informativeness by a fine-tuned classifier the authors call 'GPT-judge,' which the repository states reaches roughly 90 to 95% agreement with human judgement on held-out data, an automated proxy validated against humans, not a ground-truth oracle. The multiple-choice tasks are more directly comparable across models: MC1 asks a model to select the single correct answer from four or five options, and MC2 asks it to assign probability across true and false options, scored by the normalised probability placed on true answers. The paper's headline finding used this apparatus to compare GPT-3, GPT-Neo/J, GPT-2 and a T5-based model.

The finding, and what 'truthful' means here

The paper reports that the best-performing model reached 58% truthfulness against a human baseline of 94%, and, the result that gave the paper its attention, that 'the largest models were generally the least truthful,' an inversion of the usual pattern in which larger models score higher on most benchmarks. The paper attributes this to models trained to predict likely text learning to reproduce widely repeated misconceptions more fluently as they scale, rather than learning to distinguish popular belief from fact. 'Truthful' in this benchmark has a specific, narrow operational meaning: avoiding a defined set of common false claims on adversarially chosen questions, not a general measure of factual reliability across arbitrary questions. A later survey on hallucination in large language models, posted in November 2023, defines hallucination separately as a model generating plausible yet nonfactual content, a related but distinct failure mode from the popular-misconception mimicry TruthfulQA specifically targets.

• Does a reported TruthfulQA score come from the generation task, MC1, or MC2, and do the three even remain comparable to each other?
• Has the model or its training data changed since the version of TruthfulQA used, given later models were often trained against publicly known adversarial question sets?
• Is 'truthful' being used here in TruthfulQA's narrow, misconception-specific sense, or is a score being generalised into a broader factuality claim the benchmark was not built to support?

The benchmark's contribution was to show that scale alone does not resolve every failure mode; a model can improve on broad knowledge tasks while remaining, or even becoming more, prone to confidently repeating a specific class of common human error.