Chain-of-thought prompting helped only sufficiently large models

Asking for the steps, not just the answer

Researchers at Google posted Chain-of-Thought Prompting Elicits Reasoning in Large Language Models on 28 January 2022. The method is simple to describe: instead of showing a model a few examples of questions paired directly with answers, the prompt shows a few examples of questions paired with a written series of intermediate reasoning steps leading to the answer, then asks the model to continue the pattern on a new question. The paper reports this improves performance on arithmetic, commonsense and symbolic reasoning tasks.

What the paper reports, and for which models

The clearest reported result is on GSM8K, a set of grade-school math word problems: the paper states that a 540-billion-parameter model, prompted with only eight chain-of-thought exemplars, reached accuracy it describes as state of the art, surpassing a fine-tuned GPT-3 model paired with a separate verifier. The paper is explicit that this is a scale-dependent effect: it describes the reasoning benefit as something that emerges in sufficiently large models, and the evidence it presents is drawn from large models rather than showing a comparable gain on small ones. A later paper, posted on 21 March 2022, changed how the final answer is chosen from a chain-of-thought prompt: instead of taking the single greedy continuation, it samples several different reasoning paths for the same question and picks the answer that recurs most often, reporting further gains including 17.9 points on GSM8K and smaller but positive gains on SVAMP, AQuA, StrategyQA and ARC-challenge.

The gap between a written step and a verified one

Neither paper claims a written reasoning step is a faithful account of how the model actually arrived at its answer; a chain of thought is text the model generates the same way it generates any other text, and a plausible-looking sequence of steps can accompany a wrong final answer, or a correct answer can follow steps that do not actually support it. The self-consistency method's improvement comes from voting across multiple generated chains, which raises the odds of landing on a correct final answer without making any individual chain more trustworthy as an explanation. Applying either method to a model too small to show the reported emergence, or reading a written rationale as proof of correct reasoning, goes beyond what either paper measured.

Questions to carry into your own evaluation

• Is a chain-of-thought result being generalised from a large model to a smaller one that these papers never show the effect on?
• Is a displayed reasoning chain being treated as an explanation of the model's process, or only as text that preceded a correct answer?
• Would voting across multiple sampled chains, at the reported extra inference cost, be practical for the task at hand?

Chain-of-thought prompting is a documented gain on specific reasoning benchmarks, reported for large models and refined by sampling multiple reasoning paths rather than one. It is not evidence that a model's written steps reflect its actual computation, and its reported gains should not be assumed to hold at a model scale the papers did not test.