GPT-3 showed a model could learn tasks from prompts alone

A large model tested without fine-tuning

OpenAI posted Language Models are Few-Shot Learners on 28 May 2020, describing GPT-3, an autoregressive language model with 175 billion parameters, ten times more than the paper says any earlier non-sparse language model had. The paper's central claim is that scaling up a language model improves what it calls task-agnostic few-shot performance: GPT-3 is given a task description and a handful of examples inside the prompt text itself, with no gradient update and no change to its weights, and is then asked to complete a new instance of the task.

What the paper reports and what it flags

The paper reports that, on this basis, GPT-3 sometimes matches the performance of prior systems fine-tuned specifically for the task in question. It is equally explicit about the reverse: the authors name datasets where GPT-3's few-shot performance still struggled, and others where they say the model faces methodological difficulties tied to having been trained on a large, uncurated web corpus rather than a task-specific dataset. The accompanying repository, as retrieved on 16 September 2026, documents the evaluation apparatus rather than the model itself: unfiltered sample text with a content warning, the synthetic word-scramble and arithmetic tasks used in the paper, a training-data language breakdown, and an overlap analysis checking how much benchmark text leaked into training data. The repository does not provide the model weights.

The limit of a few-shot claim

A few-shot result describes performance with examples supplied at inference time, not a permanent skill acquired by the model, and it is sensitive to which examples are chosen and in what order, a sensitivity the paper's headline abstract does not fully characterise. The paper's own flagged concern about human ability to distinguish GPT-3's generated news-style text from real articles is a statement about one generation experiment, not a general claim about the detectability of the model's output elsewhere. Because the released repository withholds the weights, no outside party could rerun the exact evaluation independently against the identical checkpoint, which limits how far any single reported score should be generalised.

Questions to carry into your own evaluation

• Is a quoted GPT-3 score a few-shot, one-shot or zero-shot result, and how many in-prompt examples were used?
• Does the benchmark in question risk overlap with GPT-3's training data, as the paper's own overlap analysis attempts to check?
• Was the underlying model checkpoint ever made available for independent replication, or only accessible through a controlled API?

GPT-3's paper demonstrated that scale plus in-prompt examples could substitute for fine-tuning on some tasks. It reported that result alongside its own list of struggling datasets and methodological caveats, and neither the paper nor its repository offers a way to independently rerun the evaluation against the original weights.