Retrospective reference / 100 entries
The papers, releases, benchmarks, standards and rulings behind practical AI use, read for what they measured and what they left open.
Historical event dates and source dates are separate from the preparation date of this local edition. Every entry is a retrospective draft prepared 16 September 2026; none was published on its historical date.
100 entries
The transformer paper showed an attention-only sequence model beating recurrent ones on translation, at far lower training cost.
BERT's 2018 paper showed one bidirectionally pretrained model fine-tuning to state-of-the-art results across many language tasks.
The GPT-3 paper reported a 175-billion-parameter model performing tasks from in-prompt examples without updating its weights.
A 2020 paper fit power laws to loss versus model size, data and compute, and a 2022 paper later revised its recommended balance.
A 2022 paper found a compute-optimal model needs far more training tokens per parameter than the 2020 scaling recipe used.
The InstructGPT paper reported a 1.3-billion-parameter fine-tuned model preferred by human raters over the 175-billion-parameter base.
A 2022 paper found that asking a model to show intermediate steps raised reasoning scores, but the gain appeared only at large scale.
The original RAG paper trained retriever and generator jointly, an approach later practice largely replaced with separate off-the-shelf pieces.
A 2017 paper trained reward models from pairwise human comparisons on games and robotics, a method reused for language models in 2022.
The launch post names its RLHF method and lists specific failure modes, but carries no benchmark table of its own.
The paper reports smaller models trained on more tokens matching larger ones, while access stayed restricted to approved researchers.
OpenAI disclosed exam and benchmark results while declining to state parameter count, architecture or compute.
The licence permits commercial use for most companies while restricting the largest platforms and any use to train rival models.
The paper reports wins over Llama 2 13B and Llama 1 34B on its own benchmarks, released without the field-of-use limits a custom licence would add.
Two Anthropic posts announced the jump and a full model, while a separate paper found long contexts are not used evenly.
Google published its own scaling-law experiments and multilingual results but did not state the production model's size.
The launch reported a leading MMLU score under a specific decoding method, and Google's own video description discloses edited timing.
A 47-billion-parameter model used only 13 billion per token, which the paper reports let it match or beat Llama 2 70B.
The method trains harmlessness from a written set of principles and AI-generated comparisons rather than human-labelled examples.
Google's long-context report and a later benchmark paper show what a needle-in-a-haystack test measures, and what it does not.
Anthropic's Claude 3 model card ties benchmark scores to their sampling conditions, including where Opus falls short of expert accuracy.
Meta's Llama 3.1 paper and announcement show a 405-billion-parameter open model needs infrastructure most developers do not have.
OpenAI's GPT-4o announcement and its later system card disclose different levels of detail about rollout and voice-mode risk.
OpenAI's o1 announcement and system card show what hidden chain-of-thought reasoning discloses, and what it withholds.
DeepSeek's V3 and R1 papers state what their training-cost figure covers and how the comparison to OpenAI's o1 was actually made.
Anthropic's computer-use announcement and documentation show the capability requires dedicated sandboxing, not just a warning label.
Qwen2.5's announcement and model cards show that licence terms differ by model size within the same named family.
Meta's own wording and LMArena's model records show the tested Llama 4 Maverick variant differed from the released model.
Google's announcement and developer documentation show how Gemini 2.5's default reasoning is measured, and how it is billed.
The 2020 MMLU paper defined a 57-subject multiple-choice test whose single average later masked large gaps between subjects.
The 2022 HELM paper standardised scenarios and metrics across 30 models to expose how unevenly prior work had compared them.
The 2022 BIG-bench paper tested 204 crowd-sourced tasks and found most models still trailed human raters despite scale gains.
Two 2023 to 2024 papers show where GPT-4 judges track human preference and where pairwise votes and Elo-style ratings can mislead.
The 2023 SWE-bench paper and its 2024 Verified subset show how little of the original issue set even strong models could resolve.
The 2021 Codex paper introduced pass@k on 164 problems, and repeated sampling changed the result far more than a single run.
The 2021 GSM8K paper trained verifiers to pick correct answers, and a 2024 study found real but bounded overfitting on it.
The 2023 GPQA paper found PhD-level experts scored far higher than skilled non-experts with unrestricted internet access.
Three 2023 to 2024 studies show test data can leak into training data and propose different ways to detect it.
The 2018 datasheets proposal asks creators to record what a dataset should not be used for, and model cards extended it.
The LoRA paper reported matching full fine-tuning quality on GPT-3 175B while training far fewer parameters and less GPU memory.
The FlashAttention paper reported wall-clock speedups from reordering GPU memory access, not from approximating attention.
The paper reported large pretraining speedups from routing each token to one expert, at the cost of extra memory and serving complexity.
The paper found accuracy fell when relevant text sat in the middle of a long context, not just when the context grew longer.
GPTQ reported low-loss 4-bit compression, and llama.cpp turned that into inference consumer hardware could run.
A small model drafts tokens a large model verifies, producing identical text to standard decoding at lower latency.
The paper reported RLHF-level preference tuning from a classification loss alone, and practitioners later probed where that holds.
The paper reported fewer hallucinated facts when reasoning traces were interleaved with tool actions, on its own tasks.
A research method taught models to call tools via self-supervision, then a vendor API standardised structured calling.
Self-consistency reported gains from sampling multiple reasoning paths; Tree of Thoughts added search over them, at a token cost.
Anthropic's specification defines how hosts, clients and servers exchange messages, and leaves consent and security to whoever implements a connector.
OpenAI's ada-002 announcement and the FAISS paper show what an embedding index measures, and why a close vector is not the same as a relevant one.
OpenAI's Structured Outputs guarantees schema-valid JSON through constrained decoding, but a valid schema does not check whether the values inside are right.
Anthropic's and OpenAI's 2024 prompt-caching announcements specify different discounts, minimum lengths and lifetimes for reusing a processed prefix.
OpenAI's and Anthropic's batch APIs cut cost by half in exchange for a same-day turnaround window, for work that does not need an immediate answer.
The lm-evaluation-harness and OpenAI Evals projects show why naming a benchmark does not make two implementations of it comparable.
Hugging Face's own posts record why the Open LLM Leaderboard retired its original benchmarks in 2024 and what a leaderboard rank does not establish.
The SWE-agent paper reports that redesigning an agent's command interface, not just its underlying model, changed its measured SWE-bench resolve rate.
OpenAI's Operator launch materials and system card describe a browser-driving research preview built around confirmation prompts and site restrictions.
GitHub's 2022 general-availability post and a 2023 controlled study offer two different kinds of evidence for Copilot's effect on coding speed.
Simon Willison named prompt injection in 2022; later research showed the weakness is structural, not fixable by better prompt wording.
OWASP's Top 10 for LLM Applications turned scattered security concerns into a ranked list that changed between its 2023 and 2025 editions.
The AI RMF, released in January 2023, organises AI risk management into govern, map, measure and manage, without certifying any system.
Published in July 2024, the Generative AI Profile extends the AI RMF with risks and suggested actions specific to generative systems.
Regulation 2024/1689 phases in rules by risk category, with extra duties for general-purpose models judged to carry systemic risk.
Signed in October 2023, the order set compute-reporting duties for large models; a new administration rescinded it fifteen months later.
The GPT-4 System Card describes red-teaming, model mitigations and residual risk that OpenAI says remain brittle.
The Responsible Scaling Policy set AI Safety Levels with a conditional pause, then was revised as capabilities and safeguards changed.
TruthfulQA's 817 adversarial questions found the largest models tested were the least likely to avoid common human misconceptions.
The March 2023 announcement set a per-token price 10x below the prior GPT-3.5 rate, a recorded rate change rather than a current price to quote.
A 2022 paper measured three eras of AI training compute growth, each with its own doubling time and its own margin of error.
Stanford HAI's 2024 report compiles measurements on models, cost, and policy from many sources, and it names its own gaps.
The Open Source Initiative's 1.0 definition requires data information, code and weights under open terms, a bar most open-weight models do not meet.
Llama 2 and Llama 3 both cap free use at 700 million monthly users, but only Llama 3's terms require naming any derivative model after it.
The court found bad faith, not mere error, in submitting and standing by six non-existent judicial opinions generated by a chatbot.
The 2024 ruling rejected the airline's claim that a chatbot is a separate agent responsible for its own words.
Google said its fix for one bias problem created another, and it disabled image generation of people while it retrained the feature.
A filed complaint alleges unauthorised training and near-verbatim reproduction; OpenAI's public reply calls the claims without merit.
The Office's 2023 guidance and 2025 report both require a human author, disclosure of AI content, and case-by-case review of prompts.
The zero-shot method closed a robustness gap but inherited bias from its labels.
The public release followed a research paper on latent diffusion and a use-restricting licence.
OpenAI's diffusion decoder on CLIP embeddings was released gradually to test its safety systems.
An open speech model trained on 680,000 hours of weak supervision measured robustness, not just accuracy.
SAM's zero-shot masks and the SA-1B dataset moved segmentation toward a foundation-model pattern.
The 2021 specification defines a signed provenance record that generators began attaching to media.
Google's imperceptible watermark expanded from images to text, with detection limited by design.
OpenAI's system card documents nine months of testing and a layered provenance stack behind Sora.
The 2023 system card documents refusal testing and a jailbreak vector hidden inside images.
An 800-gigabyte open corpus named its sources, and one component later became unavailable.
RULER's synthetic tasks show many models fail well below their advertised context length, and why a single needle test misses it.
The 2015 subword paper and current tokenizer documentation show why token counts, not words, set cost and context use.
The 2019 nucleus-sampling paper shows decoding shapes output, and clarifies what temperature does and does not control.
Three papers show a model judge can be swayed by answer order, length and self-recognition, and how to calibrate for it.
A controlled 2023 comparison found retrieval outperformed fine-tuning for injecting new or changing facts into a model.
MTEB's eight-task benchmark found no embedding model wins everywhere, so a leaderboard rank is not a retrieval guarantee.
A harness team's retrospective and HELM show how prompt format, scoring and decoding choices shift benchmark scores.
OpenAI, Anthropic and Google document different deprecation notice periods, so a hosted model's availability is not permanent.
A 2024 paper found spending compute at inference can beat a larger model on some problems, at a real per-answer cost.
The 2015 distillation method still explains small models, but usage terms now govern whether distilling one is allowed.
Two 2013 papers showed word vectors trained on raw text captured similarity and analogy, and marked what such a vector does not represent.
Anthropic researchers built deceptive models with hidden triggers and found standard safety training could not remove the behaviour.
Try another category or clear your search.
