Open weights reached frontier scale but not frontier ease

Two releases, three months apart, one scale claim

Meta released the first Llama 3 models — 8B and 70B parameters — on 18 April 2024, trained, according to the announcement Introducing Meta Llama 3, on more than 15 trillion tokens with an 8,192-token context window. Three months later, on 23 July 2024, Meta released Llama 3.1, adding a 405-billion-parameter flagship alongside upgraded 8B and 70B versions, all extended to a 128,000-token context, described in Introducing Llama 3.1. The 405B model was Meta's stated attempt to match closed frontier models with openly released weights, a scale previously associated only with proprietary systems.

What the paper reports and what the licence changed

The Llama 3 technical report describes the 405B model as a dense transformer — not a mixture-of-experts design — natively supporting multilinguality, coding, reasoning and tool use, and states it delivers quality 'comparable' to GPT-4 across a range of tasks without giving a single head-to-head score in its abstract. Separately from the model itself, Meta's July announcement records a licence change: outputs from Llama models, including the 405B, could now be used to train or improve other models, opening the door to distillation at a scale the company says had not previously been permitted under its open terms. A licence change is a legal fact stated in the announcement, not a technical claim about the model, and the two should not be conflated when deciding what an open-weight release actually permits.

What running the model actually requires

Training the 405B model used, per the announcement, more than 16,000 H100 GPUs — infrastructure far beyond a research lab or individual developer. For inference, Meta quantised the model from 16-bit to 8-bit precision specifically so it could run within a single server node, and the announcement itself concedes that using the 405B 'requires significant compute resources and expertise' for an average developer. Open weights are not the same as an accessible model: the herd paper's multimodal extensions (image, video, speech) are explicitly described as experiments still under development and not broadly released alongside the text models, a distinction easy to miss in coverage that treats 'Llama 3.1' as one uniform release.

• Is the licence term permitting distillation from this model's outputs the version currently in force, or has it since changed?
• What hardware footprint does deploying the 405B model actually require, against the 8B or 70B alternatives in the same family?
• Which capabilities described in the paper, the multimodal extensions in particular, were research previews rather than part of the released weights?

Open weights at frontier scale shift what a lab can inspect and fine-tune, but they do not remove the resource gap between having a model file and being able to run it. That gap, and the specific licence terms attached to the weights, matter more to a deployment decision than the headline comparison to a closed competitor.