The Evaluating Self

8Emergent Abilities of Large Language Models

Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean and William Fedus, Transactions on Machine Learning Research, 2022

Linked to 13 chapters: supports 8 · contradicts 4 · mechanism 10 · same claim 3 · pairing 2

A survey of jumps: the argument in five moves Absent, thenpresent Eight tasksthat jump Promptingstrategies Metrics andexplanations Beyond scale Not in the smallmodels, there inthe large. So notpredictable. Few-shot tasks sitat chance, thenrise: arithmetic,Persian QA, MMLU. Chain of thought,scratchpad, instruc-tion tuning: no gainbelow a threshold. Exact match hideswhat the loss sees.No account of thejump itself. Data and objectivemove the threshold.Few can run theexperiment at all. A survey, not an experiment: the shape of a curve, and what the field could not forecast.
Figure 8.0 — The paper in five moves: a definition, eight curves that jump, four prompting strategies that only help past a threshold, a metric that may be doing the work, and the reasons scale is not the whole story.

By 2022 the standard result about language models was that they improve smoothly. Cross-entropy loss falls as a power law in compute, parameters and data, over more than seven orders of magnitude, and the curve can be fitted on small models and used to forecast large ones. Wei and sixteen co-authors at Google Research, Stanford, the University of North Carolina and DeepMind assemble the counterexamples. They collect tasks on which models of every size below some threshold score at chance, and models above it score well above chance, and they give the pattern a name and a definition. The paper introduces no model and runs no new training. It is a survey with a claim attached: that a part of what these systems can do could not have been read off the smaller versions of them, that the authors cannot say why, and that the field should treat this as a problem rather than a slogan. They also say, in a paragraph most later citations skip, that the threshold is a property of the models and the data and not of the ability, and that a better-trained small model may well reach a capability that a worse-trained large one missed.

An ability not present in smaller models

The broad idea is borrowed from Jacob Steinhardt and, behind him, Philip Anderson's 1972 essay More Is Different: "Emergence is when quantitative changes in a system result in qualitative changes in behavior." The authors narrow it to something testable. An ability is emergent if it is "not present in smaller models but is present in larger models". The consequence follows at once: such an ability cannot be predicted by extrapolating a scaling law fitted below the threshold, because below the threshold there is nothing to fit. Plotted against scale, the curve is flat at random until a critical point and then climbs steeply. The authors borrow a second word for that shape from Huberman and Hogg, writing in 1987 about dynamical systems: a phase transition.

Absent in the smaller models, present in the larger ones performance near random extrapolating from the small models predicts this well above random model scale (training compute, log) Schematic. Flat at chance, then a rise that fitting the smaller models does not forecast.
Figure 8.1 — The shape the definition names, drawn schematically. What makes the rise emergent is not its steepness but the fact that the flat stretch to its left carries no sign of it.

Scale here means training compute, measured in floating-point operations, on the horizontal axis; the same curves drawn against parameter count appear in an appendix and have the same shape, because dense Transformer families have scaled the two roughly together. Dataset size is not plotted, since many model families hold the number of training examples fixed across sizes. The authors are careful that no single quantity captures scale: Chinchilla has a quarter of Gopher's parameters at similar training compute, and sparse mixture-of-expert models carry more parameters per unit of compute than dense ones. Emergence, they conclude, is better viewed as a function of several correlated variables than of one.

The disclaimer that follows is the part of the paper that its title does not advertise. The scale at which an ability is first observed is not an immutable property of that ability. Higher-quality data can move it down. The authors state plainly that their goal is not to claim that a specific scale is required for anything, only to collect cases where a capability has so far appeared abruptly.

Few-shot prompted tasks

The first set of cases comes from prompting, in the form GPT-3 made standard: the pre-trained model is shown a few input-output pairs and then a new input, and completes the answer with no gradient update of any kind. A task is emergent in this setting when performance stays at random until a certain scale and then rises well above it. The paper gives eight such curves across five model families.

Four come from BIG-Bench, a crowd-sourced suite of more than two hundred benchmarks. On modified arithmetic — three-digit addition and subtraction and two-digit multiplication — GPT-3 and LaMDA sit near zero for several orders of magnitude of compute before jumping at 2×10²² training FLOPs, or 13 billion parameters, for GPT-3, and at 10²³ FLOPs, 68 billion parameters, for LaMDA. Transliterating from the International Phonetic Alphabet, recovering a word from its scrambled letters, and answering questions in Persian all turn at about the same place. TruthfulQA, a benchmark built adversarially against GPT-3, is never answered above chance by any GPT-3 model; Gopher stays at chance until 5×10²³ FLOPs, 280 billion parameters, where it rises more than twenty points above random. Grounded conceptual mappings, in which the model must read a cardinal direction off a textual grid world, work only in the largest GPT-3. On the Massive Multitask Language Understanding benchmark, which aggregates fifty-seven subject tests, no model at or below about 10 billion parameters beats guessing on the average across topics, and models from 70 to 280 billion beat it substantially.

The last of the eight is the one the authors return to. Word in Context is a semantic benchmark on which GPT-3 and Chinchilla both fail at their largest sizes. Brown and colleagues, reporting that negative result for GPT-3, blamed the decoder-only architecture and the autoregressive objective and proposed training a bidirectional model of comparable size as the remedy. What actually removed the failure was scaling the same kind of model further: PaLM at 2.5×10²⁴ FLOPs, 540 billion parameters, rises above chance without any of the architectural changes suggested.

Where each few-shot ability first rose above chance training FLOPs parameters 10²² 10²³ 10²⁴ toxicity classification (Gopher) 3-digit addition (GPT-3) 4–5-digit addition (GPT-3) MMLU, 57 topics (GPT-3) conceptual mappings (GPT-3) MMLU, 30 topics (Chinchilla) MMLU, 26 topics (Gopher) truthfulness (Gopher) word in context (PaLM) 7.1B 13B 175B 175B 175B 70B 280B 280B 540B Compute and parameter counts as reported in Table 1 of the paper; one point per ability.
Figure 8.2 — The nine named few-shot abilities in Table 1 of the paper, at the compute and parameter count where each was first reported above chance. Word in Context, at the far right, is the one that a smaller model was thought to need a different architecture for.

Augmented prompting strategies

The paper then extends its definition sideways. A technique, not just a task, counts as emergent if it gives no improvement, or does active harm, until applied to a model past a certain size.

Chain-of-thought prompting, which asks the model to write intermediate steps before the answer, beats plain prompting on grade-school maths word problems only from 10²³ training FLOPs, around a hundred billion parameters; below that it is worse than saying nothing. Instruction tuning — finetuning on a mixture of tasks phrased as instructions so the model can follow a description of an unseen one — damages performance at 7×10²¹ FLOPs, 8 billion parameters, and below, and helps from 10²³. The scratchpad, where the model is trained to emit the intermediate results of a computation, lifts eight-digit addition only from about 9×10¹⁹ FLOPs, 40 million parameters. Calibration — having a model estimate the probability that its own answer is right — is better done by asking the model to judge its proposed answer true or false than by reading off option probabilities, but only from about 3×10²³ FLOPs, 52 billion parameters.

Four strategies, and the scale at which each starts to pay scratchpad, 8-digit addition instruction tuning chain of thought calibration via P(True) 8.9×10¹⁹, 40M 1.3×10²³, 68B 1.3×10²³, 68B 2.6×10²³, 52B 10²⁰ 10²¹ 10²² 10²³ 10²⁴ training FLOPs no gain, or harm helps Thresholds from Table 1. Below its own threshold, instruction tuning lowered performance.
Figure 8.3 — Four strategies and the compute at which each begins to pay, from Table 1. The thresholds are three and a half orders of magnitude apart, so the word emergent names a shape, not a size.

Two caveats sit beside these. Sanh and colleagues induced instruction-following in an 11-billion-parameter encoder-decoder model shortly after the finding above, and the paper reports it. And across the paper's whole catalogue the thresholds run from 40 million parameters to 540 billion, which is to say that emergence marks a discontinuity in a curve and not any particular quantity of hardware.

Why it happens, and what the metric may be hiding

Against dozens of examples the authors set few explanations, and they say so. Some intuitions are available for particular tasks: a reasoning problem requiring a sequence of steps may need a network deep enough to run them; a closed-book question needs enough parameters to hold the compressed knowledge that answers it. Neither predicts a threshold.

The serious objection they raise against themselves concerns measurement. Exact string match awards nothing for a nearly correct long answer, and a multi-step arithmetic problem is scored only on the final number. A metric like that can turn accumulating partial progress into an apparent jump. The authors test the idea three ways. On the six emergent BIG-Bench tasks they consider, cross-entropy loss falls steadily through exactly the range of scales where exact match, BLEU and accuracy sit at random, which means the small models were improving and the reported metric was not showing it. On three generative tasks they recompute the curves under every metric BIG-Bench offers, including ROUGE, BLEURT and sequence F1, all of which give partial credit, and the emergent shape survives all of them. And emergence appears on plain classification tasks, where partial credit does not arise.

The same models, two measures exact match or accuracy training compute cross-entropy loss training compute Below the point where the reported score leaves chance, the loss has been falling all along. Schematic, after the cross-entropy curves given for six BIG-Bench tasks.
Figure 8.4 — What the two measures show over the same models, drawn schematically from the cross-entropy analysis in the paper's appendix. The loss falls the whole way; the reported score moves only at the end.

Their conclusion is deliberately incomplete. The metric hides real gains below the threshold, and the metric alone does not explain why the reported score should move suddenly rather than gradually once it starts to move, nor does the smooth loss curve let anyone say in advance where that will be.

A related analysis asks which kinds of task jump. Two of the authors hand-classified all 210 BIG-Bench tasks: 58 improved smoothly, 25 were emergent with GPT-3 or LaMDA, 42 became emergent only with PaLM, and 45 were flat, meaning no model of any size beat chance. Sorted by keyword, the highest proportions of emergent tasks were analogical reasoning, word sense disambiguation, truthfulness, social reasoning and emotional understanding. Arithmetic and mathematics scored low, which the authors found unexpected given that arithmetic supplied the earliest examples of the phenomenon. They report no pattern: there are, as they put it, no clear trends. The flat categories are more legible, and read as a list of what these models were not built for — visual reasoning, where eight of thirteen tasks are flat, along with non-language tasks, repeated interaction, long contexts, computer code and multi-step problems.

Beyond scaling

Scale is not the only lever, and the paper's own evidence says so. PaLM at 62 billion parameters performs above chance on fourteen BIG-Bench tasks where LaMDA at 137 billion and GPT-3 at 175 billion are at chance, despite less compute and fewer parameters; the likely causes are training data with more code and more languages in it, and a different encoding of digits. A continued pre-training stage on a mixture-of-denoisers objective unlocked several emergent tasks at fixed size. Once an ability is known to exist, it tends to descend: instruction-following reappeared in an 11-billion-parameter model, and a 1.3-billion-parameter InstructGPT model was preferred by human raters over models a hundred times larger. There are limits on the other side too. Scaling meets hardware constraints; some tasks may lie so far outside any training distribution that they never emerge; and an ability may emerge and then plateau below the level anyone wanted.

Risks scale as well, and the authors devote a section to them without claiming they are emergent by their own definition. Bias on ambiguous contexts in the BBQ benchmark increases with scale. Larger models memorise more of their training data. Larger GPT-3 models imitate human falsehoods more, though Gopher at 280 billion reversed that on a multiple-choice version of the same benchmark. A prize was running, as they wrote, for tasks that get worse as models get bigger.

The change they call sociological is the one that has aged best. Language processing had been a discipline of task-specific models; scaling turned it into a discipline of general-purpose ones. The marker they choose is the moment a few-shot prompted general model takes a benchmark record from a finetuned specialist: GPT-3 on TriviaQA and PiQA, PaLM at 540 billion on three arithmetic reasoning benchmarks, Flamingo at 80 billion on six visual question-answering benchmarks. Those results are not emergent under the definition, since their scaling curves are smooth. They mark a shift in what the field builds.

That shift has a consequence the authors state only obliquely. An emergent ability is invisible below its threshold, so whether one exists can be established only by someone with a model above it. Lowering the threshold, they write, matters for making research on these abilities available more widely — which is an admission that at 10²³ floating-point operations a question of fact has become a question of budget. The open problems they list follow from the same position: better architectures and training, including sparse experts, variable compute per input and external memory; more data, after the Chinchilla result showed earlier work had underestimated how much was needed; better understanding of why prompting works at all; the frontier tasks that no model could then do, which cluster in abstract reasoning; multilingual emergence, where Persian question-answering required both PaLM's data and 62 billion parameters; and, first among them, why any of this happens, since an explanation would turn the survey into a forecast.

Since 2022

The paper's central example did not survive intact. Rylan Schaeffer, Brando Miranda and Sanmi Koyejo argued in 2023, in a paper titled Are Emergent Abilities of Large Language Models a Mirage? (the next chapter), that the sharp transitions are largely produced by the discontinuous metrics used to score the tasks rather than by any change in the models. The dispute that followed settled one thing firmly and left the other open. The shape of a published curve does depend on the measure — a fact Wei and colleagues had themselves demonstrated in their appendix, where cross-entropy fell steadily beneath a flat exact-match line. What the dispute did not dissolve is the practical claim: on the measure a user of a system actually experiences, where an eight-digit sum is right or wrong, the transition is still abrupt, and nobody forecast which tasks would cross at which scale.

The response from the laboratories was to treat predictability as an engineering requirement. OpenAI's report on GPT-4 in March 2023 predicted the final model's loss, and its pass rate on a coding benchmark, from runs using a small fraction of the compute — an explicit attempt to put capability forecasting back on the smooth footing the scaling laws had promised. The Inverse Scaling Prize published its results the same year, adding tasks that get worse with scale and several that get worse and then better again, so that the catalogue of curve shapes is now longer than flat-then-jumping.

The forecasts in the paper's own Beyond scaling section held up better than its headline. Chain-of-thought prompting, presented here as something only hundred-billion-parameter models could use, was absorbed into training and by 2024 was behaviour that models produced without being asked. Instruction tuning descended to models small enough to run on a laptop. Many of the tasks on the flat list — no model above chance in 2022 — became routine; Suzgun and colleagues had already taken the twenty-three hardest BIG-Bench tasks, on which no model beat the average human rater, and shown that chain-of-thought prompting carried PaLM past that rater on most of them. And the sociological claim needs no defence at all: the general-purpose model won completely, and the task-specific model it displaced is now the unusual thing to build.

Claims drawn from this chapter · 8

evidenceAugmented prompting strategies

A prompting trick hurts small models and helps large ones

Whether a prompting technique helps depends on the size of the model it is used on. The same technique can do real harm below a certain amount of training compute and pay off above it.

Becausechain-of-thought is worse than saying nothing below 10²³ FLOPs, instruction tuning damages performance at 7×10²¹ and below, and the scratchpad pays from 9×10¹⁹ — thresholds three and a half orders of magnitude apart.
LimitsThe thresholds are measured with the same discontinuous metrics the mirage argument targets, and chain-of-thought has since been absorbed into training, so the technique-model pairing the paper measured no longer exists in that form.
PracticeTest a prompting technique on the model you will actually use. Advice that works on a frontier system can be worse than nothing on a small one, and the threshold is not a property of the technique.
After AIPrompting advice circulates as though it were about prompts. This result says it is about a prompt and a model together, which is why the same trick helps on one system and hurts on another.
evidenceWhy it happens, and what the metric may be hiding

Nobody can predict which tasks a bigger model will suddenly do well

Wei and colleagues found no feature that predicts which tasks a language model will suddenly become able to do as it grows. The categories that jumped, such as analogies and social reasoning, were not the ones the early examples suggested.

Becausetwo of the authors hand-classified all 210 BIG-Bench tasks — 58 smooth, 25 emergent with GPT-3 or LaMDA, 42 emergent only with PaLM, 45 flat — and found analogical reasoning, word sense disambiguation, truthfulness and social reasoning highest with arithmetic low, which they call unexpected and report as no clear trends.
LimitsA hand classification by two authors of tasks scored under the metrics the mirage argument disputes, and the flat list reads as what these models were not built for, which is a pattern of a kind.
PracticeDo not forecast which of your tasks the next model will handle. Test the ones that matter on each release, because the categories that moved last time do not predict the ones that will move next.
After AIThe most reliable finding here is a negative one about forecasting, and it is the part least often quoted. Planning that assumes you know which of your work is next to be automated has no support in this data.
evidenceBeyond scaling; Since 2022

Once a big model shows an ability smaller models soon get it

Once a large model has shown that an ability is possible, smaller models acquire it soon after. Instruction-following appeared in an eleven-billion-parameter model shortly after being reported to need far more compute, and a 1.3-billion InstructGPT was preferred to models a hundred times larger.

Becauseinstruction-following was induced in an eleven-billion-parameter encoder-decoder shortly after the finding that it needed 10²³ FLOPs, a 1.3-billion InstructGPT was preferred by raters over models a hundred times larger, and chain-of-thought was absorbed into training and produced unprompted by 2024.
LimitsThree cases, each of which followed a deliberate effort to reproduce a known capability; nothing here says every ability descends, or how far, and the descent is driven by post-training rather than by scale.
PracticeDo not build a dependency on the largest available model for a capability that has just been demonstrated. Wait for the descent, and audit which domains a small model now covers before paying for a frontier one.
After AIThe frontier is the expensive place to learn what is possible and the wrong place to run production work. The gap between demonstration and commodity has been about a year and is what makes the buying decision.
opinionBeyond scaling

Only owners of very large models can test claims about new abilities

Whether an ability appears only above a certain model size can be checked only by someone who already owns a model that large. Wei and colleagues report thresholds around 10²³ floating-point operations, which makes a question of fact a question of budget.

Becausethe authors remark that lowering the threshold matters for making research on these abilities more widely available, at a time when the thresholds they report run to 10²³ floating-point operations.
LimitsAn inference from the paper's own aside rather than an argument it makes; open-weights models above several of these thresholds now exist, and the rebuttal to this paper was written by academics without one.
PracticeNote who could have run the experiment when you read a capability claim. Where only a handful of organisations can test a proposition, the absence of contradiction is not evidence.
After AISchaeffer could rescore only one open family, which is this claim demonstrating itself inside the dispute it describes. The asymmetry has narrowed since and has not closed.
evidenceWhy it happens, and what the metric may be hiding

Small models were improving all along while the headline score stayed flat

The paper's own appendix shows that on these tasks the smaller models were getting steadily better the whole time, even though the score the paper reports stayed flat until a certain size.

Becauseon the six emergent BIG-Bench tasks considered, cross-entropy loss falls steadily through exactly the range of scales where exact match, BLEU and accuracy sit at random.
LimitsFalling loss under a flat score establishes that something improved, not that the improvement was the ability in question; the dispute since is over whether a smoothly improving quantity underneath makes the step unreal.
PracticeKeep a continuous measure beside any pass-or-fail score you track, including of your own judgement. A flat record of outcomes can sit on top of steady movement that the scoring rule cannot show.
After AIThe evidence that dismantled the paper's headline was printed in the paper's own appendix, by its own authors. That is a good standard to hold model evaluations to, and a rare one.
evidenceFew-shot prompted tasks

Some AI tasks stay at chance for a long time then jump

Wei and colleagues show eight tasks, each given to the model as a handful of examples in the prompt, across five model families: scores stay at chance over orders of magnitude of training compute, then rise well above it.

Becausemodified arithmetic turns at 2×10²² FLOPs for GPT-3 and 10²³ for LaMDA, TruthfulQA is never above chance for any GPT-3 model while Gopher rises more than twenty points at 5×10²³, and MMLU is beaten by no model at or below about ten billion parameters.
LimitsEvery curve here is drawn with a scoring rule that demands an exactly correct answer, and Schaeffer's reply shows that such a rule turns smooth improvement into a step; the underlying per-token loss was falling throughout.
PracticeA model that cannot do a task at all is weak evidence that the task is out of reach for the class. Re-test the tasks you wrote off when a larger system arrives, rather than carrying the verdict forward.
After AIThis is the paper behind the expectation that the next model will suddenly do something. The expectation was right about the user's experience and wrong about the mechanism, and both halves are still in circulation.
evidenceBeyond scaling

Wei's paper shows harms grow with model size just as abilities do

Wei and colleagues report that as models get bigger they also get worse in some ways: more bias on ambiguous questions, more training text memorised, and more imitation of human falsehoods. The paper does not call these risks emergent.

Becausebias on ambiguous contexts in BBQ rises with scale, larger models memorise more of their training data, larger GPT-3 models imitate human falsehoods more, and a prize was running for tasks that get worse as models get bigger.
LimitsReported as a list of findings from other groups rather than measured here, and the Inverse Scaling Prize results since include tasks that get worse and then better again, so the curve shapes are more various than a monotone risk claim.
PracticeRun your failure-mode review against each new model rather than assuming the last review holds. Some of what got worse did so because the model got better, and that class of regression does not announce itself.
After AIThe capability announcement and the risk announcement describe the same event. This paper was already saying so in 2022, next to the headline everyone took away instead.
evidenceWhy it happens, and what the metric may be hiding

Whether a skill appears depends on the model and its training data

The size at which a new ability appears is a fact about the particular models and what they were trained on, not about the ability itself. Most people who cite Wei's paper skip that paragraph.

BecausePaLM at 62 billion parameters is above chance on fourteen BIG-Bench tasks where LaMDA at 137 billion and GPT-3 at 175 billion sit at chance, on less compute, with more code and more languages in the data and a different encoding of digits named as the likely causes.
LimitsThe causes are the authors' plausible attribution rather than a controlled comparison — no two of these families differ in data alone — so the claim that data rather than scale moved the threshold is inferred.
PracticeAudit by model and domain rather than by size. Whether a system can do your task is a fact about what it was trained on, which is why a smaller model sometimes beats a larger one on exactly the work you need.
After AIModel choice for real work turns on this and not on the leaderboard. The paper says so in a paragraph that its own headline buried.

All claims · Source map