14Conditions for Intuitive Expertise

Daniel Kahneman and Gary Klein, American Psychologist 64(6), pp. 515–526, 2009

Linked to 33 chapters: supports 19 · contradicts 2 · mechanism 20 · same claim 6 · pairing 5

Kahneman and Klein came to this paper from opposite sides of a long quarrel. Klein had spent his career studying how firefighters, nurses and soldiers make good decisions without deliberating, and belongs to the community that calls itself naturalistic decision making. Kahneman had spent his running experiments in which intuitive judgment turned out to be wrong, and is identified with the heuristics and biases programme. They expected, when they sat down together over several years, to disagree on most things. They found that they agreed on almost everything that mattered, and the paper's subtitle records the surprise: a failure to disagree. What they agree on is more specific than the commonplace that intuition is sometimes brilliant and sometimes foolish. Intuition, they say, is recognition. It is trustworthy when two conditions are met: the environment must be regular enough to offer valid cues, and the person must have had an adequate chance to learn them. Where both hold, skill develops in anyone with the talent for it. Where either fails, intuitions still arrive, feel just as compelling, and are wrong more often than they seem. The feeling of confidence that comes with a judgment tells you nothing about which case you are in. What separated the two authors at the end was not a finding but a temperament: one still delights in the comeuppance of overconfident experts, the other still flinches at the word bias.

Two traditions

The naturalistic side descends from Adriaan de Groot's studies of chess in the 1940s, which found that grandmasters saw the promising move almost at once while weaker players never even considered it. Chase and Simon in 1973 described this as perceptual skill, a repertoire of some fifty to a hundred thousand recognisable patterns that takes about a decade of serious play to assemble. Simon drew the definition the paper adopts: intuition is the recognition of patterns stored in memory. Klein's own starting point was a 1986 study of fireground commanders, who decide under time pressure that rules out any orderly comparison of options. The hypothesis was that they compared two. They compared none. A commander typically generated a single course of action, ran it forward in imagination to see whether it would work, adopted it if it would, adjusted it if it had a flaw, and only if it could not be fixed reached for the next most plausible option. Klein called this recognition-primed decision, and it was found again in system design, military command and offshore oil platforms.

The movement crystallised after a failure rather than a success. In 1988 the USS Vincennes, an Aegis cruiser carrying the most advanced systems in the fleet, shot down an Iranian airliner. The Navy funded a programme of research on decision making under stress, and in 1989 thirty researchers met to look for what firefighters, plant controllers, officers and highway engineers had in common. The aim of the field that emerged is to demystify intuition by finding the cues experts use, even where the experts cannot say what they are. Its method is the cognitive task analysis, a structured interview about specific incidents. Crandall's study of nurses in a neonatal intensive care unit is the paper's standing example. The nurses could tell that an infant was developing sepsis before the blood tests came back and could not explain how. The interviews surfaced a set of cues, some of them the opposite of the signs of infection in adults, that later became a training programme.

The heuristics and biases side descends from Paul Meehl's 1954 monograph, which reviewed about twenty studies comparing clinical predictions with simple statistical models and found the models more accurate in nearly every one, even though they used less of the information. Meehl blamed base-rate neglect and, in a later paper on why he no longer attended case conferences, the clinicians' uncritical trust in their own intuitions. Goldberg's 1970 bootstrapping study made the point sharper. He fitted a regression model to each of twenty-nine clinicians' diagnoses of 861 personality profiles, then used each clinician's model to predict new cases. The model of the judge beat the judge nearly every time. The only explanation is that human judgment is noisy enough to impair its own validity. Kahneman read Meehl in 1955 while assessing officer candidates for the Israeli army. He describes the strong sense of knowing each candidate, and the conviction that he could foretell how they would do, which the statistical feedback from officer school, showing that the assessments were nearly worthless, did nothing to shake. He named that the illusion of validity. The first study in the programme proper, published in 1971, asked methodologists and statistics textbook authors to choose sample sizes for experiments. They chose badly, and by intuition, though they knew the rules.

Three contrasts

The two communities differ first in stance. Heuristics and biases researchers are trained to look for a comparison between an expert and a formal rule and to expect the expert to lose, and they lean towards replacing judgment with algorithms wherever possible. Naturalistic researchers are trained to look for what an expert sees that a novice does not, and to pass it on. The authors say the sharpest difference between the two of them was emotional, not intellectual. They differ second in setting. One side works in university departments and controlled laboratories, the other among practitioners in the field. But there is no logical tie between the setting and the claims: the recognition-primed model has been tested in the laboratory, and anchoring has been shown in courtrooms and property markets. They differ third, and most consequentially, in what counts as an expert. Naturalistic research usually relies on peer judgment. If colleagues say that had one commander been present instead of another the fire would not have spread so far, that commander is the expert. The heuristics and biases standard is optimality: a professional whose predictions are beaten by a linear combination of a few variables is suboptimal, however much better than a novice. A clinician can pass the first test and fail the second at once, and much of the historical quarrel is the two tests being confused for each other.

Where intuitions come from

Both traditions accept the two-system vocabulary. Intuitive judgments arrive without effort or awareness of the cues that evoked them, and belong to System 1. Checking them, calculating, filling in a tax form or parking in a narrow space are deliberate, effortful and belong to System 2, which also does the self-monitoring that concurrent effort impairs. The recognition-primed model uses both: the automatic arrival of an option and the deliberate simulation of it. The difference between the traditions is which class of intuition each studies. One studies intuitions that come from experience and show skill. The other studies intuitions that come from simplifying heuristics and show no skill at all, and these are the ones prone to systematic bias.

Simon's definition carries the argument. "Intuition is nothing more and nothing less than recognition." The nurse recognising a sick infant is doing, in principle, what a child does in recognising a dog, or what anyone does in hearing fatigue in a familiar voice on the telephone. No magic is involved, and the demystification has a consequence. Recognition requires something to recognise. The environment must contain cues that are valid, meaning that they are statistically or causally tied to what follows, whether or not anyone can state them. A child cannot say what makes a dog a dog. There are almost certainly early signs that a building is about to collapse in a fire or that an infant is about to show symptoms. There is almost certainly no public information that predicts the price of a stock, because if there were the price would already reflect it, and so a trader's intuition about a stock deserves less trust than a commander's about a building, whatever the trader's experience.

Validity is not always obvious in advance. Tetlock's fifteen-year study of expert political forecasting, published in 2005, found that highly educated specialists did no better at long-range prediction than attentive newspaper readers, and the authors read the consistency of the failure as a verdict on the task rather than the forecasters. Large-scale history is not forecastable. They offer a thought experiment: had the three fertilised eggs that became Hitler, Stalin and Mao been female, the century would have gone differently, but in what way no one could say. Some environments are worse than unpredictable. Hogarth calls an environment wicked when the feedback it gives confirms a wrong intuition. His example is the physician of the early twentieth century who could sense which patients on the ward would develop typhoid, and confirmed it by palpating their tongues without washing his hands. High validity, on the other hand, does not require certainty. Bridge and poker are high-validity games: skill at finding the favourable bet improves steadily without any bet being sure.

The second condition is the chance to learn. Chess is said to need ten thousand hours, but most skills need far less; a child does not need thousands of dogs. The nurse has seen enough sick infants, and the commander has attended enough fires and imagined many more in years of talking about them. Without that opportunity a correct intuition is luck, and the authors state that they do not believe in magic. Talent then sets the pace. Every child learns dogs, but not every devoted player becomes a grandmaster, and Fischer and Kasparov saw patterns their peers could not find unaided, though the peers could see they were right once shown. What is called creativity fits the same account. The Remote Associates Test asks for the one word that goes with three others. Cottage, swiss and cake is easy. Dive, light and rocket is hard, yet everyone recognises the answer once it is given, because the association was in memory all along.

Imperfect intuition

Both authors agree that most of what System 1 produces is skilled and works. They disagree only about how common the exceptions are. The exceptions arise when a condition is unmet, either because the environment is irregular, as in political forecasting, or because the person has not had the chance to master it, as with firefighters at a skyscraper fire whose structural heat shielding has failed in a way none of them has met. Baseball scouts in Michael Lewis's account had every chance to learn and still judged players by their looks, which the authors call a plain case of prediction by representativeness.

Where an intuition does not come from skill it comes, like the skilled kind, from memory, and three laboratory phenomena show the mechanisms. Frederick's bat-and-ball problem states that a bat and ball together cost a dollar and ten cents and the bat costs a dollar more than the ball. The answer that arrives at once, ten cents, is wrong, and a moment's check shows it, yet about half of students at the most selective universities and nine in ten elsewhere give it without checking. Business books that celebrate leaders who went with their gut, the authors add, are usually celebrating luck. Anchoring is the second. People first asked whether the average German car costs more or less than a hundred thousand dollars estimate the average far higher than those first asked about thirty thousand, by about half the gap between the anchors, because the first question brings Mercedes and BMW to mind and the second a Beetle. The estimate is a deliberate act, but the sample it draws on was biased in the automatic stage, and the victims deny any effect. The third is attribute substitution. Told that Julie read fluently at four and asked to guess her graduating grade average, people produce a grade as impressive as her reading, which is a matching of percentiles that ignores the weak correlation between the two. A hard question has been answered by silently swapping in an easy one, which is what a heuristic is.

None of these mechanisms operates where skill exists; a skilled person applies the skill. The difficulty is that in the absence of skill an answer still arrives, and there is no subjective marker that distinguishes it from the skilled kind. A single response comes to mind, as a perception does, and most of the time trusting it is right. But confidence is not a measure of accuracy. It is a measure of the internal consistency of the evidence the judgment was built on, which is why evidence that is both redundant and flimsy produces judgments held with too much conviction, presented too assertively, and believed more than they deserve. The only safe way to assess a judgment, one's own or anyone's, is to ask about the validity of the environment and the judge's history of learning it.

Fractionated expertise

Shanteau's 1992 review sorted occupations by whether experienced practitioners showed real expertise. Livestock judges, astronomers, test pilots, soil judges, chess masters, physicists, mathematicians, accountants, grain inspectors, photo interpreters and insurance analysts did. Stockbrokers, clinical psychologists, psychiatrists, college admissions officers, court judges, personnel selectors and intelligence analysts did not. His explanatory factors, predictable outcomes, enough experience and good feedback, are the paper's two conditions under other names, with static rather than dynamic stimuli added. Three professions appeared on both lists: nurses, physicians and auditors. The authors call this fractionated expertise and say it is the rule. An auditor skilled with receivables may be poor at spotting fraud. A weather forecaster is good at temperature and rain and poor at hail. A physician meets diagnoses that are new to them.

The two authors reach fractionation from different examples. Klein thinks of the expert who performs a constant task and meets an unfamiliar case, and counts the ability to notice that a case is anomalous, and to switch to deliberate search, as a mark of real expertise; the medical literature on diagnosis describes exactly this. Kahneman thinks of professionals who use knowledge that is real for one purpose to answer questions of another kind. A financial analyst, a psychotherapist or an intelligence officer may know a great deal about a company, a patient or a conflict and have had ample feedback on short-term tasks, while feedback on long-term judgments is delayed, sparse and ambiguous. That pattern breeds overconfidence. Experts detect anomalies in the situation in front of them; nobody is good at detecting patterns in the inner situation that would show where a judgment came from. Klein's experts seem less prone to it, partly because error costs them personally, and forecasters and engineers habitually decline questions outside their competence and know that a better expert exists elsewhere.

Algorithms

The Meehl paradigm, in which a professional is set against a formula, is a sore point between the communities: a source of pride on one side and distrust on the other. The evidence is not in dispute. Grove and colleagues' meta-analysis of 136 comparisons in 2000 found the mechanical prediction better in about half, equal in most of the rest and worse in a handful; the tasks with the largest margins included predicting college performance, throat infections, psychiatric length of stay, job turnover, suicide attempts and juvenile delinquency. The authors reject the usual reading of this as proof of human ineptitude. The tasks were low-validity tasks, the algorithms were often wrong too, and no study has shown a clinician missing a strong cue an algorithm caught. Human learning is efficient where cues are simple and feedback is fast, except in wicked environments. What the formula has is two things a person lacks: it can extract weakly valid cues from noise, and it applies them consistently every time. Karelaia and Hogarth's meta-analysis attributes most of the algorithmic advantage to consistency alone. The scoring of personal loans is the model case, a low base-rate outcome where formulas have replaced loan officers and made decisions fairer, faster and cheaper.

Algorithms win, then, in two different places: where validity is so low that human inconsistency is fatal, and where it is so high that the ceiling is reached and the only human contribution is the occasional lapse of attention, as with the driverless trains in airports. Between them the naturalistic objections hold. A usable algorithm needs confidence in its list of variables, a measurable criterion, a body of similar cases, a cost that the gain justifies, and conditions that will not change under it, and it must remain under human supervision, which is hard to keep up, because operators become passive and less vigilant once a machine is in charge, a pattern already named automation bias. Formal aids meet resistance, sometimes for good reasons, and Kahneman sees the resistance as an obstacle where Klein sees it as a warning. They agree on the semi-formal middle. Klein's premortem asks a team that has settled on a plan to imagine that it has already failed and to spend two minutes writing down why, then to go round the table starting with the leader. It rests on the finding that people generate more reasons for an outcome once told it is certain, and it reopens the doubts an organisation suppresses as it commits.

Since 2009

The paper's two conditions became the standard test for when to trust an expert, and Kahneman retold it two years later in his own book as the chapter on expert intuition. Meehl's result has not moved; the 2000 meta-analysis is still the reference. Anchoring, the effect the authors chose as their sturdiest example, was among the findings that replicated cleanly when large multi-site replications began in 2014, while several other laboratory effects from the same era did not. Tetlock's verdict on political forecasting was softened by his own later work. In the forecasting tournaments that began in 2011, ordinary volunteers who received feedback, practised and kept scores beat intelligence analysts on questions a few months out, which is what the paper's conditions predict once feedback is fast and the horizon short; the long horizon stayed unforecastable. Klein's premortem passed into ordinary management practice. The consistency argument for formulas grew into Kahneman's last book, on noise, and the warning about supervision of algorithms grew with the systems it described. What has not changed is the part of the paper that its authors seemed least pleased with: each community still finds the other's favourite cases uninteresting, and a psychology of judgment that takes both seriously is still mostly a hope.