43Sources of Power

Gary Klein, Sources of Power: How People Make Decisions, MIT Press, 1998

Linked to 27 chapters: supports 14 · contradicts 12 · mechanism 13 · same claim 5 · pairing 7

Klein set out to correct a balance. For twenty-five years, he writes, research on decision making had concentrated on the limits of the decision maker, and had produced training courses, decision aids and expert systems meant to keep fallible people out of the way. He proposed to document the strengths instead, and to look for them where they are used: in fire stations, intensive care units, tank turrets, cockpits and warship command centres. The method was to find people with twenty years in the job, ask them for their hardest recent case, and take the story apart minute by minute. What came back overturned the hypothesis he had started with. Experienced commanders under extreme time pressure did not weigh two options against each other; they did not weigh any. A situation was recognised as one of a familiar kind, a course of action arrived with the recognition, and the commander tested that one course by imagining it carried out. The capacities that make this possible are the book's sources of power: intuition, mental simulation, storytelling, analogy, spotting leverage points, reading another person's intent, the shared mind of a team, and rational analysis alongside these rather than above them. The analytical method taught in decision courses, Klein concludes, is not the ideal that experience approximates. It is the fallback for people who do not yet know what to do.

Learning from the firefighters

The study began with a single paragraph. In 1984 the U.S. Army Research Institute asked for proposals on how commanders decide under uncertainty and severe time stress, having spent a decade building decision aids for battle commanders that no one would use. Klein's small company won the contract and went looking for people who decide fast and pay for their mistakes.

He is candid about how badly he designed the study. Six features went into the plan and four were wrong. Analogical reasoning was expected everywhere and was almost absent: in twenty-five years of fires nothing had stayed separate enough to be recalled as a case, and when a memory did surface it was of a fragment, as when a chief saw billboards on a roof, remembered supports burning through at an earlier fire, and pushed the crowd back. The two-option hypothesis was the centrepiece and it collapsed first. It came from Peer Soelberg's 1967 dissertation at the MIT Sloan School, which found that students taught the rational choice method made a gut choice about their own jobs and then built a case for it against one token rival. They were not deciding, they were assembling a justification.

Klein expected the same from fireground commanders and thought the hypothesis daring. It was conservative. At the first background conversation, before the real interviews began, a commander told his startled listeners, "I don't make decisions," and added that he could not remember ever making one. Pressed, he agreed that options existed at a fire — whether to call a second alarm, where to send crews — but said it was usually obvious what to do, and that there was no time for anything else: the building would burn down while he listed the alternatives. He was using the word decision as Soelberg's students had used it, to mean laying two or more options side by side and comparing their strengths and weaknesses. Klein calls that comparative evaluation. The commanders insisted they never did it, and the interviews never produced evidence that they did.

Two features of the plan survived. Fireground commanders were the right people to study, since military commanders turned out to use the same strategies; and asking for hard cases rather than routine ones was right, because routine cases produce tired generalities while hard ones produce the stories a commander tells back at the station. The interviews were built around those stories, looking for decision points: moments where another course was available and known.

The recognition-primed decision model

What the stories showed was not a refusal to compare options but no need to compare them. A commander arrives at a four-storey apartment building, finds flames climbing a laundry chute, recognises a vertical fire and sends crews above it; minutes later, seeing smoke under the eaves, he recognises that the fire has reached the top floor, abandons the attack, switches to search and rescue and calls a second alarm. At no point does he hold two courses of action in mind at once. Experience let these men see even a non-routine situation as an instance of a prototype, which carried with it the goals worth pursuing, the cues worth watching, what to expect next, and what to do.

The team classified 156 decision points, with two coders on each and a third to arbitrate disagreements, and put anything doubtful into the comparative category so as not to flatter the finding.

Eighteen decisions were comparative, and about half of those came from a single incident: an oil tank farm that caught fire on Christmas night, where volunteer chiefs used to barn fires faced a wall of flame a hundred feet high. On the third day they made what one called their first effective decision, which was to stop fighting the fire and plan — the strongest example of deliberate, option-weighing decision making in the whole study, produced by men who were, for that fire, novices. Eleven decisions invented a procedure that had not existed. The remaining 127, 80 percent of the total, were recognitional.

The second puzzle was how a single option gets evaluated when there is nothing to compare it with. The answer is in a rescue on a highway overpass, where a lieutenant cycles through a rescue harness, the same harness attached from behind, and a rescue strap, imagining each used on a semi-conscious woman lying across a metal strut, finding a fault in each, and improvising with a ladder belt. The whole sequence took about a minute; he looked at several options and never compared two. Klein calls this singular evaluation and ties it to Herbert Simon's satisficing: you take the first option that works, the way you stop at the first plausible filling station rather than searching for the best one in town. Before the study Klein had assumed that novices leap at the first idea while experts deliberate. It is the other way round.

The model that fuses recognition with singular evaluation comes in three variations, and the difference between them is where the uncertainty sits.

In the first, the situation is recognised and the typical action follows, an if, then with the expertise concentrated in knowing that the if has been met. In the second the information does not match one prototype cleanly, or an expectancy has been violated, and the decision maker has to gather cues and build a story before recognition can do its work: if what?, then. In the third the action is recognised but its consequences are not obvious, so it is run forward in imagination and adopted, adjusted or abandoned: if, then what?

Klein then went looking for places where the model would break. The percentages move with experience and with the kind of decision, and never fall to zero.

Tank platoon leaders on their first course at Fort Knox, some of whom had never ridden in a tank, fell below 50 percent. Forest Service incident commanders used recognition as often as fireground commanders when the decision was about fighting the fire, and much less often when it was about disciplining someone, a task in which they were less expert. Army brigade command teams, drilled to generate three options, produced one comparison in twenty-seven decision points across a five-hour planning session. Independent groups repeated the result on airline crews in a simulator, on offshore installation managers, on British Army officers and on electronic warfare technicians.

None of this says that comparison is never right, and Klein sets out the conditions that pull each way. His case for the other side is the Denver police handgun dispute, settled by a formal analysis that separated the city's value trade-offs from the ballistics experts' technical judgments.

The power of intuition

Intuition, in Klein's account, is the recognition of patterns, and its reputation is strange because the patterns are subtle enough that people cannot say what they noticed. His first formal interview produced a case he did not want: a lieutenant insisted on telling him about the time extrasensory perception saved his crew. It is a small house fire, apparently in the kitchen. The hose crew sprays and the fire roars back; they spray again with the same result and retreat a few steps. The lieutenant feels that something is wrong, cannot say what, and orders everyone out. The floor where they had been standing collapses into the fire below.

Close questioning turned up what he had actually used. The living room was hotter than a small kitchen fire should have made it. It was very quiet, and fires are noisy. The water was having no effect. The seat of the fire was in a basement he did not know existed, directly beneath him; the floor muffled the sound and the heat came up through it. He was reacting not to events but to the absence of expected ones, which is why he could not describe it and why he called it a sixth sense.

The commander, Klein records, was pleased to see how his experience had worked and also shaken, having come to rely on a power he now suspected he had never had. The same pattern turned up at the end of the Gulf War in HMS Gloucester, whose anti-air warfare officer identified a radar contact as a Silkworm missile within five seconds, forty seconds before the ship's altitude-finding radar could confirm it, on a scope that did not distinguish it from an American aircraft. He said the track had felt as though it were accelerating, and the tape showed no acceleration at all. What he had used was the distance from the coast at which the track first appeared, which is a function of altitude: he had read altitude off a radar that does not measure it.

The clearest case is nursing. Beth Crandall studied nurses in a neonatal intensive care unit who could tell that a premature infant was developing sepsis and could not say how; the infants weighed a few pounds, some under two, and an infection can kill before antibiotics take hold. Asked how they did it, the nurses said experience, or intuition, and had nothing further to offer. Asked instead for particular babies and particular shifts, they remembered exactly what had caught their attention. Crandall compiled the cues and had them checked by neonatologists. Some were already in the medical literature; almost half were not, and several ran opposite to the signs of infection in adults. An adult with an infection becomes more irritable. A premature infant becomes less so, and a baby who cried every time it was lifted for weighing and one day did not cry was a warning. The nurses were rarely working from one cue but from patterns of several, each slight. The list became training material, with the cues embedded in the stories they came from, and nursing students still recalled them weeks later.

The power of mental simulation

A senior developer at the National Fire Academy stood up mid-meeting, closed the door, and said in a lowered voice: "To be a good fireground commander, you need to have a rich fantasy life." He closed the door because imagination is as embarrassing a thing to credit as intuition. Klein's name for it is mental simulation: imagining people and objects, running them through a series of transformations, and watching what happens. It is not a picture but a sequence of pictures.

He and Crandall collected seventy-nine cases from their own fireground records, from published accident analyses and from interviews. Two regularities came out of the set. The simulations were built like machines, one thing driving the next, and they were small machines: each turned on a handful of factors, rarely more than three, and ran through about six transitions before it finished. Klein states this as a specification — three moving parts, six steps. Experience buys room within the constraint, since a familiar sequence can be treated as a single step; but the constraint tightens sharply when the parts interact rather than act in turn.

What experience contributes shows up in an interview Klein arranged on an impulse. When Poland decreed the switch to a market economy on 1 January 1990, he asked three people to imagine how the year would go. Andrzej Bloch, a Polish economist, reduced it to three variables — inflation, unemployment, the exchange rate — and ran them forward, putting inflation near 1,000 percent for a few months before it fell back. A year later that forecast proved accurate to the quarter. One of Bloch's best students and a professor of political science who had spent a sabbatical in Poland could not build a simulation at all. They had two variables instead of three, did not know the current rates, had no sense of what would count as a high one, and each predicted the outcome that would have pleased him.

Mental simulation fails in a particular way, and the failure is the same capacity turned against itself: anything inconvenient can be imagined away. Klein takes the term de minimus explanation from Charles Perrow for the habit of minimising each inconsistency as it arrives. Two physicists descending the wrong side of a mountain near Aspen identify the lake below them as the one on their route, notice a dock that the lake does not have, and conclude that someone must have built it that morning.

Marvin Cohen's argument is that the process is usually self-correcting, and that explaining away mild discrepancies is often right because the evidence itself may be poor. What fails is the counting. Klein offers himself as the example: orienteering, he turned west when he meant to turn east, then accommodated a creek that was not on the map and a curving path that was supposed to be straight, until he reached a feature he could only explain by turning the map through 180 degrees. Nobody keeps a tally of what has been discounted, so the alarm does not sound on time.

Two remedies come out of this. Cohen's crystal ball asks officers for an explanation they are confident in, tells them flatly that it is wrong without saying why, and makes them produce another. Klein's premortem asks a team that has settled on a plan to imagine that it has been carried out and has failed, and to write down why. In testing it lowered confidence in the plan, which was the point.

The Vincennes shootdown

Klein interviewed Will Rogers III, who had commanded the AEGIS cruiser USS Vincennes. In April 1988, two Iranian F-4s circled toward his ship and the flagship it was escorting, and the lead aircraft switched its radar to target acquisition mode, which the rules of engagement then treated as a hostile act. Rogers did not fire. He put himself in the pilots' position — daylight, no covering force, against a ship with better radar and better missiles — could not imagine them attacking, and jammed their radars until they left.

On 3 July 1988 he used the same method and reached the opposite conclusion. The Vincennes was in a surface battle with Iranian gunboats, one of its two useful guns had fouled, and it was manoeuvring hard when Iran Air flight 655, an Airbus twenty-seven minutes behind schedule, took off from the same mixed-use airport and flew toward it. Three minutes and nine seconds passed between the track being judged significant and the missiles leaving. The cues fitted a hostile aircraft: a military transponder return, no answer to repeated radio challenges, a take-off timed to the gunboat attack, and a descent toward the ship. The airliner story had too many holes in it.

Two of the loose ends in the right-hand column were not facts. Flight 655 never squawked a military code and never descended; it climbed throughout, and the USS Sides, watching the same picture from a ship that was not under attack, read it correctly as an airliner. The transponder error has a mundane cause: a crewman kept the aircraft hooked for about ninety seconds while the range gate stayed parked over the end of the runway, so the system interrogated whatever was sitting there, and a military aircraft happened to take off at that moment.

The official Fogarty report explained the altitude error as scenario fulfilment, crew members unconsciously distorting a readout to fit a conviction they had formed, and two members of a panel of decision researchers told a congressional subcommittee that it was a plain case of expectancy bias. Klein's objection is methodological. Had the ship held fire and been hit by an F-14, the same explanation would have been available in mirror image: the crew ignored the base rates, since of 150 challenges issued in the Gulf that month, 83 percent went to Iranian military aircraft and 1.3 percent to aircraft that proved commercial. An account that fits whatever happens explains nothing.

His alternative came from Rogers. The Vincennes had labelled the airliner 4474 and the Sides had labelled the same aircraft 4131; the network merged them, kept the Sides's number and returned 4474 to the pool, from which it was reissued minutes later to an American attack aircraft several hundred miles away that was descending and accelerating. The change does not appear to have been announced clearly. Altitude was not on the large display at all: it sat as a four-digit number in a list on a small side screen with no trend shown, so that working out whether a track was climbing took, by Rogers's estimate, five to ten seconds in a room full of competing voices — and on a pitching ship a crewman was likelier to key in a track number than hook the track with a trackball. When the captain asked what 4474 was doing, the answer that came back was accurate and about a different aircraft. A later reanalysis found, in place of the bedlam the report describes, two consistent sets of reports: one matching the Airbus, one matching the other aircraft.

The power to see the invisible

Expertise, on the usual account, is accumulated knowledge, and the image it produces is of someone slowed by the weight of it. Klein's claim is the reverse: experience lightens people, by changing what they can see. He lists eight things experts see and novices do not — patterns, anomalies, the whole situation, the way things work, openings for improvisation, the past and the future of what is in front of them, differences too fine for anyone else, and their own limitations.

The most striking of these is the ability to notice an event that did not occur. Novices cannot form expectancies, so nothing can fail to happen for them. Klein gives an example from his own company: a new researcher concluded after a few months that the place ran permanently on the edge of its deadlines, having seen two documents go out close to the wire and having no way to notice the calm with which ten proposals had gone out, which was the thing that had changed. The literary case is the dog in "Silver Blaze" that did not bark. The same expectancies that a correct reading of a situation generates are the instrument that catches a wrong one.

Whether all this survives pressure was tested on chess, where skill is measured. Roberta Calderwood, Klein and Beth Crandall ran two tournaments, one among three masters and one among three class B players, each played twice under regulation time of about two minutes fifteen seconds a move and twice under blitz rules that worked out at roughly six seconds. Grandmasters rated the hard positions on a five-point scale.

The masters' move quality barely moved, and their blunder rate held at around seven or eight percent under both conditions. The class B players blundered on 11 percent of their regulation moves and 25 percent of their blitz moves. Even so, the weaker players got three moves in four right at six seconds each, which is the finding Klein keeps returning to when people assume that stress ruins decisions.

A second experiment went at the model's central claim directly. Sixteen players thought aloud over four middle-game positions, reporting every move they considered and particularly the first. Of the 124 legal moves available across the four boards, a panel of grandmasters had awarded points to twenty, so random sampling would have put about one first move in six among the good ones. Four in six of the first moves considered scored points, and the weaker players showed the effect almost as strongly as the stronger ones. To the objection that players might be generating and filtering whole option sets below awareness, Klein answers that the set to be filtered would have to include every legal move, every illegal one and every non-move, which is unbounded; something must be generating the reasonable candidates in the first place.

The last thing on the list is the expert's view of himself: knowing your memory limits, and sensing early that you are losing the thread rather than after you are lost. Klein illustrates it by cooking an unfamiliar recipe badly, unable to taste the dish in advance from its ingredients and unable to see the missing event — that it is not bubbling at the edges, and so is not done.

Stories, and the problem of hyperrationality

A story, in Klein's account, is a package of causal relationships set in a real context, which makes it a report of an experiment: under these conditions, with these causes acting at once, this happened. It buys richness at the cost of precision, since nobody controlled the conditions. A nurse in the neonatal unit sees an infant turn blue-black and the blood pressure bottom out while the monitor holds steady at eighty beats a minute, and knows at once that air has filled the sac around the heart, because she has seen it once before, in a baby who died. The team around her points at the monitor. She pushes their hands away, listens through a stethoscope, finds no heartbeat, begins compressions, and puts a syringe in the neonatologist's hand. The monitor was reporting electrical impulses, which continued. The story carries a warning about instruments, a lesson about expertise, and a piece of information about the hospital that no procedure document contains: when it is acceptable to make a scene.

Klein's interviewing technique, the critical decision method, is built on this: four passes through one incident, ending with the question of where a novice would have gone wrong and why. The nurses had nothing to say about how they detected sepsis until they were asked for particular babies.

Against this Klein sets what he calls hyperrationality, the attempt to do all one's thinking with a single source of power. He is careful not to take the anti-intellectual side: rational analysis reduces the chance of overlooking an option, comes closer to error-free work than anything else, and is a cornerstone of science and medicine. It is foveal vision — the only part of the eye that makes fine discriminations, and useless for staying oriented. His limits are structural rather than psychological. Analysis requires decomposing a situation, and there are no natural elements to decompose it into: a fire has different constituents for a firefighter, an insurance adjuster and an arson investigator. Rules take an if-then form, and the difficulty is always in deciding whether the if has been met. Setting up the calculations demands estimates of probability and value that people cannot supply. Inference chains grow combinatorially, in machines as in heads.

That leaves the question of when experience should not be trusted, and Klein answers it first by looking at his own errors. Of more than six hundred coded decision points he could identify twenty-five where the decision maker had done the wrong thing and could have known better. Sixteen came from thin experience — a commander who did not call a second alarm because he did not know that a balloon-frame building would carry fire through its supports. Some came from missing information. The rest came from mental simulation explaining away early warnings, as when a nurse assigned to a baby outside her usual care attributed a distended belly to a family trait, blood in the stool to a feeding tube, and a small aspirate to its being small, and the infant was critically ill the next day. Stress he treats as a lesser factor than it is usually given credit for: it costs cues, working memory and attention, but it does not warp the choice made on the information at hand, and it would damage a formal comparison far more than a recognition.

The serious limit is elsewhere. Expertise depends on the domain giving feedback tight enough to learn from, and Klein takes James Shanteau's conditions as the test: experience does not become expertise where the domain shifts under you, where the thing to be predicted is human behaviour, where feedback is scarce, or where there is too little repetition to build a sense of typicality. Under those conditions experience buys smooth routine and nothing more. He extends the point further than most of his field would. Legislators cannot learn the long-term effects of the laws they pass, so their experience is expertise in the procedures of the job and not in its judgments. What fills the gap where cause and effect cannot be traced is ritual — reorganisations, fashions in therapy, adjustments of diet to the latest report — and his conclusion is that superstition has changed its content and not its hold.

Since 1998

The recognition-primed model became the standard description of expert decision making under time pressure, and naturalistic decision making grew from a 1989 meeting of thirty researchers into a field with its own conference and handbooks. The critical decision method left Klein's firm and became general equipment for cognitive task analysis, set out at length in Working Minds in 2006. The U.S. Marine Corps built recognitional decision making into its planning doctrine, and tactical decision games of the kind John Schmitt designed for Klein's squad-leader course are still played. The premortem passed into ordinary management practice and was adopted by the researcher whose programme Klein had spent his career arguing against.

That argument ended in agreement. In 2009 Klein and Daniel Kahneman published the results of several years of trying to disagree, and could not: intuition is recognition, and it is worth trusting when the environment offers valid cues and the person has had a fair chance to learn them. Both conditions hold in Klein's examples and fail in most of Kahneman's, which is why the two literatures had looked incompatible for thirty years. The settlement leaves parts of this book weaker than they read in 1998. The rejection of decision biases as an explanation of field error was overstated — the conditions, not the setting, decide whether judgment is trustworthy — and the claim that formal methods are a fallback for novices does not survive in domains where the cues are weak, since there consistent rules beat consistent people. Shanteau's list, which Klein cites near the end, turned out to be the more durable part of his own answer.

Other parts have held. The 80 percent figure rests on interviews coded by the researchers who ran them, which Klein says plainly in the chapter where he asks whether his work is science; it has nonetheless been reproduced by independent groups in cockpits, on oil platforms and in army command posts, and no competing description of expert decision making under time pressure has replaced it. The neonatal cues became a training programme and then something larger: continuous monitoring of heart-rate characteristics, built to catch the same deterioration earlier than a person can, was tested in a randomised trial of very low birthweight infants and reduced mortality. The rule of three moving parts and six transitions remains what it was, a regularity in seventy-nine cases that no one has since tried to confirm or refute.

The Vincennes has not settled. Klein's reconstruction, with its recycled track number, has never displaced scenario fulfilment in the literature that cites the incident, and the case is still taught as a failure of judgment under stress rather than as a failure of a display that hid the one number that mattered. His recommendation, that altitude trend belong on the large screen, was acted on, and the wider principle he drew from it — elicit the decisions an operator has to make before building the interface — is now ordinary practice. Klein himself went on to the question the book leaves open. Having shown that experience supplies the first workable option, he spent twenty years on where a new option comes from when none of the familiar ones will do, the subject of Seeing What Others Don't in 2013, and on the argument of Streetlights and Shadows in 2009, that the procedures organisations write to keep people from erring are the same procedures that stop them becoming expert.