12Thinking, Fast and Slow

Daniel Kahneman, Farrar, Straus and Giroux, 2011

Linked to 44 chapters: supports 17 · contradicts 5 · mechanism 31 · same claim 4 · pairing 4

Kahneman's book is an account of two ways the mind produces judgments, and of what goes wrong when the fast way does the work and the slow way signs off on it without looking. He gives the two ways names, System 1 and System 2, and warns at once that they are characters in a story rather than parts of the brain. System 1 runs by itself, without effort or a sense of control, and produces impressions, intuitions and feelings continuously. System 2 is what we take ourselves to be: the deliberate, effortful reasoner that can follow rules, compare options and check an answer. The book's claim is that System 2 is lazy, that it endorses most of what System 1 hands it with little or no modification, and that the errors which follow are systematic enough to be named and predicted. The first part describes the machinery. The third part shows what happens when confidence is built on it: an illusion of understanding the past, an illusion of validity in predicting the future, and a settled answer to the question of when an expert's intuition can be trusted. The fourth part turns to choices, and to the discovery that people evaluate outcomes as gains and losses from a reference point rather than as states of wealth, with losses weighing about twice as much as gains. The book was written, he says at the end, for critics and gossipers rather than decision makers, because it is far easier to see someone else walking into a minefield than to notice you are about to.

Part I: two systems

The book opens with a photograph of an angry woman and the sum 17 × 24. You know the woman is angry before you have decided to look; you know the sum is a multiplication you could do, and you do not do it without deciding to. That is the whole distinction. System 1 operates automatically and quickly, with little or no effort and no sense of voluntary control. System 2 allocates attention to effortful mental activities, including complex computations, and is associated with the experience of agency, choice and concentration. When System 2 is engaged the pupils dilate, and Kahneman spent years using pupil size as a gauge of mental effort: holding four digits and adding one to each is hard, adding three is near the limit, and at the peak of effort people go effectively blind to what they are looking at. Attention is a budget, and there is a law of least effort: given several ways to reach the same goal, people gravitate to the least demanding.

The Müller-Lyer lines, which the reader has met in chapter 2, are his model of the relation between the two. You can measure the lines, know they are equal, and still see one as longer; the knowledge does not change the impression, it only lets you distrust it. Cognitive illusions work the same way. Continuous vigilance against them is impossible and would be tedious, so the best that can be done is to learn which situations are dangerous and slow down there.

System 2 is not only effortful but lazy, and the third chapter tests this directly. A bat and a ball cost $1.10 together and the bat costs a dollar more than the ball; the intuitive answer, ten cents, is wrong, and more than half the students at Harvard, MIT and Princeton gave it. Those who did had not checked an answer that a few seconds of effort would have rejected; they had also missed the social cue that nobody sets a puzzle that easy. People who fail such questions are also more likely to accept other suggestions from System 1: to prefer a smaller sum now to a larger sum next month, to pay more for overnight delivery. The chapter's account of self-control as a depletable resource, restored by glucose, and its study of parole judges whose approvals fell towards zero before each meal, are the parts of the book that have fared worst since, and the closing section returns to them.

Chapter 4 describes the associative machine. Reading the words "bananas" and "vomit" together produces, in a second, a bodily reaction, a causal story and a readiness for related words, without any instruction to do so; ideas evoked trigger other ideas in a spreading cascade, and the cascade's defining property is coherence. Priming is the experimental handle on this: exposure to a word or an idea changes the ease with which related ones come to mind, and, in the studies Kahneman reports, changes walking speed, generosity and votes. His verdict on those studies, that disbelief is not an option, is the strongest unhedged claim in the book, and the one he has since withdrawn.

Chapter 5 is where the machinery meets evaluation. System 1 keeps a running dial between cognitive ease and strain. Ease means no threat and no need for effort; strain summons System 2. The trouble is that the dial has many inputs and they are interchangeable: a clear typeface, a repeated statement, a rhyme, a good mood and a pronounceable name all turn it the same way, and none of them is traceable afterwards. So familiarity is felt as truth. Repeat a false statement and it gains credence; print a claim in a legible font and it is believed more than the same claim in a faint one; a rhyming aphorism is judged more insightful than the same thought unrhymed. The reverse holds too. Given the bat-and-ball questions in a washed-out grey font, students made fewer errors, because strain had woken System 2. Kahneman's own driving test is the confession at the centre of the chapter: if the answer felt familiar, he assumed it was probably true.

Chapter 6 gives System 1 its main job, keeping a model of what is normal and registering surprise; one repetition is enough to make an odd event feel ordinary. Norms are also applied backwards: two headlines a day apart, treasuries rising because a capture may not curb terrorism and treasuries falling because the capture boosts risky assets, both read as explanations, and a statement that can explain two contradictory outcomes explains nothing. We see causes as directly as we see colour, and that appetite is dangerous where statistics are required, because System 1 cannot think statistically.

Chapter 7 names the consequences. System 1 resolves ambiguity by betting on one reading and keeps no record of the alternatives; doubt is System 2's work, and a busy System 2 leaves us credulous. Beliefs we already hold recruit supporting arguments, sound or not. Liking one thing about a person makes us like everything about them, the halo effect, and Kahneman's remedy, learned grading essays, is to decorrelate error: score each question across all papers before moving to the next, keep witnesses apart, have committee members write their positions down before the discussion. Then the principle he shortens to WYSIATI, what you see is all there is. System 1 works only with the evidence in front of it, and the measure of its success is the coherence of the story it can build, not the completeness of the information. That is why confidence is so poor a signal. In one experiment participants heard only one side of a legal case, knew they had heard only one side, and were more confident in their judgment than those who heard both. Knowing little makes it easier to fit everything into a pattern.

Chapters 8 and 9 supply the last mechanism. System 1 continuously produces basic assessments, of threat, mood, similarity, of whether a face looks competent, and these stand ready to be used as answers to questions that were never asked. A rating of competence from a face seen for a tenth of a second predicted the winners of about seventy per cent of a set of American elections, most strongly among voters who watched a lot of television and knew little else. When a hard question arrives, the target question, and no answer comes quickly, System 1 answers an easier one, the heuristic question, and intensity matching fits the answer onto the scale of the original. How happy are you with your life becomes what is my mood right now; will this candidate succeed becomes does she interview well. System 2 has the chance to reject the substitute and usually endorses it, and you may not notice that the question you answered was not the one you were asked.

Part II, which this chapter passes over, is the catalogue that grew from that mechanism: anchors, availability, representativeness, the law of small numbers, regression to the mean and the taming of intuitive prediction. It is the material of the 1974 paper with Amos Tversky that made the field.

Part III: overconfidence

Part III turns from mechanism to consequence. Chapter 19 is about the past. We make sense of what happened with simple stories that credit talent and intention and leave out luck and the things that did not happen; a good story creates an illusion of inevitability. Because we cannot recover what we believed before, we underestimate how surprised we were, and hindsight then corrupts evaluation: a decision is judged by its outcome rather than by whether the process was sound, so a physician who took a reasonable risk that failed is blamed and a lucky gambler is praised. The business books that find the secrets of successful firms are stories of this kind, and the firms they praise regress towards the mean afterwards.

Chapter 20 is about the future, and it begins with Kahneman's own failure. As a young officer he assessed candidates for leadership by watching them haul a log over a wall, formed vivid and confident impressions, and learned from feedback that the forecasts were nearly worthless. Knowing this changed neither the confidence nor the practice. He named the experience the illusion of validity, and the chapter finds it running an industry: stock pickers whose selections do worse than the stocks they sold, fund managers whose year-to-year rankings are uncorrelated, twenty-five wealth advisers whose pairwise consistency over eight years averaged a correlation of .01, and political forecasters who, in Tetlock's study of eighty thousand predictions, did worse than assigning equal odds. The question, he concludes, is not whether the experts are well trained but whether their world is predictable. And the lesson for anyone receiving a judgment is stated plainly: a declaration of high confidence mainly tells you that the person has built a coherent story, not that the story is true.

Chapter 21 gives the alternative. In the 1950s Paul Meehl showed that simple statistical rules matched or beat trained clinicians at predicting outcomes, and some two hundred studies since have not reversed the score. Experts lose for two reasons: they try to be clever with complex combinations of cues, and they are inconsistent, giving different answers to the same case on different days. A formula never varies, and a formula with equal weights on a few sensible variables does nearly as well as one fitted by regression; Virginia Apgar's five-variable score for newborns is the model. Kahneman's own contribution was to redesign the Israeli army's interview so that six traits were scored separately and in order before any overall judgment was allowed, and to discover that a final intuitive judgment, made after the disciplined scoring, added value. The rule that came out of it is to trust intuition only after the objective information has been collected and scored, and, when you make the final decision yourself, to resist the wish to invent reasons for overriding the formula.

Chapter 22 is the one the reader is asked to read closely, and it is a report of an argument. Gary Klein, the leading student of expert decision making in the field, distrusted the biases programme and admired the firefighters and nurses he studied; Kahneman admired Meehl and had spent a career on the failures of clinicians and forecasters. They agreed to write together on one question: when can you trust an experienced professional who claims to have an intuition? They agreed at once on what intuition is. In Herbert Simon's account, which both accept, the situation provides a cue, the cue gives access to something stored in memory, and the memory supplies the answer.

Intuition is nothing more and nothing less than recognition.

They agreed too on how it is acquired: slowly, as a large collection of small skills, ten thousand hours for a chess master. The disagreement, it turned out, was about which experts each had in mind. Klein's fireground commanders and Kahneman's stock pickers are both confident, and the confidence tells you nothing. What does is the provenance of the intuition, and they settled on two conditions: an environment regular enough to be predictable, and an adequate opportunity to learn its regularities through prolonged practice with feedback that is quick and unequivocal. Chess, firefighting and anaesthesia meet both. Long-term political forecasting and stock picking meet neither, and there, intuitive hits are due either to luck or to lies. In between are the wicked environments in which practice teaches the wrong lesson, like the physician who felt patients' tongues for typhoid and, by not washing his hands, developed a sense of infallibility. The paper they wrote was subtitled "a failure to disagree"; what remained was temperament, Klein's default of trust and Kahneman's of scepticism.

Intuition cannot be trusted in the absence of stable regularities in the environment.

Chapter 23 applies the same idea to plans. Kahneman's own team, writing a textbook on judgment for schools, estimated two years to finish; asked for the record of comparable teams, their one member who knew it reported that forty per cent had never finished and the rest had taken seven to ten years, and rated the team slightly below average. Everyone noted this and carried on; the book took eight years and was never used. The inside view forecasts from the case in hand and cannot allow for what it does not know; the outside view asks for the statistics of the reference class and adjusts from there. The Scottish Parliament building was estimated at forty million pounds and cost about four hundred and thirty; rail projects worldwide overestimated passengers by an average of a hundred and six per cent over three decades with no improvement. The cure is a procedure, reference class forecasting, not a resolve to be realistic.

Chapter 24 asks why the fallacy is so durable and answers that it is one face of a general optimism, which he calls possibly the most significant of the biases for its consequences. Entrepreneurs put their own odds of success far above the base rate for businesses like theirs; inventors given the lowest possible grade for their idea continued anyway and on average doubled their losses; chief financial officers' forecasts of the stock market correlated slightly less than zero with what happened, and their eighty per cent confidence intervals were surprised two-thirds of the time, more than three times as often as they should have been. Two mechanisms do much of the work, substituting an absolute judgment for a comparative one, and neglecting competitors' plans. And a social one keeps it in place: confident forecasters are rewarded and honest ones are laughed out of the room, so acting on pretended knowledge is often preferred. His remedy for organisations is Klein's premortem: before committing, the group writes the short history of how the plan failed a year on.

Part IV: choices

Part IV is the work that won the Nobel prize, and Kahneman tells it as a correction of a three-hundred-year-old error. Bernoulli explained why people refuse fair gambles by proposing that money has diminishing utility, so that the pain of losing outweighs the pleasure of an equal gain. The account has no reference point. It cannot tell the difference between two people who both have five million today when one had one million yesterday and the other nine, and it cannot explain why people become risk-seeking when all their options are bad. Nobody noticed for three centuries, which he calls theory-induced blindness: once a theory is accepted, its flaws hide in what it ignores.

Prospect theory, built with Tversky, puts the reference point back. Outcomes are evaluated as gains and losses from where you stand, usually the status quo, sometimes an expectation or an entitlement. Sensitivity diminishes in both directions, so the difference between a hundred and two hundred dollars feels larger than between nine hundred and a thousand. And losses loom larger than gains: the smallest gain that makes people willing to accept an even chance of losing a hundred dollars is about two hundred, and the ratio runs between one and a half and two and a half across studies. Together these give the value function its shape, concave for gains, convex for losses, with a kink at the reference point where the loss side falls away more steeply. Kahneman says that if prospect theory had a flag, this curve would be on it.

The consequences fill the rest of the part. The endowment effect: people who have just been given a mug demand about twice as much to sell it as others will pay to buy it, though not for goods held for exchange, and not among experienced traders. Loss aversion as a conservative force: the poor live below their reference point, so every choice is between losses; reforms create a few losers who fight harder than the many winners; golfers sink more putts for par than for birdie because a bogey is a loss. The fourfold pattern: because small probabilities are overweighted and near-certainties underweighted, people are risk-averse for likely gains and unlikely losses, and risk-seeking for unlikely gains and likely losses, which is why lotteries and insurance are both sold, and why a losing side fights on past the point of sense.

Chapters 30 to 34 extend the picture. Rare events are overweighted when they are vivid and imaginable, and a risk described as "one person in a thousand" moves people more than the same risk as a tenth of a per cent. Narrow framing, deciding each case on its own, costs money that broad framing saves; the remedy is a standing policy for a class of risks and a refusal to look at the score too often. Mental accounts keep score, and closing one at a loss is painful enough that people hold losing stocks, finish losing projects and drive into blizzards because the tickets were paid for. Regret, the punishment we administer to ourselves for departing from the default, biases choice towards convention. Cases judged side by side are judged by a rule; cases judged alone are judged by feeling, and the two disagree. And framing: ninety per cent survival and ten per cent mortality describe the same operation, and physicians chose surgery far more often under the first description, which shows, he says, that our moral intuitions are about descriptions rather than substance, and that medical training is no defence.

Conclusions

The conclusions restate the three pairs, the two selves, Econs and Humans, and the two systems, and end on the last. System 1 is the origin of most of what we do right as well as wrong; skill grows where the environment is regular, practice is adequate and feedback is fast. The trouble is that System 1 gives no warning when it moves from skill to heuristic. An intuitive answer arrives with the same speed and confidence whether it comes from expertise or from substitution, and the only way System 2 can tell the difference is to slow down and build the answer itself, which it is reluctant to do. Kahneman reports that decades of study have not made his own intuitive thinking less prone to overconfidence or the planning fallacy; what he has gained is the ability to recognise situations where errors are likely, and much more progress in recognising others' errors than his own. Organisations can do better than individuals because they think slowly and can impose procedures, and a shared vocabulary for judgment errors is the most practical lever he offers. People make better decisions, he ends, when they expect to be judged by how the decision was made and not only by how it turned out.

Since 2011

The book's standing has changed unevenly since it appeared, and the reader should know where.

The fourth chapter has not survived. The priming studies Kahneman reported, elderly words slowing walkers, money making people less helpful, eyes on a poster raising honesty-box takings, largely failed to replicate when tried with larger samples, and the ego-depletion results of chapter 3, self-control as a resource that glucose restores, failed a large multi-site replication in 2016. Kahneman has said in print that he placed too much faith in underpowered studies and that the chapter's "disbelief is not an option" was a mistake. The irony he would appreciate is that the book's own Introduction describes the error, experts exaggerating what small samples can show, and that its remedy, the outside view, would have caught it.

The core has held. The two-system description remains a useful fiction, as he called it. The catalogue of heuristics and biases, prospect theory, loss aversion and the fourfold pattern have been replicated across countries and decades. Meehl's finding that simple rules beat experts in low-validity environments has only strengthened. And the Kahneman–Klein conditions for trusting an intuition, regularity plus feedback, remain the most usable thing in the book, precisely because they ask nothing about the expert's confidence.

Two limits are the book's own. Kahneman is candid that individual debiasing barely works, that his prescriptions are for organisations and critics rather than for the person in the moment, and that the slow-down remedy is least likely to be applied when it is most needed. And the book says almost nothing about how to acquire good judgment, only how to recognise its absence; for that, Klein's side of the collaboration and the literature on deliberate practice have to be read with it.