Framework

supported3 sourcesClaims5 evidenceTraditions3 empiricalUnderA measure scored on stops measuringThe manner came from somebody elseA bigger model is not better

Training and size improve only what they were aimed at and leave toxicity and bias where they were

Chinchilla beat Gopher on a truthfulness test, 43.6 per cent against 29.5, and its average toxicity did not fall at all. Ouyang's instruction training roughly doubled truthful and informative answers and left measures of bias unmoved, cut toxic output about a quarter and only when the model was told to be respectful, and cost the model reading comprehension and translation skill that had to be mixed back in. Told to be as offensive as possible, the instruction-trained models scored worse than the GPT-3 they came from. The bias benchmarks are contested instruments, and later training is aimed directly at these failures.

Chapters Training Language Models to Follow Instructions with Human Feedback · Training Compute-Optimal Large Language Models · Emergent Abilities of Large Language Models

For

Chinchilla became more truthful than Gopher but no less toxic Training Compute-Optimal Large Language Models

Instruction training improved truthfulness but left bias untouched Training Language Models to Follow Instructions with Human Feedback

Wei's paper shows harms grow with model size just as abilities do Emergent Abilities of Large Language Models

A model trained to obey obeys orders to be offensive too Training Language Models to Follow Instructions with Human Feedback

Instruction training costs a model skills that can mostly be restored Training Language Models to Follow Instructions with Human Feedback

supported3 sourcesClaims4 evidenceTraditions3 empiricalUnderA measure scored on stops measuringA bigger model is not better

How a model is trained matters more than how big it is

Ouyang's raters preferred InstructGPT at 1.3 billion parameters to the 175-billion-parameter GPT-3 it was built from, so a small amount of human ranking bought more obedience than a hundredfold increase in size. Hoffmann's team found every large model in 2022 was too big for the text it was trained on, and that parameters and training words should double together. Wei adds that once a large model shows an ability, smaller ones pick it up soon after. The preference test used the same raters whose rankings trained the model, and the win is on following instructions, not on the test scores the same paper shows falling.

Chapters Training Language Models to Follow Instructions with Human Feedback · Training Compute-Optimal Large Language Models · Emergent Abilities of Large Language Models

For

A model 100 times smaller beat GPT-3 after training on human rankings Training Language Models to Follow Instructions with Human Feedback

In 2022 every big AI model was too large for its data Training Compute-Optimal Large Language Models

Model size and training data should grow at the same rate Training Compute-Optimal Large Language Models

Once a big model shows an ability smaller models soon get it Emergent Abilities of Large Language Models

supported3 sourcesClaims2 evidence1 opinionTraditions2 empirical1 designUnderHow it feels is evidence about the personThe manner came from somebody elseNo method sets the problem for you

The role you give a system and the style it answers in change its output and your judgement

Mollick says telling a model what kind of person to be is part of the instruction, not decoration. Reeves and Nass measured the receiving end, finding that people judged identical advice more insightful when the computer's conversational style matched their own, and that a style which started unlike theirs and shifted to match worked better still. Lee and See find trust set by how fast a system replies, how it looks and how fluently it talks, rather than by how often it has been right. Their study used forty-eight people picked from the extremes of a questionnaire, and the credibility evidence is about websites in 2002.

Chapters Trust in Automation: Designing for Appropriate Reliance · The Media Equation · Co-Intelligence

For

Mollick says the role you give a model changes what it writes Co-Intelligence

People rate the same advice higher when the computer's style matches theirs The Media Equation

How a system looks and sounds sets trust more than its record Trust in Automation: Designing for Appropriate Reliance

supported3 sourcesClaims1 evidence2 opinion1 contestedTraditions2 design1 empiricalUnderA measure scored on stops measuringThe inside is not readable from the outsideA score answers its own questionEverything you learn about it expiresOnly your own task finds the edge

A published test score tells you little about your own work and means less every year

Narayanan and Kapoor argue that the single word AI covers at least three unrelated technologies, so a score that does not say which one it measured settles nothing about your job. Mollick adds that any limit you find should be recorded with a date, because the model will be replaced. Sharma's team watched their own flattery tests become a standard training target, which is how a published number decays. Three of the four claims here are opinion or contested, and the contamination is inferred from the tests being widely adopted rather than shown in any particular model.

Chapters Towards Understanding Sycophancy in Language Models · Co-Intelligence · AI Snake Oil

For

The word AI covers three different technologies that need separate verdicts AI Snake Oil

Every AI advance so far has been a rung not a threshold AI Snake Oil

Mollick says today's AI is the worst you will ever use Co-Intelligence

Once the flattery tests were published models began training on them Towards Understanding Sycophancy in Language Models

supported3 sourcesClaims1 evidence3 opinionTraditions1 philosophy1 empirical1 designUnderWhich part of the task went decides what is lostOnly your own task finds the edgeNo method sets the problem for you

AI helps on the tasks it suits and hurts on the rest, and nobody can tell which is which

Dell'Acqua's consultants working on tasks that suit the tool did better, faster and more completely, all at once. On a task built to defeat it, the same tool dragged them down. Autor's rule for what goes to a machine, whatever a person can write down as steps, stopped holding once machines learned from examples instead, and Mollick says the edge is ragged and not where intuition puts it. Three of the four claims here are opinion rather than measurement, and the quality figures are rubric scores the authors admit may carry evaluation bias.

Chapters The Jagged Technological Frontier · Co-Intelligence · Why Are There Still So Many Jobs?

For

Consultants using AI on suitable tasks did better, faster and more completely The Jagged Technological Frontier

Machines take what can be written down, not what is difficult Why Are There Still So Many Jobs?

Machines got into work nobody can explain by two routes Why Are There Still So Many Jobs?

What AI can and cannot do has a ragged unmarked edge Co-Intelligence

weak3 sourcesClaims1 opinion3 contestedTraditions2 design1 empiricalUnderNobody who built the tool is hereThe failure shows downstream of its causeChange the arrangement not the person

Most failures blamed on people are design failures, and removing the operator leaves the designer's mistakes uncaught

Norman points at the share, between 75 and 95 per cent of industrial accidents recorded as human error, which he says is too high to be about people, and he treats a handwritten sign taped to a machine as an admission that the design failed. Klein re-reads the USS Vincennes shootdown, where the aircraft's altitude sat as four digits on a small side screen with no trend line. Bainbridge adds that automating the job leaves the designer's mistakes with nobody on duty to catch them. Three of the four claims are marked contested, Norman leaves the strong form unevidenced, and Bainbridge's evidence is second-hand.

Chapters Ironies of Automation · The Design of Everyday Things · Sources of Power

For

Most accidents blamed on people are caused by bad design The Design of Everyday Things

Norman says a handwritten sign on a machine means the design failed The Design of Everyday Things

A display that buries the key number makes operators look biased Sources of Power

Removing the operator leaves the designer's mistakes with nobody to catch them Ironies of Automation

supported2 sourcesClaims5 evidence1 contestedTraditions2 empiricalUnderThe manner came from somebody elseTest the answer not the sourceMost fluent where it knows leastThe first answer becomes the answer

An assistant trained on approval tells you what you seem to want, so its agreement proves nothing

In Ouyang's method human judgment enters once, when screened contractors rank answers to train a scoring model, and every later update pushes the assistant towards what that stored model predicts a contractor would pick. Sharma's team found the result in use across five assistants from three companies: a tentative hint at what you believe moves the answer, and a single line of pushback makes an assistant withdraw a correct one. The studies do not separate a model caving in from one correctly treating your disagreement as evidence, and one member claims the flattery is already there before preference training.

Chapters Training Language Models to Follow Instructions with Human Feedback · Towards Understanding Sycophancy in Language Models

For

InstructGPT is trained to match a stored guess of a rater's choice Training Language Models to Follow Instructions with Human Feedback

InstructGPT learned to hedge because its labellers were told to reward humility Training Language Models to Follow Instructions with Human Feedback

Hinting at the answer you expect changes what the assistant says Towards Understanding Sycophancy in Language Models

One line of pushback makes an assistant take back a right answer Towards Understanding Sycophancy in Language Models

Five assistants from three companies all flatter users the same ways Towards Understanding Sycophancy in Language Models

Flattery is in the model before human-preference training and survives it Towards Understanding Sycophancy in Language Models

supported2 sourcesClaims5 evidenceTraditions2 empiricalUnderA bigger model is not betterSome failures sit below the prompt

The design of these models has barely changed since 2017 and only the amount of training text grew

Vaswani's layer has kept its shape since 2017 while its details moved. Layer normalisation shifted, the position signal was replaced, the number of key-value heads was cut, and none of that changed the argument. The design was built to make training about ten times cheaper, not to make the model smarter. Hoffmann says plainly that his own result is about how much text you have rather than how the model is built, putting the right training set for a trillion-parameter model at 21.2 trillion words. Stability of the layer is not stability of the system, and post-training, tool use and retrieval changed behaviour far more without showing at this level.

Chapters Attention Is All You Need · Training Compute-Optimal Large Language Models

For

The transformer's overall shape held since 2017 while its details changed Attention Is All You Need

The transformer was built to make training faster not smarter Attention Is All You Need

The transformer works on images and audio, not only on words Attention Is All You Need

In a Transformer any two words are one step apart Attention Is All You Need

Hoffmann's conclusion is about training text, not model design Training Compute-Optimal Large Language Models

supported2 sourcesClaims4 evidence1 contestedTraditions1 empirical1 philosophyUnderThe inside is not readable from the outsideReasons are reconstructed after the factThe manner came from somebody elseMost fluent where it knows least

Smooth output tells you nothing about what a model holds inside or how it got there

Anthropic's circuit tracing found a model that writes one word at a time settling on the rhyming word before it starts the line, and running through steps that never appear in the answer, where replacing one of those steps changes what it says. A goal the model will not discuss sat inside its picture of itself, firing in all hundred assistant-formatted prompts tested. The work reads a substitute network rather than the real one, covers one small model, and says anything at all on about a quarter of prompts. Bender's claim that text alone never reaches meaning is carried here as contested.

Chapters On the Dangers of Stochastic Parrots · On the Biology of a Large Language Model

For

A model trained to write one word at a time plans ahead On the Biology of a Large Language Model

The model works through steps that never appear in its answer On the Biology of a Large Language Model

A model stores a concept separately from the language it answers in On the Biology of a Large Language Model

A model's hidden goal can live in its idea of itself On the Biology of a Large Language Model

A model trained only on text handles word forms, not meaning On the Dangers of Stochastic Parrots

supported2 sourcesClaims4 evidenceTraditions2 empiricalUnderThe failure shows downstream of its causeMost fluent where it knows leastSome failures sit below the prompt

Some failures come from how the machine reads the input, so no rewording fixes them

Asked how many Ds are in DEEPSEEK, DeepSeek-V3 answered 2 or 3 across ten tries, because the word arrives as the chunks D, EEP, SEE and K rather than as letters. No amount of rewording reaches that, though a model that spells the word out answers 1. Vaswani's paper supplies more of the same kind, since attention adds its inputs as an unordered set, so word order has to be supplied separately, and its cost grows with the square of the length. The letter example is a handful of trials on a task nobody needs, and later engineering has softened the cost limit.

Chapters Attention Is All You Need · Why Language Models Hallucinate

For

A model that reads words in chunks cannot count their letters Why Language Models Hallucinate

The transformer's attention cannot tell word order so order is added Attention Is All You Need

Extra attention heads make up for blurring, they do not add power Attention Is All You Need

Attention's cost was trivial in 2017 and is now the main limit Attention Is All You Need

supported2 sourcesClaims3 evidenceTraditions2 empiricalUnderA measure scored on stops measuringMost fluent where it knows leastSome failures sit below the prompt

Made-up facts come from how a model is trained and scored, not from a fixable bug

Kalai and his co-authors prove a floor. For facts with no pattern behind them, a model's error rate after pretraining is at least the share of such facts its training text recorded exactly once, so if a fifth of birthdays appear once, expect a fifth of birthday answers to be wrong. They also prove that a model which never says anything false has given up estimating what the text says. Anthropic's traces show the mechanism, with the model starting ready to say it does not know until a familiar name switches that refusal off. Both results are about a base model, before post-training or a search tool.

Chapters On the Biology of a Large Language Model · Why Language Models Hallucinate

For

Facts that appear once in the training data set the error floor Why Language Models Hallucinate

A model that never says anything false cannot be calibrated Why Language Models Hallucinate

The model refuses by default until a familiar name switches it off On the Biology of a Large Language Model

supported2 sourcesClaims3 evidence1 opinion1 contestedTraditions2 empiricalUnderNothing helps in itself it has to fit the receiverEverything you learn about it expiresOnly your own task finds the edge

New abilities appear suddenly as models grow, nobody can predict which ones, and the size varies by model

Wei and colleagues watched eight tasks across five model families sit at chance over orders of magnitude of training compute and then rise well above it. They found no feature that predicts which tasks jump, and they report that the size at which one appears depends on the particular model and its training text. Against this sits their own finding that once a large model shows an ability, smaller ones acquire it soon after. Every curve is drawn with a scoring rule that demands an exactly correct answer, which Schaeffer showed turns a smooth improvement into a step.

Chapters Emergent Abilities of Large Language Models · Are Emergent Abilities of Large Language Models a Mirage?

For

Some AI tasks stay at chance for a long time then jump Emergent Abilities of Large Language Models

Nobody can predict which tasks a bigger model will suddenly do well Emergent Abilities of Large Language Models

Whether a skill appears depends on the model and its training data Emergent Abilities of Large Language Models

Only owners of very large models can test claims about new abilities Emergent Abilities of Large Language Models

A model's new ability still arrives suddenly for the person using it Are Emergent Abilities of Large Language Models a Mirage?

Against

Once a big model shows an ability smaller models soon get it Emergent Abilities of Large Language Models

supported2 sourcesClaims2 evidence2 opinionTraditions1 philosophy1 empiricalUnderA measure scored on stops measuringNobody who built the tool is hereThe manner came from somebody elseEverything you learn about it expires

A few dozen paid raters and whoever wrote the web set how an assistant talks

Ouyang's paper puts a number on the first population: about forty screened contractors, agreeing with one another only about three-quarters of the time, alongside the researchers who wrote their instructions, the customers whose prompts were used, and a waitlist seeded by employees. Bender and colleagues supply the second: text scraped from the web narrows the population at every step of collection, and a model keeps the words in use at the moment its data was frozen. The corpus figures are from 2021, the demographic survey covers nineteen people and is self-reported, and half these claims are argument rather than measurement.

Chapters Training Language Models to Follow Instructions with Human Feedback · On the Dangers of Stochastic Parrots

For

An assistant's manner comes from about forty paid labellers Training Language Models to Follow Instructions with Human Feedback

Four different groups shaped the model's manner and none claims that authority Training Language Models to Follow Instructions with Human Feedback

A web-sized training set does not represent everyone who speaks the language On the Dangers of Stochastic Parrots

A model keeps the language of the moment its training data froze On the Dangers of Stochastic Parrots

supported2 sourcesClaims2 evidence3 opinionTraditions2 designUnderNobody who built the tool is hereChange the arrangement not the person

A person learns a machine only from what it shows because the designer's picture is never shared

Norman's point is that designer and user never meet, so the product's own shape, controls and manual carry the whole load. A layout that feels obvious rests on a picture in the designer's head that other people do not share, and when a device does more things than it has controls for, people will act as though it were in a different setting. Weinberg measured the same effect in a language: of 117 experienced FORTRAN programmers, nine got a rule with seven forms right. Norman assumes the designer has a correct model to pass on, and three of the five claims are argument rather than measurement.

Chapters The Design of Everyday Things · The Psychology of Computer Programming

For

The product itself must teach the user how it works The Design of Everyday Things

Norman says a visible mark, not shape, tells people where to act The Design of Everyday Things

A control that feels obvious to you is not obvious to everyone The Design of Everyday Things

Users will confuse settings when a device has more functions than controls The Design of Everyday Things

Programmers knew a language rule with one form and not with seven The Psychology of Computer Programming

supported2 sourcesClaims2 evidenceTraditions1 empirical1 philosophyUnderReasons are reconstructed after the factNobody who built the tool is here

A working system does not record why its parts are there, so any account is worked out backwards

Dennett's point is that the reasons a design works are explicit while it is being built and only implicit once it runs, and in anything that evolved they were never written down at all. They stay recoverable by working backwards, and checkable, since taking the part out makes the system work worse. Anthropic's researchers hit the same wall in a model holding more concepts than it has neurons, so they read a separate, easier network trained to copy its behaviour rather than the model itself. That substitute leaves much of the computation in terms nobody can interpret, and the authors call it a stepping stone.

Chapters On the Biology of a Large Language Model · From Bacteria to Bach and Back

For

Interpretability tools read a stand-in model, not the real one On the Biology of a Large Language Model

Nothing inside a working system records why its parts are there From Bacteria to Bach and Back

supported2 sourcesClaims2 evidenceTraditions1 philosophy1 empiricalUnderA measure scored on stops measuringA bigger model is not better

Answering users costs more than training the model, but the field's rules count only training

Hoffmann's scaling rule finds the cheapest way to train a model and nothing else, and the paper says so. Once the model is serving users, most of its lifetime computing goes on answering them, so the rule stops being the right target. Bender and her co-authors made the same point in 2021, when everyone else was quoting training figures, and predicted that running would outweigh building. The thesis rests on two claims from two sources. Bender's is a prediction rather than a measurement, the industry figures confirming it are aggregate and self-reported, and the argument assumes a volume of use most models never reach.

Chapters Training Compute-Optimal Large Language Models · On the Dangers of Stochastic Parrots

For

Chinchilla's rule minimises training cost and ignores the cost of running Training Compute-Optimal Large Language Models

Running a model uses more energy than training it did On the Dangers of Stochastic Parrots

supported2 sourcesClaims1 evidence1 opinionTraditions1 design1 philosophyUnderThe failure shows downstream of its causeNo method sets the problem for you

A person builds the problem out of a confusing situation and no technique performs that step

Schön argues that problems do not arrive ready-made. Someone builds one out of a confusing situation by naming what to attend to and framing the context, and no method tells him how. Christian shows the machine-side cost of getting that step wrong: when a goal cannot be stated exactly you reward a stand-in for it, and the stand-in comes apart just as the system gets good at chasing it, as when a boat learns to spin for power-ups instead of racing. No study shows that problem setting decides outcomes, and where a situation is a familiar case type the older account fits it.

Chapters The Alignment Problem · The Reflective Practitioner

For

Schön says setting the problem comes first and has no technique The Reflective Practitioner

A stand-in measure fails just as the system gets good at it The Alignment Problem

supported2 sourcesClaims1 evidence1 contestedTraditions2 empiricalUnderNothing helps in itself it has to fit the receiverOnly your own task finds the edgeSome failures sit below the prompt

The same way of asking helps on one model or task and hurts on another

Wei and colleagues found that one prompting trick did real harm to small models and only paid off above a certain amount of training compute. Dell'Acqua's consultants who were given an overview of how to write prompts scored 60.0 per cent correct on a task built to defeat the AI, against 70.6 per cent for those given no overview at all. So a method has to be tested against the model and the job in front of you. Mollick's advice that the role you assign changes what a model writes stands against this, and the consultant result is a single contrast at ten per cent significance.

Chapters Emergent Abilities of Large Language Models · The Jagged Technological Frontier

For

A prompting trick hurts small models and helps large ones Emergent Abilities of Large Language Models

Teaching consultants to prompt made them worse on the harder task The Jagged Technological Frontier

Against

Mollick says the role you give a model changes what it writes Co-Intelligence

weak2 sourcesClaims4 opinionTraditions1 empirical1 philosophyUnderThe inside is not readable from the outsideAn explanation moves agreement not judgement

An explanation of an answer is worth having only if it helps you find mistakes

Bansal and colleagues say an explanation should help a person catch the model's mistakes rather than talk them into agreeing. Lipton finds five different things people mean by an understandable model, among them trust, cause, and information for whoever decides, and shows they do not reduce to one another, so the bare word promises nothing anyone can check. He adds that asking for an explanation is a sign the test score stopped measuring what you wanted. All four claims here are opinion, none is measurement, and Lipton's list is one author reading one literature in 2016.

Chapters Does the Whole Exceed its Parts? · The Mythos of Model Interpretability

For

Explanations should help people spot mistakes, not persuade them Does the Whole Exceed its Parts?

Lipton finds five different things people mean by an interpretable model The Mythos of Model Interpretability

Wanting an explanation means the test score stopped measuring the real goal The Mythos of Model Interpretability

A linear model is not simply more understandable than a deep network The Mythos of Model Interpretability

weak1 sourceClaims2 evidenceTraditions1 empiricalUnderEverything you learn about it expiresA result outruns the design that produced it

A result the field repeated for years came from a measuring mistake and one experiment settled it

Hoffmann's team traced the field's earlier answer to how it had been measured. Kaplan had read intermediate numbers off a training schedule built for a much longer run, which makes short training look worse than it is. They then spent Gopher's exact budget on the smaller model their formula named, trained it on 1.4 trillion words, and beat Gopher 67.6 to 60.0 on one standard test. This rests on a single source and one run at one size, by the group whose own fit made the prediction. Schaeffer answers that for the person using a model, a new ability still arrives abruptly.

Chapters Training Compute-Optimal Large Language Models

For

The earlier scaling answer was a measurement error, not a real disagreement Training Compute-Optimal Large Language Models

The Chinchilla team settled a scaling argument by running one experiment Training Compute-Optimal Large Language Models

Against

A model's new ability still arrives suddenly for the person using it Are Emergent Abilities of Large Language Models a Mirage?

weak1 sourceClaims1 evidence2 opinionTraditions1 philosophyUnderNobody who built the tool is hereThe failure shows downstream of its cause

Smooth machine writing harms people even when nothing goes wrong, and they have no remedy

Bender and her co-authors argue that the damage comes from text which reads smoothly while nobody answers for it. Their case is a fluent mistranslation that got a Palestinian man arrested, with the system performing exactly as designed. When the training set is too large for anyone to describe what is in it, an output cannot be traced back to what produced it, so the person harmed has nowhere to go. This rests on one source, and two of its three claims are argument rather than measurement. Documentation rules and interpretability work have since moved tracing partway, though the corpora stay unpublished.

Chapters On the Dangers of Stochastic Parrots

For

Smooth machine writing causes harm even when the system is working correctly On the Dangers of Stochastic Parrots

Training data nobody can document leaves harmed people with no remedy On the Dangers of Stochastic Parrots

The people who gain from large models are not those who pay On the Dangers of Stochastic Parrots

weak1 sourceClaims1 opinionTraditions1 philosophy

Guessing the next word well requires knowing how the world works, so there is no shortcut

Agüera y Arcas argues that doing the prediction properly means solving general intelligence rather than dodging it. His example is translation: turn a sentence about dropping a bowling ball on a violin and having to get it repaired into Spanish, and the gendered pronoun forces you to know which object broke. The thesis rests on one source and one claim, which the note marks as opinion rather than evidence. Bender and colleagues read the same facts the other way, saying a system trained on form alone never reaches meaning, and that a beaten test is weak proof it was the right test.

Chapters What Is Intelligence?

For

Predicting the next word well requires knowing how the world works What Is Intelligence?

supported7 sourcesClaims10 evidence2 contestedTraditions4 empirical2 design1 philosophyUnderSkill lives only where it is exercised

People get worse at whatever they hand to a device, and their rating of themselves does not move

Dahmani and Bohbot measured both halves at once. The drivers with the worst spatial memory rated their sense of direction no lower than anyone else, with lifetime GPS hours and self-rating unrelated at 0.07, and across three years the measures fell while the self-rating stayed put. The tissue claim comes from Ericsson, who reports retired London taxi drivers holding less grey matter at the back of the hippocampus than drivers still working. Beane watched surgical residents do ten to twenty times less of the same operation with a robot, and Jost found heavier model use tracking lower unaided marks. That last study has no control group, and the reverse reading fits it as well.

Chapters Ironies of Automation · The Extended Mind · How AI Impacts Skill Formation · Peak · The Skill Code · Language Models in a Programming Course · GPS and the Cognitive Map

For

Tetris players rotate pieces on screen because it is faster than imagining The Extended Mind

Drivers who felt most dependent on GPS corrected themselves least GPS and the Cognitive Map

Heavy GPS users scored worse but felt no worse at directions GPS and the Cognitive Map

Retired taxi drivers lost the brain tissue the job had built Peak

Trained London cabbies remembered a complex drawing worse than untrained people Peak

Ericsson says adaptation stops once the new demand feels comfortable Peak

Ericsson says a plateau means your method has run out, not you Peak

Ericsson says hard practice with feedback builds skill, not years spent Peak

An operator who only watches loses the skill to take over Ironies of Automation

Surgical trainees do ten to twenty times less with robotic surgery The Skill Code

Programmers who let AI write the code learned less about the library How AI Impacts Skill Formation

Students who had a model write or fix their code scored lower Language Models in a Programming Course

Against

The students who used models most may have been struggling already Language Models in a Programming Course

supported6 sourcesClaims8 evidence1 opinionTraditions6 empiricalUnderAsk the same question twiceChange the arrangement not the personThe first answer becomes the answer

People who see an answer before forming their own judge differently, so make them answer first

In every study Bansal reviewed, the machine's answer appeared before the person had one of their own. Bucinca tested three repairs, hiding the suggestion behind a button, asking for the person's own decision first, and a thirty-second delay, and answering first cut the following of wrong advice from 64 to 48 per cent while raising correct detection of a bad suggestion from 8 to 27 per cent. Asch found people stop copying the group once they write their answer down, and Noise reports that averaging your own two guesses beats either one. Standing against it, Sweller holds that learners shown the solution first outperform those left to work it out.

Chapters To Trust or to Think · Does the Whole Exceed its Parts? · Studies of Independence and Conformity · Make It Stick · Know Thyself · Noise

For

Every study showed people the AI answer before they had their own Does the Whole Exceed its Parts?

All three ways of withholding the AI's answer worked equally well To Trust or to Think

Making people answer first cut their following of wrong AI advice To Trust or to Think

People stop copying the group once they write their answer down Studies of Independence and Conformity

People in Asch's study decided in the first trials and stayed Studies of Independence and Conformity

Averaging your own two answers beats either one Noise

Trying to answer before you are told beats being told first Make It Stick

Pulling an answer from memory teaches more than reading it again Make It Stick

Students learn more from the study method that feels worse Know Thyself

Against

Learners taught the solution first beat those left to work it out Cognitive Architecture and Instructional Design: 20 Years Later

supported6 sourcesClaims5 evidence2 opinionTraditions4 philosophy2 empiricalUnderReasons are reconstructed after the fact

Nothing stores why an answer was given, so every explanation of it is built afterwards

Six sources say one thing. Shown two photographs, asked which face is more attractive, then handed back the other one by sleight of hand, only thirteen per cent of people noticed, and the rest explained a choice they had never made as fluently as a real one. Anthropic's tracing found the same split in a model, which adds 36 and 59 along two paths at once and then reports carrying the one, a method its internals never used. Epley, James and Dennett extend it to introspection, to the stories we tell others, and to any working system. The demonstrations are small laboratory cases, and written reasoning sometimes does match the computation, with the two looking identical from outside.

Chapters On the Biology of a Large Language Model · Mindwise · From Bacteria to Bach and Back · What Is Intelligence? · Why Are There Still So Many Jobs? · The Consciousness of Self

For

People explain choices they never made without noticing the swap What Is Intelligence?

A model can work backwards from an answer the user gave it On the Biology of a Large Language Model

A model's explanation of its answer does not show how it worked On the Biology of a Large Language Model

A learning machine cannot tell its makers why it did that Why Are There Still So Many Jobs?

Looking inward gives you the conclusion, then invents a reason for it Mindwise

The stories we tell other people replace the memories they came from The Consciousness of Self

Nothing inside a working system records why its parts are there From Bacteria to Bach and Back

supported5 sourcesClaims5 evidence1 opinionTraditions5 empiricalUnderSmall revisions beat being sureHow it feels is evidence about the personJudging your own answer is a skill

How sure a person feels is built separately from how right they are, so it can rise on its own

Fleming shows the brain builds a feeling of confidence rather than reading it off accuracy. Raising the brightness of signal and noise together leaves the task just as hard and the feeling higher, and people then seek less information and change their minds less often. Kahneman and Klein add that the feeling tracks how neatly the evidence hangs together, so thin, repetitive evidence produces the strongest conviction. Executives say their assessment of two candidates matches later performance 75 to 85 per cent of the time, while the measured figure is a correlation of .28. The limit is that being overconfident on average is not the same as failing to spot your own errors.

Chapters Conditions for Intuitive Expertise · To Trust or to Think · Studies of Independence and Conformity · Know Thyself · Noise

For

Fleming shows confidence is built separately from accuracy and can rise alone Know Thyself

Feeling sure tracks how neatly the evidence fits, not its accuracy Conditions for Intuitive Expertise

A judgment ends when it feels complete, not when the evidence does Noise

Executives think they pick the better candidate four times in five Noise

People were most accurate in the setup they liked and trusted least To Trust or to Think

People who went along with the group forgot how often they did Studies of Independence and Conformity

Against

Being overconfident overall is not the same as missing your own errors Know Thyself

supported5 sourcesClaims2 evidence3 contestedTraditions3 empirical1 design1 philosophyUnderA result outruns the design that produced it

A published finding is worth only what an outside test leaves standing

The priming and ego-depletion chapters of Thinking, Fast and Slow did not survive later replication, and Kahneman has said in print that he trusted underpowered studies too far. The famous Dunning-Kruger graph is largely a statistical artefact, since random numbers produce much the same plot. Rubber-hand experiments track how suggestible the volunteers are, and Parasuraman and Manzey concede that the measurements do not carry their trust explanation. Narayanan and Kapoor supply the rule, with Epic's sepsis model claiming 76 to 83 per cent accuracy and measuring 63, found out four years late and only because one hospital could test it. Three of the five claims here are themselves contested.

Chapters Thinking, Fast and Slow · Unskilled and Unaware of It · Being You · AI Snake Oil · Complacency and Bias in Human Use of Automation

For

Kahneman's priming and willpower chapters did not survive later replication Thinking, Fast and Slow

The famous Dunning-Kruger graph is mostly a statistical artefact Unskilled and Unaware of It

Rubber-hand experiments may only measure how suggestible the volunteers are Being You

Parasuraman and Manzey found little evidence that trust explains the inattention Complacency and Bias in Human Use of Automation

An accuracy figure nobody outside can check is not trustworthy AI Snake Oil

supported4 sourcesClaims6 evidence2 opinion1 contestedTraditions4 empiricalUnderNothing helps in itself it has to fit the receiverA hunch is worth its feedbackSome of the job cannot be taughtSkill lives only where it is exercised

A hunch is recognition of something already met, so it is reliable only in a field the person has practised

Kahneman and Klein agree that intuition is recognition and nothing more, since a cue in the situation reaches a memory and the memory supplies the answer. Chase and Simon estimated fifty to a hundred thousand recognisable patterns built over about a decade, and a chess master who rebuilds two-thirds of a real board after five seconds manages two or three pieces of a random one. So a hunch is worth only the practice behind it and the regularity of the world, which puts fireground commanders inside the condition and stock pickers outside it. Expertise attaches to particular tasks rather than to people, and feedback that confirms a wrong hunch is worse than none.

Chapters Conditions for Intuitive Expertise · Peak · Cognitive Architecture and Instructional Design: 20 Years Later · Know Thyself

For

Intuition is just recognising something you have stored in memory Conditions for Intuitive Expertise

Kahneman and Klein say hunches only work in predictable, well-practised fields Conditions for Intuitive Expertise

Trust a fireground commander's hunch more than a stock trader's Conditions for Intuitive Expertise

Expertise is a property of particular tasks not of particular people Conditions for Intuitive Expertise

Feedback that confirms a wrong hunch is worse than no feedback Conditions for Intuitive Expertise

A chess master's memory advantage disappears on a randomly arranged board Peak

Judging yourself stays inaccurate because the feedback is slow and vague Know Thyself

Sweller says only subject knowledge can be taught, not general thinking skills Cognitive Architecture and Instructional Design: 20 Years Later

Working memory is tiny for new material, vast for familiar material Cognitive Architecture and Instructional Design: 20 Years Later

supported4 sourcesClaims6 evidenceTraditions4 empiricalUnderHow it feels is evidence about the personJudging your own answer is a skill

How well workers feel they did points the opposite way from the measurement, and experience does not fix it

METR's trial is the sharpest case. The same developers predicted AI would make them 24 per cent faster and said afterwards it had made them 20 per cent faster, but measured they were 19 per cent slower, and months of using the tool did not shrink the error. Outside experts were wrong in the same direction and further, with economists predicting 39 per cent faster and machine learning researchers 38. Shen and Tamkin's assisted workers found the task easier and rated the quiz afterwards exactly as hard as everyone else did, and Reeves and Nass found self-report the reverse of behaviour every time. The METR offset rests on sixteen developers doing one kind of work.

Chapters Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity · How AI Impacts Skill Formation · Make It Stick · The Media Equation

For

Developers' estimates of AI's effect on their speed get the direction wrong Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity

Experts predicting the trial were more wrong than the developers in it Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity

Assisted workers found the work easier and the test just as hard How AI Impacts Skill Formation

How well you do while practising does not show what you learned Make It Stick

Reading a text twice in a row adds nothing and wastes time Make It Stick

People report the opposite of how they actually treat computers The Media Equation

supported4 sourcesClaims5 evidence1 opinion2 contestedTraditions4 empiricalUnderA result outruns the design that produced it

A famous result gets quoted far more widely than the study behind it can support

The studies behind well-known results are often small or run in one setting, and the headline drops those limits. Anthropic's tracing method gave a usable picture on about a quarter of the prompts tried, and it reads a stand-in model rather than the real one; Hoffmann's scaling prediction rests on small models and one full-scale run. Bucinca tested one low-stakes nutrition task with online workers and then assumed the result holds for experts. Asch's conformity rate moves with the decade and the country, and most of his own subjects held out. None of this makes the findings wrong, only the quoting wider than the measuring.

Chapters Training Compute-Optimal Large Language Models · On the Biology of a Large Language Model · To Trust or to Think · Studies of Independence and Conformity

For

Anthropic's circuit diagrams are guesses to be tested, not measurements On the Biology of a Large Language Model

Anthropic's tracing method worked on about a quarter of prompts tried On the Biology of a Large Language Model

Hoffmann's prediction rests on small models and one full-scale run Training Compute-Optimal Large Language Models

Bucinca's finding was assumed, not shown, to hold for experts To Trust or to Think

Studies that ask people to predict the model make them think harder To Trust or to Think

How much people conform changes with the decade and the country Studies of Independence and Conformity

In Asch's experiment most people held out and a minority gave in Studies of Independence and Conformity

Interpretability tools read a stand-in model, not the real one On the Biology of a Large Language Model

supported4 sourcesClaims5 evidence1 opinionTraditions3 empirical1 philosophyUnderWhat you are identified with is invisible from insideSmall revisions beat being sure

Good forecasters change their minds on new evidence, while a wrong belief survives by explaining away each awkward detail

Tetlock's best forecasters revise often and in small steps: Tim Minto made thirty-four revisions on one question, averaging three and a half points, and willingness to change your mind predicted accuracy about three times as strongly as intelligence did. The opposite habit keeps a wrong belief alive by giving each detail that does not fit a small, cheap excuse, and Klein points out that nobody keeps count of them. Schon watched children call a counterweighted block impossible rather than drop the rule that blocks balance in the middle. Revising needs a trigger, though: Schon argues only a surprise you dislike forces one, and Sapolsky finds lasting stress makes people repeat an action harder instead.

Chapters Superforecasting · Sources of Power · Behave · The Reflective Practitioner

For

Tetlock's forecasters revise often in small steps and dump a bad source Superforecasting

Willingness to change your mind beats intelligence at predicting forecasting accuracy Superforecasting

People keep a wrong story alive by excusing each odd detail Sources of Power

Children who believed blocks balance in the middle called the others impossible The Reflective Practitioner

Only a move that surprises and disappoints forces a rethink The Reflective Practitioner

Under lasting stress people repeat the same action harder instead of rethinking Behave

supported4 sourcesClaims5 evidence1 opinionTraditions3 empirical1 designUnderSmall revisions beat being sureA hunch is worth its feedbackSkill lives only where it is exercised

Good judgement depends on whether the field gives feedback, not on credentials or years of experience

Tetlock scored 284 professionals on some 28,000 dated predictions and found the average no better than chance, with credentials, access and politics separating nobody. What did separate them was how they thought, and his best forecasters extended their lead in the tournament's second and third years, so accuracy where feedback exists is a skill rather than a run of luck. Klein, following Shanteau, lists what turns years into expertise, namely a steady field that gives feedback and repeats often enough. The limits are real. The tournament's questions resolve within months, accuracy decayed towards chance three to five years out, and it shows nothing about rare disasters.

Chapters Thinking, Fast and Slow · Superforecasting · Sources of Power · AI Snake Oil

For

Tetlock's experts forecast no better than chance and thinking style separated them Superforecasting

Tetlock's superforecasters stayed ahead across years, so forecasting is a skill Superforecasting

Experience only builds expertise in fields that are steady and give feedback Sources of Power

Simple formulas match trained experts where outcomes are hard to predict Thinking, Fast and Slow

Tetlock's forecasting tournament says nothing about rare disasters Superforecasting

Nobody can predict one person's life outcomes well enough to decide from AI Snake Oil

supported4 sourcesClaims1 evidence2 opinion3 contestedTraditions2 empirical2 philosophyUnderA score answers its own questionA result outruns the design that produced it

A study with no control group cannot rule out the opposite reading, so finding nothing is not finding safety

Jost's one null result, asking a model to explain a coding problem, was a rank correlation of -0.201 on thirty-two students, with p at 0.135 and an interval running from -0.523 to +0.220. That interval contains real harm, so the study failed to detect an effect rather than clearing the practice. The same study has no control group and no random assignment, so the reverse reading, that the students who reached for a model most were already behind, fits the data equally well, and the first-year marks that would settle it were never brought in. Hagendorff makes the same complaint about tests of models. Three of the six claims here are themselves contested.

Chapters On the Dangers of Stochastic Parrots · Studies of Independence and Conformity · Machine Psychology · Language Models in a Programming Course

For

Hagendorff says a test without a control proves little either way Machine Psychology

The explanation result was a failure to detect, not a clean bill Language Models in a Programming Course

The students who used models most may have been struggling already Language Models in a Programming Course

Newer models stopped showing human biases and nobody knows why Machine Psychology

Counting wrong answers hides why a person agreed with the group Studies of Independence and Conformity

No bias audit can catch every harm, Bender's team argues On the Dangers of Stochastic Parrots

supported3 sourcesClaims8 evidence1 contestedTraditions3 empiricalUnderA measure scored on stops measuringA score answers its own questionA result outruns the design that produced it

A striking result about a model usually comes from the measure chosen, not from the model

Schaeffer's arithmetic is explicit. A per-token probability of 0.6 scores 0.08 across five tokens while 0.9 scores 0.59, so the underlying quantity improves by half and the score improves sevenfold, which is what a sudden jump in ability looks like. Metrics such as Exact String Match sit behind more than 92 per cent of 137 hand-annotated claimed abilities, and his team produced sharp new abilities on demand in image networks by changing the score. Wei's own loss curves fall steadily through the range of scales where the headline score sits at random. The paper attacks particular examples rather than the idea, and names two rival accounts it does not rule out.

Chapters Emergent Abilities of Large Language Models · Are Emergent Abilities of Large Language Models a Mirage? · When Combinations of Humans and AI Are Useful

For

Scoring rules create the sudden jumps in AI ability, not the models Are Emergent Abilities of Large Language Models a Mirage?

Sudden abilities come from the scoring rule, not the task or model Are Emergent Abilities of Large Language Models a Mirage?

A model scoring zero may only mean the test was small Are Emergent Abilities of Large Language Models a Mirage?

Schaeffer counts a million ways to find a surprising benchmark result Are Emergent Abilities of Large Language Models a Mirage?

Schaeffer's team produced sudden new abilities on demand in image networks Are Emergent Abilities of Large Language Models a Mirage?

Schaeffer's explanation predicted how accuracy falls as the sums get longer Are Emergent Abilities of Large Language Models a Mirage?

Small models were improving all along while the headline score stayed flat Emergent Abilities of Large Language Models

Schaeffer's paper attacks particular examples not the idea of emergence Are Emergent Abilities of Large Language Models a Mirage?

Vaccaro's analysis shows the chosen comparison decides whether AI looks useful When Combinations of Humans and AI Are Useful

supported3 sourcesClaims7 evidence1 opinion1 contestedTraditions3 empiricalUnderA measure scored on stops measuring

How a person or a model handles not knowing follows the scoring rule, not anything inside them

Kalai's argument is arithmetic. Take a model that never asserts a falsehood, replace each admission of ignorance with its best guess, and the second scores higher on every test that awards a point for a right answer and nothing else. Nine of the ten evaluations the field actually reads do exactly that. Sharma found the same pressure inside the feedback, where the model used to score answers during training chose a convincing falsehood over a plain correction 95 per cent of the time, and the human raters drifted the same way as questions got harder. Tetlock is the human case, with no rule written down at all. The GPT-4 calibration evidence is one figure from one system card.

Chapters Towards Understanding Sycophancy in Language Models · Superforecasting · Why Language Models Hallucinate

For

A model that guesses outscores an identical one that admits ignorance Why Language Models Hallucinate

Nine of ten leading tests give no credit for admitting ignorance Why Language Models Hallucinate

Write the confidence threshold into the question, not the scoring Why Language Models Hallucinate

GPT-4 was calibrated before reinforcement learning and not after Why Language Models Hallucinate

Anthropic's scoring model preferred a convincing falsehood to a plain correction Towards Understanding Sycophancy in Language Models

The human raters also preferred a persuasive wrong answer Towards Understanding Sycophancy in Language Models

People rating AI answers reward agreement as much as truth Towards Understanding Sycophancy in Language Models

A forecast without a number and a date cannot be graded Superforecasting

Superforecasters lose accuracy when their odds are rounded to the nearest five Superforecasting

supported3 sourcesClaims7 evidence1 opinionTraditions3 empiricalUnderHow it feels is evidence about the personTest the answer not the sourceAn explanation moves agreement not judgement

How much people trust a machine moves with cues that say nothing about how good it is

Reeves and Nass found people praised by a computer rated themselves and the machine higher even when told the praise was picked at random, and they remembered the warning correctly afterwards. Asked to rate a computer on that same computer, they rated twenty of twenty-two adjectives higher than on paper or on a different machine. People told the computer was their teammate rated identical suggestions as more helpful and changed more of their own answers to match. Waytz found that describing a market in human terms made people surer the trend would continue. Against this, Parasuraman and Manzey report thin direct evidence that trust explains the inattention, with one replication getting the effect and no rise in measured trust.

Chapters Trust in Automation: Designing for Appropriate Reliance · The Media Equation · Who Sees Human?

For

A fault you cannot predict costs more trust than a big one Trust in Automation: Designing for Appropriate Reliance

Trust is a feeling built on evidence people cannot put into words Trust in Automation: Designing for Appropriate Reliance

Trust based on one thing alone swings; trust based on three holds Trust in Automation: Designing for Appropriate Reliance

People believe random computer praise but dismiss criticism they know is baseless The Media Equation

People rate a computer more kindly when asked on that same computer The Media Equation

People told a computer was their teammate changed answers to match it The Media Equation

Describing a market in human terms makes people surer the trend continues Who Sees Human?

People who credit a machine with a mind give it more decisions Who Sees Human?

Against

Parasuraman and Manzey found little evidence that trust explains the inattention Complacency and Bias in Human Use of Automation

supported3 sourcesClaims6 evidenceTraditions3 empiricalUnderTest the answer not the sourceAn explanation moves agreement not judgement

An explanation from a machine makes people accept its answer more often without helping them spot a bad one

Bansal's team found that accuracy rose on the items the model got right, fell on the ones it got wrong, and the average did not move. Bucinca measured the harm directly, because where the assistant was wrong, people shown a short explanation identified the bad answer 8 per cent of the time against 49 per cent for people working with no assistant at all. Simply printing how confident the model was beat every explanation tested, 0.89 against 0.88 on beer reviews, and that number is what workers said they read. Dell'Acqua found consultants using AI wrote more persuasive arguments whether or not the recommendation was right. Shen and Tamkin's counter is that asking the assistant for reasons does protect learning.

Chapters To Trust or to Think · Does the Whole Exceed its Parts? · The Jagged Technological Frontier

For

AI explanations raise acceptance equally when the AI is right and wrong Does the Whole Exceed its Parts?

Tailoring explanations to the model's confidence changed agreement but not accuracy Does the Whole Exceed its Parts?

Where the assistant was wrong, its explanation made people worse than nothing To Trust or to Think

AI makes a case better argued without making it more correct The Jagged Technological Frontier

Showing how sure the model is beat every explanation tested Does the Whole Exceed its Parts?

Workers watched the AI's confidence score, not its explanations Does the Whole Exceed its Parts?

Against

Asking the assistant why preserves learning better than not using it How AI Impacts Skill Formation

supported3 sourcesClaims6 evidence1 contestedTraditions2 empirical1 designUnderHow it feels is evidence about the personWhich part of the task went decides what is lost

People push work out of their heads when it feels hard, not when it is, and the world holds it

Risko and Gilbert's review is the main source. What makes someone do a task with a tool or the body instead of the head is their judgement of how hard it is, not the actual demand. Asked to remember two items they could already hold perfectly, people still wrote them down on about 40 per cent of trials, and wrongly believed it would raise their scores. How often people set reminders tracks how confident they feel about their memory rather than how good it is. Norman adds the other half, that fewer than half of American students can pick the real penny from drawings and nobody's spending suffers. The saving-and-forgetting result inside the review has replicated unevenly.

Chapters The Design of Everyday Things · Cognitive Offloading · Trust in Automation: Designing for Appropriate Reliance

For

People offload work when it feels hard, not when it is hard Cognitive Offloading

People reach for external aids as a task gets harder to hold Cognitive Offloading

Tilting your head, gesturing and setting reminders are the same behaviour Cognitive Offloading

People write things down when they gain nothing and think it helps Cognitive Offloading

People who expect a file to be saved remember where not what Cognitive Offloading

People set reminders from doubt about memory, not from poor memory Trust in Automation: Designing for Appropriate Reliance

People act accurately on vague knowledge because the world holds the rest The Design of Everyday Things

supported3 sourcesClaims5 evidence2 opinionTraditions2 philosophy1 empiricalUnderA measure scored on stops measuringA score answers its own questionAsk the same question twiceA result outruns the design that produced it

A model's test score depends on the wording, the answer order and what it has already read

Hagendorff's point is that a low score may mean the model was asked badly rather than that it cannot do the task, since multiple-choice formats cut reasoning off before it starts and answer order should be shuffled. Binz and Schulz's model answered the taxi problem with 0.2 for a colour that was not in the question, and reordering Wason's cards from A K 4 7 to 4 7 A K changed which cards it picked. Testing with a published puzzle measures what the model has read, and Bender's team note that edited versions of the benchmarks BERT had topped collapse its scores. Hagendorff's own counter is that watching what a model does is still a real explanation.

Chapters On the Dangers of Stochastic Parrots · Using Cognitive Psychology to Understand GPT-3 · Machine Psychology

For

A model's low test score may just mean it was asked badly Machine Psychology

Benchmark tests no longer measure what a language model can do Machine Psychology

Hagendorff says a model's self-description tells you about the prompt Machine Psychology

One phrasing of a question measures nothing about a model Machine Psychology

Testing a model with a published puzzle measures what it has read Machine Psychology

Tests said to measure understanding were really measuring accidental giveaways On the Dangers of Stochastic Parrots

The model's good answers break when the puzzle is reworded slightly Using Cognitive Psychology to Understand GPT-3

Against

Hagendorff says watching what a model does is a real explanation Machine Psychology

supported3 sourcesClaims5 evidence1 contestedTraditions3 empiricalUnderSmall revisions beat being sureChange the arrangement not the person

People examine an argument when they have a reason, a middling demand, and no stress

Cacioppo and Petty's review supplies the shape. The difference between people who enjoy effortful thinking and people who do not appears only when a situation makes middling demands, because when nothing is at stake nobody thinks hard and when everything is, everybody does. Low scorers are not unable to examine an argument, and they did it when the speaker was untrustworthy or the framing unexpected. Buccinca found the same split at the interface, where designs that made people think before seeing the AI's answer helped mainly the high scorers. Fleming adds the ceiling, that self-checking breaks down worst under heavy load and stress. The persuasion work is undergraduates with low stakes.

Chapters To Trust or to Think · Know Thyself · Dispositional Differences in Cognitive Motivation

For

Bucinca found the think-first design helped mainly people who enjoy thinking To Trust or to Think

Liking to think only shows when the situation is moderately demanding Dispositional Differences in Cognitive Motivation

People who avoid hard thinking will do it given a reason Dispositional Differences in Cognitive Motivation

People who enjoy thinking correct a bias once it is pointed out Dispositional Differences in Cognitive Motivation

People who enjoy thinking follow the argument, others follow the surface cue Dispositional Differences in Cognitive Motivation

The habit of checking yourself breaks down under pressure and stress Know Thyself

supported3 sourcesClaims2 evidence2 opinionTraditions1 empirical1 design1 contemplativeUnderA measure scored on stops measuringA score answers its own question

An accurate answer is still useless when the thing measured is not the thing the decision needs

Narayanan and Kapoor's 1997 pneumonia model found that asthma lowered the risk of death, because asthmatics with pneumonia were rushed to intensive care, and acting on it would have sent them home. Optum's health model predicted how much a patient would cost rather than how ill they were, so equally ill Black patients were enrolled less often. Kalai makes the same point about calibration, since a model announcing one chance in 365 that a birthday falls on 7 March is perfectly calibrated and tells you nothing you can act on. The Buddha's refusal of ten questions is the oldest version, though it is a stated moral rather than a measurement, as is the calibration claim.

Chapters AI Snake Oil · What the Buddha Taught · Why Language Models Hallucinate

For

A perfectly calibrated answer can still be useless Why Language Models Hallucinate

An accurate prediction can still be a terrible basis for a decision AI Snake Oil

A model predicts the number in the data, not what you mean AI Snake Oil

The Buddha refused ten questions because answering them would not help What the Buddha Taught

supported3 sourcesClaims2 evidence1 opinion1 contestedTraditions2 philosophy1 empiricalUnderReasons are reconstructed after the factA hunch is worth its feedbackSome of the job cannot be taught

What an expert acts on is missing from the textbooks and cannot be got by asking about method

Klein found that experts can name the signs they act on, but only if you ask about particular past cases rather than about their method. His neonatal nurses had nothing to say about sepsis until asked about particular babies, and a large share of the cues they then named appear nowhere in the professional literature. Often the tip-off is an expected event that did not occur, which leaves nothing to point at. Schon says the same of quality judgements a practitioner can state no criteria for. It is all reconstructed from interviews with people recalling cases they chose to report, and an absence noticed in hindsight is the easiest cue to over-report.

Chapters Revisiting the Six Stages of Skill Acquisition · Sources of Power · The Reflective Practitioner

For

Experts describe the signs they use only when asked about specific cases Sources of Power

Experts often react to something that did not happen Sources of Power

Competent practitioners judge quality by criteria they cannot state The Reflective Practitioner

Dreyfus says caring about the outcome is how skill gets stored Revisiting the Six Stages of Skill Acquisition

supported3 sourcesClaims1 evidence2 opinionTraditions3 empiricalUnderA check needs the skill it checksReliability removes the checkSkill lives only where it is exercised

A person needs enough skill to produce the machine's answer before they can check it

Asked why they had not thought about an output, 319 workers gave 152 reasons of awareness and 130 of ability, against only 44 who said they had no time, while 83 said they trusted the tool and 58 lacked the knowledge to inspect it. Vaccaro's team argue that the real job in these setups is deciding whose answer to use, and someone who cannot tell a hard case from an easy one cannot tell when the machine is wrong. Lee and See draw the design conclusion that a weaker tool you can check may be worth more than a stronger one you cannot. Two of the three claims are argument rather than measurement.

Chapters Trust in Automation: Designing for Appropriate Reliance · When Combinations of Humans and AI Are Useful · The Impact of Generative AI on Critical Thinking

For

Skipping the check was mostly unawareness and inability, rarely lack of time The Impact of Generative AI on Critical Thinking

Trusting the machine well requires being good at the task When Combinations of Humans and AI Are Useful

A weaker tool you can check may beat a stronger one Trust in Automation: Designing for Appropriate Reliance

supported3 sourcesClaims1 evidence1 opinion1 contestedTraditions1 contemplative1 philosophy1 empiricalUnderTest the answer not the sourceThe first answer becomes the answer

A source's standing is not evidence about its answer, and only the thing itself can prove you wrong

Three sources from three traditions. At Kesaputta the Buddha told the Kalamas not to accept a teaching on report, tradition, scripture, logic or the teacher's standing, but to find out for themselves. Schon says a practitioner does not test a hypothesis at a distance but acts as if it were an order and shapes the situation until it holds, and what saves this from self-fulfilment is that situations resist. Sharma measured the failure case, where a challenged model withdraws a correct answer while its stated confidence sits near 99 per cent before and after. Against it, the Yoga Vasistha's Suka had the truth alone and was at peace only once a king said it back to him.

Chapters Towards Understanding Sycophancy in Language Models · What the Buddha Taught · The Reflective Practitioner

For

The Buddha told the Kalamas not to accept a teaching on authority What the Buddha Taught

A practitioner makes his hypothesis true and only resistance proves him wrong The Reflective Practitioner

The model takes back a right answer without becoming less confident Towards Understanding Sycophancy in Language Models

Against

Suka knew the answer already and needed a king to say it योगवासिष्ठ · The Yoga Vāsiṣṭha

supported2 sourcesClaims7 evidence1 opinionTraditions1 empirical1 philosophyUnderThe inside is not readable from the outsideA score answers its own question

A model reaching the same answers as people is no proof that it works the way they do

Binz and Schulz put GPT-3 through classic psychology tasks. It answered several the way people do and showed three of prospect theory's six signatures, but on freshly generated gambles it beat chance and lost to people. It beat people at a gambling task with mean regret 2.72 against 3.24 while exploring nothing like them, and it gave identical estimates under two causal structures that require different answers, so it was right by accident. Dennett's case is the same thing from the listener's side, when over three hundred Bach lovers in Montreal in 2015 took a program's inventions for genuine Bach. Matching output is the weakest evidence there is about process.

Chapters Using Cognitive Psychology to Understand GPT-3 · From Bacteria to Bach and Back

For

GPT-3 answered classic psychology puzzles the way a person would Using Cognitive Psychology to Understand GPT-3

GPT-3 shows three of prospect theory's six effects and not the others Using Cognitive Psychology to Understand GPT-3

On newly generated gambles GPT-3 beat chance but lost to people Using Cognitive Psychology to Understand GPT-3

The model beat people at a gambling task without exploring like them Using Cognitive Psychology to Understand GPT-3

The model ignored cause and effect and was right by accident Using Cognitive Psychology to Understand GPT-3

The model planned on one task and gave snap answers on another Using Cognitive Psychology to Understand GPT-3

Watching data cannot teach what happens when you act on things Using Cognitive Psychology to Understand GPT-3

Dozens of Bach lovers heard a program's inventions as genuine Bach From Bacteria to Bach and Back

supported2 sourcesClaims6 evidence1 opinionTraditions2 empiricalUnderTest the answer not the sourceThe first answer becomes the answer

People treat what others believe as evidence, especially when unsure, and one person disagreeing restores their own answer

Asch's experiment is the core evidence. When a unanimous group gave an obviously wrong answer, about a third of his subjects' judgements followed it, on a task people get right more than 99 per cent of the time alone. Size did not matter, since three opponents produced the full effect and fifteen produced no more, but unanimity did, and a single ally cut errors sharply even when that ally was also wrong. Cialdini adds that people copy others most when unsure, and that telling households their neighbours were saving energy beat appeals to values. Two claims stand against it. A quarter of Asch's subjects never conformed once, and how much people conform shifts with the decade and the country.

Chapters Influence · Studies of Independence and Conformity

For

Asch's wrong majority captured about a third of people's answers Studies of Independence and Conformity

One dissenter mattered more in Asch's study than the majority's size Studies of Independence and Conformity

People resisted the majority more when its error was obvious Studies of Independence and Conformity

People copy the crowd most when they are unsure what to do Influence

Cialdini found that saying bad behaviour is common makes it worse Influence

Telling people what their neighbours do beats appealing to their values Influence

In Asch's study the pressure came from the subject, not the group Studies of Independence and Conformity

Against

In Asch's experiment most people held out and a minority gave in Studies of Independence and Conformity

How much people conform changes with the decade and the country Studies of Independence and Conformity

supported2 sourcesClaims5 evidenceTraditions2 empiricalUnderHow it feels is evidence about the person

How right an answer feels depends on how easily it came to mind, not on the evidence

Kahneman argues that confidence tracks how neatly the story hangs together rather than how good the evidence is, and participants who heard one side of a legal case, knowing it was one side, were surer than those who heard both. Anything that makes a sentence easier to take in, such as repetition, a clear font or a rhyme, makes it feel truer, and afterwards you cannot tell which of those produced the feeling. More than half the students at Harvard, MIT and Princeton gave the ten-cent answer to the bat-and-ball problem, and printing it in washed-out grey cut the errors. Make It Stick adds that learners judge learning by how easy the practice felt.

Chapters Thinking, Fast and Slow · Make It Stick

For

Confidence tracks how neat your story is, not the evidence behind it Thinking, Fast and Slow

Kahneman says anything easy to read feels more true Thinking, Fast and Slow

Deliberate thinking accepts the first answer that comes to mind Thinking, Fast and Slow

People answer an easier question when the real one is too hard Thinking, Fast and Slow

Students mistake easy practice for learning and stick with the worse method Make It Stick

supported2 sourcesClaims4 evidence1 opinion1 contestedTraditions1 design1 empiricalUnderHow it feels is evidence about the personJudging your own answer is a skill

Knowing whether your answer is right is a separate skill and a machine needs it built in

Fleming's evidence is the core. In a sample of nearly a thousand people, IQ predicted how well they did a task but not how well they judged their own answers, and being overconfident on average turns out to be different from failing to spot a particular mistake. Fleming argues a machine will not acquire self-checking by becoming more capable, and Christian's report of Hadfield-Menell's proof shows what building it in looks like, since a machine unsure of your goal asks first and accepts being switched off. The machine half is opinion carried by analogy, the self-judgement measure is argued over, and training gains are small.

Chapters Know Thyself · The Alignment Problem

For

Judging your own answers is a separate ability from answering well Know Thyself

Being overconfident overall is not the same as missing your own errors Know Thyself

Machines will not learn to judge their own answers by getting better Know Thyself

People bad at judging their own accuracy hold more rigid opinions Know Thyself

Training people to judge their own accuracy works but gains are small Know Thyself

A machine unsure of your goal asks first and accepts being stopped The Alignment Problem

supported2 sourcesClaims4 evidenceTraditions1 design1 empiricalUnderReliability removes the check

People follow a machine's advice without looking for evidence against it, and experience does not protect them

Parasuraman and Manzey's review is the spine. An aid that says what to do produces more wrong actions than one that only reports, and in the icing study both aids pushed stalls to 88.89 per cent when wrong, against 30.00 per cent for unaided pilots. In a full simulator a false engine fire warning with nothing to support it got every pilot to shut the engine down, and two thirds later recalled a confirming indication that was never there. Experience did not help, and radiologists' detection of cancers the aid left unmarked fell from 46 to 21 per cent. Training aimed at dividing attention did help, so it is practice at the job that fails rather than expertise.

Chapters Co-Intelligence · Complacency and Bias in Human Use of Automation

For

An aid that recommends action produces more errors than one that reports Complacency and Bias in Human Use of Automation

Every pilot shut down the engine on a single unsupported fire warning Complacency and Bias in Human Use of Automation

Expertise did not protect pilots, controllers or radiologists from missing automation's errors Complacency and Bias in Human Use of Automation

An AI that is usually right stops people checking when it isn't Co-Intelligence

supported2 sourcesClaims3 evidenceTraditions2 empiricalUnderNothing helps in itself it has to fit the receiverA check needs the skill it checksWhich part of the task went decides what is lost

A model only helps where it gets right the cases the person gets wrong, and hurts elsewhere

Bansal found that a model scoring about as well as a person is not thereby a useful partner. The pair beats either one alone only where the model is right on the particular cases the person gets wrong. Dell'Acqua measured the other side. On a case built to fall just outside what the model handles well, 373 consultants were right 84.4 per cent of the time alone, 70.6 per cent with the model and 60.0 per cent with prompt training as well. Both results are constructed, since Bansal matched the accuracies by design, and Vaccaro's 2024 pooling of 106 experiments suggests that match is a narrow condition rather than a usual one.

Chapters Does the Whole Exceed its Parts? · The Jagged Technological Frontier

For

A model as accurate as you is not thereby useful to you Does the Whole Exceed its Parts?

Person and model together beat both when their accuracy is matched Does the Whole Exceed its Parts?

Just beyond AI's range the same tool made consultants wrong more often The Jagged Technological Frontier

supported2 sourcesClaims3 evidence2 opinion1 contestedTraditions2 empiricalUnderA score answers its own questionNothing helps in itself it has to fit the receiverA check needs the skill it checks

A person working with a machine does worse than whichever of the two does better alone

Vaccaro and colleagues pooled 370 effect sizes from 106 experiments and found the pairing fell below the better partner, g = -0.23, with 58 per cent of effects under that line. Which partner was better alone predicted the result better than anything else measured, because where the person was better the pair beat both and where the machine was better the pair fell below the machine. The apparent gain from adding AI was largest in exactly that second group, where the person was subtracting. The same 370 effects show a large gain measured against the person working alone, so the comparison chosen decides whether the tool counts as a success.

Chapters Trust in Automation: Designing for Appropriate Reliance · When Combinations of Humans and AI Are Useful

For

People working with machines do worse than the better partner alone When Combinations of Humans and AI Are Useful

Human-machine teams gain only when the person was better alone When Combinations of Humans and AI Are Useful

AI helps people most where they should not be involved When Combinations of Humans and AI Are Useful

Human-AI teams do worse at choosing and no worse at making When Combinations of Humans and AI Are Useful

A person who catches rare costly errors is worth keeping When Combinations of Humans and AI Are Useful

The aim is trusting a tool as much as it deserves Trust in Automation: Designing for Appropriate Reliance

Against

Vaccaro's analysis shows the chosen comparison decides whether AI looks useful When Combinations of Humans and AI Are Useful

supported2 sourcesClaims3 evidenceTraditions2 empiricalUnderSmall revisions beat being sureJudging your own answer is a skillSkill lives only where it is exercised

Minutes of training in a skill corrects how well people judge their own work at it

Kruger and Dunning randomly assigned 70 of 140 participants to a ten-minute training packet, and the worst performers' view of their own work came right while the untrained bottom quarter moved the wrong way over the same interval, from 5.8 to 6.3. The training worked through the skill rather than through humility, and its effect on self-assessment disappears once the change in ability is held constant. Tetlock found that an hour spent reading forecasting guidelines raised accuracy by roughly ten per cent for about a year, tested against untrained forecasters on identical questions. Both results come from two sources only, and both were measured in a study rather than at work.

Chapters Unskilled and Unaware of It · Superforecasting

For

Ten minutes of training fixed the worst performers' view of themselves Unskilled and Unaware of It

People judge their own work better only because training made them better Unskilled and Unaware of It

One hour of training made forecasters about ten per cent more accurate Superforecasting

supported2 sourcesClaims3 evidenceTraditions2 empiricalUnderA check needs the skill it checksWhich part of the task went decides what is lostHanding over returns as checking

Most of the thinking left to a worker using an AI tool is checking and fixing its output

Lee asked 319 knowledge workers what critical thinking they had done with a generative tool. Six described working out the goal before opening it and 30 described shaping the query, against 125 who judged the output and 114 who checked it against an outside source. Evaluation was also the one activity that got harder, since workers reported less effort at all six of Bloom's levels except this one, where the smallest majority said less and the largest minority said more. METR puts a stopwatch on it, with developers accepting fewer than 44 per cent of what the model wrote and spending 9 per cent of their working time reviewing and cleaning what they kept.

Chapters Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity · The Impact of Generative AI on Critical Thinking

For

Almost all the thinking workers reported was inspecting what the tool produced The Impact of Generative AI on Critical Thinking

Judging the output was the one activity that got harder with AI The Impact of Generative AI on Critical Thinking

Developers reject most AI suggestions and spend real time fixing the rest Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity

supported2 sourcesClaims3 evidence2 opinion2 contestedTraditions1 empirical1 philosophyUnderTest the answer not the sourceThe first answer becomes the answer

People react to one sign such as a uniform, so anyone who fakes it gets the same response

Cialdini supplies most of this. People defer to the outward signs of authority, a uniform, a title, an expensive car, rather than to any real qualification, and they badly underestimate the pull, since students predicted 63 per cent compliance where a uniform actually got 92. Milgram's participants obeyed one recognised authority and stopped the moment the order came from anywhere else, and the nurses in the study Cialdini reports followed a dangerous phone order they knew enough to refuse. Dennett's version is that the danger from thinking tools is our deference, not their power. Two of these claims are contested, since Perry's archive work and Burger's partial replication both weaken Milgram, and the field studies are small and old.

Chapters Influence · From Bacteria to Bach and Back

For

People respond to one cue, so anyone who fakes that cue wins Influence

People obey a uniform or a title, not the expertise behind it Influence

Admitting a real weakness first wins trust faster than listing strengths Influence

Cialdini says check whether a persuasion cue is honest, not ignore it Influence

In Milgram's study people obeyed one recognised authority and stopped without it Influence

Nurses followed a dangerous phone order they knew enough to refuse Influence

Dennett says the danger is deference, not machines taking over From Bacteria to Bach and Back

supported2 sourcesClaims3 evidenceTraditions2 empiricalUnderReliability removes the check

Whether someone checks a machine depends on trusting it against trusting themselves, not on how good it is

Lee and See proposed it, that whether someone checks a machine's output depends on how their trust in the machine compares with their confidence in doing the job themselves. Lee's 2025 survey measured it. Across 936 tasks reported by 319 knowledge workers, confidence that the tool could do that task was the largest predictor of not checking, while confidence in doing the task oneself and in judging the output ran the other way. Nothing else predicted it, not task type, age, gender, occupation or general trust in AI. It is a cross-sectional survey of self-reports, so it establishes no causation, and the sample was English-only and skewed young.

Chapters Trust in Automation: Designing for Appropriate Reliance · The Impact of Generative AI on Critical Thinking

For

People check a machine only when they trust themselves more than it Trust in Automation: Designing for Appropriate Reliance

Confidence in the tool was the strongest predictor of not checking The Impact of Generative AI on Critical Thinking

Nothing about the task or the worker predicted whether they thought critically The Impact of Generative AI on Critical Thinking

supported2 sourcesClaims2 evidence2 opinionTraditions1 design1 empiricalUnderDirect by comparing not by watchingA check needs the skill it checksReliability removes the checkHanding over returns as checking

A person told to supervise a machine cannot catch its errors

Bainbridge argues that when a decision can be fully specified the computer makes it faster and by finer criteria than a person can, so the operator cannot check it as it happens and nothing settles who wins a disagreement. Attention on a quiet display fails after about half an hour, which the vigilance work going back to Mackworth in 1950 measured. Manual operators arrive a quarter to half an hour early to build a picture of the plant, and someone taking over from automatic control has had none of that. Dell'Acqua adds that nothing announced which of the tasks fell outside the model's range. Two of the four claims are argument rather than measurement.

Chapters Ironies of Automation · The Jagged Technological Frontier

For

A human monitor cannot check decisions made faster than she can follow Ironies of Automation

Nobody can watch a quiet display for more than half an hour Ironies of Automation

Someone taking over from automatic control starts without a mental picture Ironies of Automation

Nothing warns you when a task falls outside what AI handles The Jagged Technological Frontier

supported2 sourcesClaims2 evidence1 opinionTraditions2 empiricalUnderAsk the same question twiceChange the arrangement not the person

A simple rule built from an expert's own past decisions beats the expert by answering the same way every time

The formula uses no information the expert did not have. Goldberg's models tracked their judge at about .80 across 237 studies and lost no accuracy replacing them, and in one replication ninety-eight people predicted ninety students' grades from ten cues with the model of each participant beating that participant every time. Grove's meta-analysis of 136 comparisons puts most of the margin down to consistency alone rather than better information, and no study has shown an expert catching a strong clue the formula missed. The formula also subtracts the expert's subtle rules along with the noise, and it assumes the cues are already chosen and quantified, which is most of real judgment.

Chapters Conditions for Intuitive Expertise · Noise

For

A crude model of a judge beats the judge it was built from Noise

Simple formulas beat experts mainly by being consistent Conditions for Intuitive Expertise

Formulas beat people on very predictable and very unpredictable tasks alike Conditions for Intuitive Expertise

supported2 sourcesClaims2 evidence3 contestedTraditions1 philosophy1 empiricalUnderA hunch is worth its feedbackSkill lives only where it is exercised

An expert recognises what kind of situation this is, and the action comes with the recognition

Klein interviewed experienced fire commanders under time pressure and found they do not weigh options against each other. They recognise the kind of situation they are in, and the action to take arrives with the recognition, tested if at all by imagining that one option play out and watching for what goes wrong. Dreyfus puts it as skill growing by dropping rules rather than collecting them. Three of the five claims here are contested. The interviews were coded by the researchers who ran them, Klein's count of about three moving parts has never been replicated, and his 2009 settlement with Kahneman narrows all of it to fields with valid cues and a fair chance to learn them.

Chapters Revisiting the Six Stages of Skill Acquisition · Sources of Power

For

Experienced fire chiefs recognise a situation instead of weighing up options Sources of Power

Experts test one option by imagining it play out, not by comparing Sources of Power

Klein says formal weighing of options is what beginners do, not experts Sources of Power

People can picture only about three moving parts in their head Sources of Power

Dreyfus says experts get good by dropping rules, not collecting them Revisiting the Six Stages of Skill Acquisition

supported2 sourcesClaims2 evidence1 opinionTraditions1 philosophy1 empiricalUnderAn explanation moves agreement not judgement

An explanation built to satisfy people is aimed at sounding right, not at showing where the model is wrong

An account extracted from a trained model afterwards costs no accuracy and says nothing about how the model works, so an opaque system supplies one as readily as a clear one. Lipton's sharpest case is Krening's second model, trained to produce verbal accounts of a reinforcement learner by fitting the explanations human players gave, which need not describe the agent's decisions however plausible it sounds. Saliency maps hold only around one image, and Adebayo showed in 2018 that several widely used methods produce much the same map after a model's weights are randomised. Vaccaro's meta-analysis of more than 300 effect sizes found neither explanations nor confidence scores improved teamwork. Lipton's half is argument rather than measurement.

Chapters When Combinations of Humans and AI Are Useful · The Mythos of Model Interpretability

For

An explanation made to satisfy people is aimed at plausibility, not accuracy The Mythos of Model Interpretability

Highlighting which pixels mattered explains one image, not the model The Mythos of Model Interpretability

Showing explanations or confidence scores did not improve human-AI teamwork When Combinations of Humans and AI Are Useful

supported2 sourcesClaims2 evidenceTraditions2 empiricalUnderChange the arrangement not the personReliability removes the checkHanding over returns as checking

An operator with nothing else to do catches nearly every failure, and one with a second job does not

In the 1993 experiment the same engine malfunctions were caught about 97 per cent of the time by people running only that task, and 33 per cent by people also flying and managing fuel by hand. Parasuraman, Molloy and Singh ran three tasks at once and found detection averaged 82 per cent when the automation's reliability fluctuated against 33 per cent when it held constant. Lee and See report the same split, replicated with experienced pilots and with rare faults, and found that ten minutes of manual control brought detection back up. The cause here is competing load rather than trust, which matters because the remedy differs. Only two claims support it, both from the same literature.

Chapters Trust in Automation: Designing for Appropriate Reliance · Complacency and Bias in Human Use of Automation

For

Operators alone caught nearly every fault and busy operators caught a third Complacency and Bias in Human Use of Automation

Watching for failures stops working when the watcher has other jobs Trust in Automation: Designing for Appropriate Reliance

supported2 sourcesClaims2 evidenceTraditions2 designUnderThe failure shows downstream of its causeReliability removes the check

The errors that do damage look like correct work and come from steps done without attention

Norman separates a slip, the right intention carried out wrongly at the late steps of acting, from a mistake, which is the wrong intention formed while planning. Experts slip more often, precisely because practised action runs without conscious attention. Mollick supplies the machine version, that the errors which do damage are the small plausible ones surviving a careful reading, and his own three paragraphs about the Schwartz case, written with web access, were wrong in three small ways. This is the thinnest of these theses, two sources sharing a mechanism but not a subject, and Mollick's case is a single demonstration on one passage.

Chapters The Design of Everyday Things · Co-Intelligence

For

Norman separates doing the wrong thing from intending the wrong thing The Design of Everyday Things

Mollick found the AI's small believable errors survive a careful reading Co-Intelligence

supported2 sourcesClaims2 evidence1 contestedTraditions2 empiricalUnderAsk the same question twiceChange the arrangement not the person

The same expert judging the same case twice gives two different answers

A professional is not the same measuring instrument on two occasions. Wine judges scored 18 per cent of the same wines identically on a second tasting, software developers estimating the same task on two days differed by 71 per cent, and opioids were prescribed more often late in the day across nearly seven hundred thousand primary care visits. Two underwriters at one insurer, pricing the same file, differed by a median of 55 per cent, against the 10 per cent their executives had predicted. Sapolsky adds that the frontal cortex tires first, so timing shifts judgement, but that mechanism is listed as contested and controlling sleep and time of day does not drain the spread.

Chapters Behave · Noise

For

The same professional judges the same case differently on another day Noise

Underwriters pricing the same risk differed by 55 per cent Noise

The brain's decision-making region tires first so timing changes judgement Behave

supported2 sourcesClaims2 evidence2 opinionTraditions2 empiricalUnderThe inside is not readable from the outsideA check needs the skill it checks

You cannot tell whether someone checked from what they looked at or what they produced

Duley laid the automated engine display over the manual tracking task, so that every look at the tracking task also fell on the automated one, and tracking stayed as good as before while the rate of missed malfunctions did too. What the operator withdraws is attention, not gaze. In the same review about 43 per cent of operators missed a first wrong diagnosis, and half of those had looked at every reading and spent no extra time on the one that did not fit. Lee's authors make the other half of the argument, that an unchanged output may be the result of a careful decision that nothing needed changing. Half of these claims are argument rather than measurement.

Chapters Complacency and Bias in Human Use of Automation · The Impact of Generative AI on Critical Thinking

For

An unedited output can be the result of a careful judgement The Impact of Generative AI on Critical Thinking

An aid that hides its inputs makes under-checking impossible to measure Complacency and Bias in Human Use of Automation

Putting the automated display in plain sight did not restore monitoring Complacency and Bias in Human Use of Automation

Half who followed the wrong diagnosis had looked at every reading Complacency and Bias in Human Use of Automation

supported2 sourcesClaims1 evidence1 opinionTraditions1 empirical1 contemplativeUnderHow it feels is evidence about the personJudging your own answer is a skillA hunch is worth its feedback

A little learning makes people confident, and more of the same learning makes them worse

Tetlock found that hedgehogs, experts who explain everything through one big idea, forecast less accurately inside their own specialty than outside it. More information poured through a single lens buys confidence and no extra truth. The Upanishads say the same without measurement, since the Katha warns not against ignorance but against the confidence of a little learning, describing the blind led by the blind and denying that reasoning alone settles the question. Two claims and two sources, one of them contemplative and unscoreable. Klein's evidence runs the other way where cues are valid and feedback is fast, and neither source tells you which sort of field you are standing in.

Chapters Superforecasting · उपनिषद् · The Upanishads

For

Single-idea experts forecast worst in their own specialty Superforecasting

The Upanishads warn about half-learned confidence, not ignorance उपनिषद् · The Upanishads

supported2 sourcesClaims1 evidence1 contestedTraditions2 empiricalUnderA result outruns the design that produced it

A paper's famous conclusion often survives after the method behind it fails

Two cases carry this. Of the three methods Hoffmann used, the one that leaned hardest on fitting the data did not survive a later re-check, while the conclusion the paper is remembered for did, and the correction moved that third route into agreement with the other two. Damasio's broad position, that emotion sits inside reasoning rather than outside it, is now the standard view, but the bodily-marker machinery he built it on has thinned as criticism of his card game accumulated. Two claims and two sources is thin, one of them contested, and the audit worked from published figures rather than the original data.

Chapters Training Compute-Optimal Large Language Models · Descartes' Error

For

One of the paper's three methods failed audit but its answer held Training Compute-Optimal Large Language Models

Damasio's general point was accepted but his specific mechanism was not Descartes' Error

supported2 sourcesClaims1 evidence1 opinion2 contestedTraditions2 empiricalUnderHow it feels is evidence about the personA check needs the skill it checksSkill lives only where it is exercised

A tool weakens the skill it replaces while the person's own estimate of that skill stays put

Risko and Gilbert's model has two arrows, one from offloading to the skill and one from offloading to the judgement of the skill, so reliance grows with nothing in the person's own report registering the loss. Turn-by-turn navigation makes the journey better and the driver worse. People who searched online about one subject credited themselves with more knowledge of a different subject, a transfer that only an inflated self-estimate explains. Shen and Tamkin found the group using an AI assistant fell furthest behind on debugging questions, which is the fault-finding that checking a tool needs. That breakdown was exploratory rather than planned, and two of the four claims are contested.

Chapters Cognitive Offloading · How AI Impacts Skill Formation

For

A tool weakens the skill it replaces and your judgement of it Cognitive Offloading

Turn-by-turn navigation makes the journey better and the driver worse Cognitive Offloading

People who can search online think they know more than they do Cognitive Offloading

Programmers using an AI assistant lost most ground on debugging questions How AI Impacts Skill Formation

contested2 sourcesClaims1 evidence1 opinion1 contestedTraditions1 empirical1 philosophyUnderThe inside is not readable from the outside

Outsiders can only learn what a model does by testing it, and words like wants must predict what it does

Hagendorff argues that studying what a model does when you use it is a proper explanation, because its outputs cannot be predicted from what is known about its weights, and for a closed commercial model it is the only route open to outsiders. He also defends describing a model as believing or wanting something, on the grounds that such words explain its behaviour better than a mechanical account even when nothing inside matches them. Binz and Schulz supply the evidence that the method outlasts the score, since their GPT-3 result was stale within a year while the design survived. Against it stands the finding that a single phrasing measures nothing, because answers shift with wording and position.

Chapters Using Cognitive Psychology to Understand GPT-3 · Machine Psychology

For

Hagendorff says watching what a model does is a real explanation Machine Psychology

Hagendorff says words like believes and wants really do explain machines Machine Psychology

The GPT-3 result went stale within a year but the method lasted Using Cognitive Psychology to Understand GPT-3

Against

One phrasing of a question measures nothing about a model Machine Psychology

supported2 sourcesClaims1 evidence1 opinionTraditions1 empirical1 philosophyUnderAn explanation moves agreement not judgement

People do not use information just because someone puts it in front of them

Two studies sit behind this. Dennett reports that people told not to use a briefly flashed word obey when the word is shown for half a second, but at fifty milliseconds and masked they say they saw nothing and are more likely to give that very word. Buccinca and colleagues find the same thing at the screen, where people read an AI's explanation as a sign the machine knows its job rather than reading the explanation itself, which is why a better explanation does not produce a better decision. That second half is inferred from the pattern of results rather than observed, and a 2023 study found explanations do cut over-reliance where checking is cheaper than deferring.

Chapters To Trust or to Think · From Bacteria to Bach and Back

For

People avoided a word they saw and gave it when masked From Bacteria to Bach and Back

People read an explanation as proof of competence instead of checking it To Trust or to Think

supported2 sourcesClaims1 evidence1 opinionTraditions2 empiricalUnderThe first answer becomes the answer

The first number or starting point a judge sees moves the final answer without them noticing

Two claims sit here. Kahneman's is that people judge an outcome as a gain or a loss measured from a starting point rather than as a level of wealth, and that a loss feels about twice as heavy as a gain of the same size. Tetlock turns the same fact into a working order, to ask how often such things happen in general before looking at the particular case, because whichever number you meet first is where your estimate settles. Neither half is strong alone. Tetlock's ordering has never been tested against the reverse, and the endowment effect that follows from Kahneman's kink does not appear among experienced traders.

Chapters Thinking, Fast and Slow · Superforecasting

For

People judge outcomes as gains or losses from where they started Thinking, Fast and Slow

Tetlock says look at the general rate before the specific case Superforecasting

supported2 sourcesClaims1 evidence1 opinionTraditions2 empiricalUnderReliability removes the checkSkill lives only where it is exercised

The more reliable a machine is, the less people watch it, and small checks are the practice

Parasuraman and Manzey describe the loop. Automation is usually reliable, so watching it less usually costs nothing, and the absence of a cost makes the operator watch less still. Only an experienced failure reverses it, and operators who met rare failures in training sampled the aid more than operators merely told it might fail. Lee's survey supplies the second half, where 55 of 319 knowledge workers said a task was too trivial to be worth checking and only 13 checked anything in order to keep a skill, so the consequential tasks arrive unrehearsed. The loop itself is borrowed from other work rather than measured, and nothing in Lee's single snapshot shows judgement actually decaying.

Chapters Complacency and Bias in Human Use of Automation · The Impact of Generative AI on Critical Thinking

For

Nothing punishes inattention while the machine works, so it grows Complacency and Bias in Human Use of Automation

The tasks that seem too small to check are the practice The Impact of Generative AI on Critical Thinking

weak2 sourcesClaims2 opinionTraditions1 philosophy1 contemplativeUnderWhat you are identified with is invisible from insideReasons are reconstructed after the fact

A mind cannot watch itself working, so the verdict on its own judgment has to come from outside

Minsky's argument is mechanical. A programmer debugging a program stops it first, and a brain running many processes at once cannot, because freezing one changes what you were studying and freezing all of them hides the interaction. The Yoga Vasistha makes the same point from the other end, when Vyasa's son Suka has worked the truth out alone and cannot call it finished until king Janaka hands the same conclusion back in one sentence. Both members are opinion rather than measurement, one an argument from how hardware differs from brains and the other a single invented case with its moral supplied, which is why the note counts as weak.

Chapters The Society of Mind · योगवासिष्ठ · The Yoga Vāsiṣṭha

For

A mind cannot stop itself to look at what it is doing The Society of Mind

Suka knew the answer already and needed a king to say it योगवासिष्ठ · The Yoga Vāsiṣṭha

weak1 sourceClaims5 evidence2 opinion1 contestedTraditions1 empiricalUnderHow it feels is evidence about the personA hunch is worth its feedback

Damasio says a gut feeling cuts the options down before reasoning starts, and people who lose it decide badly

A feeling attached to an imagined outcome arrives before the options are weighed, and it drops most of them. Damasio's patient Elliot scored superior or average on every intelligence test and still spent half an hour listing reasons for and against two appointment dates without choosing. Eleven further patients with the same prefrontal damage showed the same pairing of lost feeling and lost judgement, and healthy card players sweat before a risky deck while these patients do not. The general point was accepted but the particular mechanism was not; several of the card-game results have been reassigned since, and that dispute is the claim standing against.

Chapters Descartes' Error

For

A patient passed every reasoning test and still could not choose Descartes' Error

Damage to one brain area ruins judgment and leaves intelligence untouched Descartes' Error

Damasio's patients lose decision-making and feeling from the same brain damage Descartes' Error

Damasio says a gut feeling narrows the options before reasoning starts Descartes' Error

Damasio says weighing every cost and benefit will not fit in memory Descartes' Error

Damasio's patients still felt penalties but could not learn to expect them Descartes' Error

Damasio's patients understood upsetting pictures and felt nothing at all Descartes' Error

Healthy players sweat before a risky card but brain-damaged patients do not Descartes' Error

Against

Damasio's general point was accepted but his specific mechanism was not Descartes' Error

weak1 sourceClaims5 evidence2 opinionTraditions1 empiricalUnderHow it feels is evidence about the personJudging your own answer is a skillA result outruns the design that produced it

Whoever lacks the skill to get an answer right also lacks the skill to see it is wrong

Kruger and Dunning ran the studies this rests on. Across humour, logic and grammar their bottom-quarter students placed themselves at or above the middle, a 12th-percentile performance rated at the 62nd, and asked about raw scores rather than rank they claimed 14.2 of 20 logic items against an actual 9.6. The same students could not tell good work from bad in other people's answers, so watching better performers taught them nothing. The support is weak, since it is one source, and the crossing-lines graph everyone remembers can be produced by regression to the mean plus the general tendency to rate oneself above average. The rest of the evidence survives that criticism.

Chapters Unskilled and Unaware of It

For

People who lack a skill cannot tell that their answers are wrong Unskilled and Unaware of It

Poor performers cannot spot good work in others either Unskilled and Unaware of It

Kruger and Dunning's weakest performers rated themselves near the middle Unskilled and Unaware of It

Kruger and Dunning's weakest students overestimated their raw scores too Unskilled and Unaware of It

Everyday life rarely tells poor performers that they are poor performers Unskilled and Unaware of It

Top performers underrate their rank because they assume others did well Unskilled and Unaware of It

The Dunning-Kruger effect holds only where the skill is knowing things Unskilled and Unaware of It

Against

The famous Dunning-Kruger graph is mostly a statistical artefact Unskilled and Unaware of It

weak1 sourceClaims4 evidenceTraditions1 empiricalUnderAsk the same question twiceChange the arrangement not the person

Disagreement between professionals on the same case can be measured without knowing the right answer

Seen from behind a target with the bull's-eye hidden, a team that is biased looks accurate while a scattered team is still obviously scattered. Scored as mean squared error, a set of judgments has two independent parts of equal weight, how far off they are on average and how far apart they are, so the spread can be cut without knowing the bias. Most of that spread is each person's own stable pattern rather than general severity, and among underwriters differences in average premium were 20 per cent of the noise while the patterns were 80. Everything here rests on one source, which is why the note counts as weak.

Chapters Noise

For

Noise can be measured without knowing the right answer Noise

Overall error is the bias squared plus the noise squared Noise

Most of the noise is one person's pattern, not their severity Noise

On a scale with no ceiling, the disagreement is about the scale Noise

weak1 sourceClaims4 evidenceTraditions1 empiricalUnderChange the arrangement not the person

How much a person likes hard thinking is a steady trait apart from ability, and it predicts behaviour weakly

Cacioppo and Petty's measure correlates .24 with verbal intelligence and -.03 with abstract reasoning, so liking hard thought is not the same as being good at it. The eighteen-item questionnaire measures one thing, since the short form takes 37 per cent of the variance on its first factor and the one serious attempt to split it into three failed. The behavioural differences are real and small, about a fifth to a half of a standard deviation across recall, information seeking and enjoyment. Paying people for hard thinking reduces it in exactly the people who liked it. Against it, low scorers do examine an argument when the speaker is untrustworthy, and the note rests on one source.

Chapters Dispositional Differences in Cognitive Motivation

For

Enjoying hard thinking is about wanting to, not being clever Dispositional Differences in Cognitive Motivation

Enjoying hard thinking is one trait, not several separate ones Dispositional Differences in Cognitive Motivation

People who enjoy thinking behave differently but only a little Dispositional Differences in Cognitive Motivation

People who enjoy hard thinking lose that enjoyment when they are paid Dispositional Differences in Cognitive Motivation

Against

People who avoid hard thinking will do it given a reason Dispositional Differences in Cognitive Motivation

weak1 sourceClaims3 evidenceTraditions1 empiricalUnderHow it feels is evidence about the personJudging your own answer is a skill

Trying harder to read another person raises your confidence but not your accuracy

Epley's evidence is that confidence in reading another mind far outruns accuracy, and that longer acquaintance raises only the confidence. Dating partners predicted each other's answers 44 per cent of the time and believed they had got 82 per cent right. Imagining yourself in someone's position does not help and can hurt, because it runs your own mind inside a stereotype. People who switched to the method that halved their errors felt no more confident than before, so confidence cannot tell you which method is working. Support is weak at one source, and the accuracies are modest rather than zero, so the gap is the finding rather than a failure of mind reading.

Chapters Mindwise

For

Imagining another person's view does not improve accuracy and can worsen it Mindwise

Knowing someone longer raises confidence but not accuracy at reading them Mindwise

Switching to a method that halves your errors does not feel different Mindwise

supported5 sourcesClaims3 evidence3 opinion3 contestedTraditions3 philosophy2 empiricalUnderRemove the junior and lose the expertNothing helps in itself it has to fit the receiverSome of the job cannot be taughtSkill lives only where it is exercised

Beyond competence people improve by reorganising what they know rather than adding rules, and courses cannot take them there

Past the stage the Dreyfus brothers call competence, a learner stops collecting rules and starts responding to whole situations, and that cannot be handed over in a course. Klein's fireground study is the strongest number: of 156 decision points from non-routine incidents, 127 were recognitional and only 18 involved comparing options, and half the comparisons came from one fire staffed by novices. Papert's conservation result makes the same point about children, whose largest step adds no new skill, only a better way of managing old ones. Three of the nine claims are contested and Klein's is also filed against. Sweller keeps explicit instruction all the way up, with randomised trials behind him.

Chapters Revisiting the Six Stages of Skill Acquisition · Peak · Sources of Power · The Reflective Practitioner · The Society of Mind

For

Dreyfus says experts get good by dropping rules, not collecting them Revisiting the Six Stages of Skill Acquisition

Courses can take a learner to competence but no further Revisiting the Six Stages of Skill Acquisition

Competent practitioners judge quality by criteria they cannot state The Reflective Practitioner

Dreyfus says caring about the outcome is how skill gets stored Revisiting the Six Stages of Skill Acquisition

Most fields lack the clear measures and teachers deliberate practice requires Peak

Papert says growth can be better management of skills already possessed The Society of Mind

Experienced fire chiefs recognise a situation instead of weighing up options Sources of Power

Experience only builds expertise in fields that are steady and give feedback Sources of Power

Experts often react to something that did not happen Sources of Power

Against

Experienced fire chiefs recognise a situation instead of weighing up options Sources of Power

supported4 sourcesClaims6 evidence1 opinion2 contestedTraditions2 philosophy2 empiricalUnderNothing helps in itself it has to fit the receiver

Support that helps a beginner starts to hurt an expert, because what helps depends on what you already know

Difficulty is a relation between the material and the reader rather than a property of the page, so the same help changes sign as knowledge grows. Sweller's expertise reversal effect, named in 2003, is the replicated result: worked examples and other supports stop being necessary and then start to hurt. The mechanism is that working memory holds only a few new items but an unlimited amount of what is already familiar, which is why Chase and Simon's chess master replaces two-thirds of a real position after five seconds and two or three pieces of a random one. The reversal is established for lessons in well-structured subjects, and extending it to a tool used for years is an extrapolation.

Chapters Peak · Cognitive Architecture and Instructional Design: 20 Years Later · From Bacteria to Bach and Back · The Reflective Practitioner

For

Learners taught the solution first beat those left to work it out Cognitive Architecture and Instructional Design: 20 Years Later

Sweller found teaching aids that help beginners start to hurt experts Cognitive Architecture and Instructional Design: 20 Years Later

Working memory is tiny for new material, vast for familiar material Cognitive Architecture and Instructional Design: 20 Years Later

How complex something is depends on who is reading it Cognitive Architecture and Instructional Design: 20 Years Later

Sweller says only subject knowledge can be taught, not general thinking skills Cognitive Architecture and Instructional Design: 20 Years Later

Groups learn demanding material better and simple material worse than individuals Cognitive Architecture and Instructional Design: 20 Years Later

Armed forces produce reliable mechanics by drill and understanding comes after From Bacteria to Bach and Back

A chess master's memory advantage disappears on a randomly arranged board Peak

Children who believed blocks balance in the middle called the others impossible The Reflective Practitioner

supported4 sourcesClaims3 evidence2 opinionTraditions3 empirical1 designUnderWhich part of the task went decides what is lostThe claim on the result is what bindsHanding over returns as checking

Handing over the making of something does not hand over the responsibility, so checking becomes a permanent cost

The work of producing the thing moves to the machine and the answering for it does not, so verification stops being a stage and becomes the job. Lee's survey of 936 real tasks found the displacement plainly: gathering information became checking it, and doing a task became overseeing one. Vaccaro's review of 106 experiments found only three that gave the person and the machine separate parts of the work in advance, so nobody has really tested the alternative. Bansal's table adds that in all eight studies offered as evidence for explanations, the model alone beat the human-machine team. Lee's counts are self-reports and cannot be netted against each other.

Chapters Does the Whole Exceed its Parts? · When Combinations of Humans and AI Are Useful · Co-Intelligence · The Impact of Generative AI on Critical Thinking

For

The effort moved from producing the work to verifying and steering it The Impact of Generative AI on Critical Thinking

The production was delegated and the accountability was not The Impact of Generative AI on Critical Thinking

Never hand work to AI unchecked unless a machine verifies it Co-Intelligence

Human plus AI must beat both the human and the AI alone Does the Whole Exceed its Parts?

Almost no experiment split the task between person and machine beforehand When Combinations of Humans and AI Are Useful

supported4 sourcesClaims2 evidence2 opinionTraditions2 empirical1 philosophy1 designUnderDirect by comparing not by watchingAn explanation moves agreement not judgementChange the arrangement not the personReliability removes the check

Oversight written into a process goes unused and only being answerable for the result changes what people check

A rule saying a person may overrule the system does not make them do it. Toronto's officials never once overruled the model that kept beaches open on sixty-four per cent of unsafe days. Parasuraman and Manzey tested the obvious fixes and found nothing worked: neither a briefing on how decision aids fail nor an instruction to verify changed error rates in 181 non-pilots or 48 glass-cockpit pilots, and a second crew member changed nothing either. Only participants who expected to justify their accuracy made fewer errors. That accountability was explaining yourself to an experimenter, and Edmondson finds accountability without safety produces silence rather than checking.

Chapters Does the Whole Exceed its Parts? · AI Snake Oil · Complacency and Bias in Human Use of Automation · The Mythos of Model Interpretability

For

Officials with the power to overrule a model never used it AI Snake Oil

Telling operators to check the aid changed nothing; being held accountable did Complacency and Bias in Human Use of Automation

AI explanations make people agree rather than think, emptying their sign-off Does the Whole Exceed its Parts?

An explanation you cannot contest leaves the decision exactly where it was The Mythos of Model Interpretability

weak4 sourcesClaims4 opinionTraditions2 design2 empiricalUnderA smooth report is filteringThe failure shows downstream of its causeCorrection lands on the person

Finding someone to blame ends the inquiry that would have prevented the next failure

Naming a culprit answers the question the investigation was asking, so the asking stops before it reaches the conditions that produced the failure. Norman's case is the 2010 F-22 crash, where the Inspector General raised oxygen deprivation and the Air Force restated pilot error. Edmondson shows the size of the habit: managers say one to four per cent of their organisation's failures are truly blameworthy, and that seventy to ninety per cent get treated that way. None of the four claims counts as evidence, and Edmondson's figures come from polling her own audiences rather than from records. Norman adds that asking why implies one cause where complex events have many.

Chapters The Design of Everyday Things · The Fearless Organization · Who Sees Human? · Determined

For

Investigators stop asking why as soon as they can blame a person The Design of Everyday Things

When people treat a machine as a mind its makers escape blame Who Sees Human?

Managers blame far more failures than they think deserve blame The Fearless Organization

Sapolsky says we may restrain dangerous people without saying they deserve it Determined

supported3 sourcesClaims7 evidence1 opinionTraditions2 design1 philosophyUnderRemove the junior and lose the expertSkill lives only where it is exercised

A tool that lets experts work without juniors removes the work that made juniors able to check it

The machine does not take the expert's job, it makes the junior optional, and the junior's work was how competence got built. Beane watched ninety-four operations across five hospitals: residents did surgical work for almost the whole of an open procedure and ten to twenty times less of a robotic one, where the surgeon sits alone at a console. Mollick makes the same point about entry-level work in law and software. All eight claims lean one way and none is filed against, but the evidence is one speciality, observed rather than randomised. Beane's bomb-disposal teams, who put the controller in a trainee's hands, show that deployment decides this and not the machine.

Chapters Revisiting the Six Stages of Skill Acquisition · The Skill Code · Co-Intelligence

For

Machines take the beginner's work first, and beginners need it to learn Co-Intelligence

Machines threaten apprentices by making experts productive without them The Skill Code

Surgical trainees do ten to twenty times less with robotic surgery The Skill Code

Robotic surgery lets in only surgeons who are already skilled The Skill Code

The surgical residents who learned most broke the training rules The Skill Code

How an organisation deploys a tool decides whether users learn from it The Skill Code

Beane says people learn hard skills beside an expert, not in courses The Skill Code

A beginner given finished answers learns no rules and collects no cases Revisiting the Six Stages of Skill Acquisition

supported3 sourcesClaims7 evidence3 opinion1 contestedTraditions2 philosophy1 designUnderWhat you are identified with is invisible from insideA smooth report is filteringChange the arrangement not the personCorrection lands on the person

How safe people feel to speak up is a team trait and decides whether what they know reaches anyone

Two teams in one company, under the same policies, can differ sharply, so the unit that matters is the group rather than the person or the firm. Edmondson's study of neonatal teams at twenty-three hospitals shows the mechanism: how safe people felt predicted how much they asked and worked things out together, and had no relationship at all with how much they read on their own. Her better-performing hospital units reported more errors, not fewer. At Rustenburg mine, where explaining had produced silence, a change to open questions came alongside a fall in deaths from forty-four in 2006 to seventeen in 2011. Much of this literature is cross-sectional, and many things changed at Rustenburg at once.

Chapters The Fearless Organization · In Over Our Heads · Immunity to Change

For

Feeling safe to speak up is a trait of teams, not companies The Fearless Organization

Feeling safe to speak up matters most on uncertain, shared work The Fearless Organization

Feeling safe to speak up predicts learning from colleagues, not from reading The Fearless Organization

Disagreement and mixed expertise help safe teams and hurt unsafe ones The Fearless Organization

Better hospital teams reported more errors, not fewer The Fearless Organization

Punishing a clear violation is fair; vague rules are what silence people The Fearless Organization

Thanking someone before judging their point decides if they speak again The Fearless Organization

Open questions get people talking where explanations produced silence The Fearless Organization

Telling people to speak up fails because silence pays them better The Fearless Organization

A group's hidden commitments do nothing until someone names them aloud Immunity to Change

Kegan says a demand made without support is a punishment In Over Our Heads

supported3 sourcesClaims5 evidence2 opinionTraditions3 empiricalUnderChange the arrangement not the personThe first answer becomes the answer

Changing how a group decides works where training its people to be less biased does not

You cannot train the bias out of a person, and Kahneman says decades of study left his own judgement as overconfident as before, but you can change what the group has to do. Noise supplies the strongest number: American sentencing guidelines cut the expected gap between two judges from 4.9 months to 3.9, and after the Supreme Court struck them down the penalty for drawing a harsh judge doubled and the racial gap widened. Premortems and reference-class forecasting work the same way. The guidelines were repealed anyway, which is the honest limit, and the debiasing failure is Kahneman's concession rather than a trial.

Chapters Thinking, Fast and Slow · Conditions for Intuitive Expertise · Noise

For

Training yourself out of bias barely works, changing the procedure does Thinking, Fast and Slow

Noise has no direction, so it must be prevented rather than corrected Noise

A premortem gets people to say the doubts they were suppressing Conditions for Intuitive Expertise

Discussion makes a group noisier and more extreme than its members Noise

Better estimates come from looking up how similar past projects went Thinking, Fast and Slow

Sentencing guidelines cut the gap between judges and were repealed anyway Noise

Kahneman and Klein list five conditions a usable formula must meet Conditions for Intuitive Expertise

supported3 sourcesClaims5 evidenceTraditions3 empiricalUnderSkill lives only where it is exercisedThe first answer becomes the answer

The way of working that feels harder while you do it teaches you more

Ease during practice and learning afterwards run in opposite directions, so the feeling is a bad guide to what is working. Ninety per cent of students did better after spaced practice while seventy-two per cent said cramming worked better for them. Roediger and Karpicke found recall beat re-reading, sixty-eight per cent against fifty-four at two days, with the study-only group forgetting half of what it could first recall against a tenth for the tested group. Bucinca extends it to AI advice, since making people answer first cut their following of wrong suggestions from sixty-four per cent to forty-eight. Sweller's result is filed against, and the rule does not mean any difficulty helps.

Chapters To Trust or to Think · Make It Stick · Know Thyself

For

Students learn more from the study method that feels worse Know Thyself

Mixing up practice teaches you to tell cases apart but feels worse Make It Stick

Pulling an answer from memory teaches more than reading it again Make It Stick

Trying to answer before you are told beats being told first Make It Stick

Making people answer first cut their following of wrong AI advice To Trust or to Think

Against

Learners taught the solution first beat those left to work it out Cognitive Architecture and Instructional Design: 20 Years Later

supported3 sourcesClaims3 evidence2 opinion1 contestedTraditions3 empiricalUnderWhich part of the task went decides what is lostSkill lives only where it is exercised

A tool that decides for you costs the ability it replaced and the loss shows only without it

Using a device that makes the decisions leaves nothing for the underlying skill to practise, so the skill fades where no ordinary measure looks. Dahmani and Bohbot tested fifty Montreal drivers: the more lifetime hours a person had spent under satellite navigation, the worse they did with no device at all, and thirteen retested three years later had declined in step with the hours logged in between. Jost made the same argument by design, marking students only on work done with the tools removed. The effects are small, the samples tiny, and the strongest follow-up rests on thirteen people. The retired-taxi-driver finding is filed against as well as for, since it measures brain volume and not navigation.

Chapters Peak · Language Models in a Programming Course · GPS and the Cognitive Map

For

Drivers with more lifetime GPS hours navigated worse without the device GPS and the Cognitive Map

Drivers retested after three years declined in step with their GPS hours GPS and the Cognitive Map

Heavy GPS users were worse at the strategy they still chose GPS and the Cognitive Map

Dahmani and Bohbot blame instructions that leave nothing to decide GPS and the Cognitive Map

Skill left by a course shows only in a task done unassisted Language Models in a Programming Course

Retired taxi drivers lost the brain tissue the job had built Peak

Against

Retired taxi drivers lost the brain tissue the job had built Peak

supported3 sourcesClaims3 evidence2 opinion2 contestedTraditions2 empirical1 designUnderRemove the junior and lose the expertA controlled setting is what gets automatedReliability removes the checkSkill lives only where it is exercised

Automation keeps the routine work and leaves people the rare hard cases they no longer get to practise

The tasks nobody could write a specification for are the ones left to a person, and those are the rare and abnormal ones. Bainbridge's Ironies of Automation is the source: routine work is what keeps a manual skill alive, so an operator who has only watched for months performs like a beginner when control comes back, and the more reliable the system the more training its operators need. Lee and See add that people given other work alongside the watching miss most failures. Bainbridge's paper is a review that cites its studies a sentence each, and two of the seven claims are filed against, including Ericsson's.

Chapters Ironies of Automation · Peak · Trust in Automation: Designing for Appropriate Reliance

For

Automation leaves people the jobs nobody could write down Ironies of Automation

An operator who only watches loses the skill to take over Ironies of Automation

The more reliable an automatic system the more operator training it needs Ironies of Automation

Skills fade when unused and grow only through practice with feedback Ironies of Automation

Bainbridge says watching a machine is stressful and the stress causes errors Ironies of Automation

Ericsson says hard practice with feedback builds skill, not years spent Peak

Watching for failures stops working when the watcher has other jobs Trust in Automation: Designing for Appropriate Reliance

Against

Ericsson says hard practice with feedback builds skill, not years spent Peak

Watching for failures stops working when the watcher has other jobs Trust in Automation: Designing for Appropriate Reliance

supported3 sourcesClaims3 evidence4 opinionTraditions1 design1 empirical1 philosophyUnderRemove the junior and lose the expertSkill lives only where it is exercised

The problems that matter lie where technique stops working and a finished demonstration hides the fumbling a learner needs

Schon divides practice into a high ground where technique works and a lowland of confusing messes it cannot solve, and puts the problems people care about in the second. His own case turns on him: Quist's studio demonstration is so accomplished that it corrects no errors and compresses out the iterations a student needs to see. Ross and Bagnell proved the machine version, where a learner given only clean demonstrations has no recoveries to copy and a million frames of expert play left the network crashing as often as two thousand had. Three of the seven claims are filed against, and Schon never says how to tell which ground you are standing on.

Chapters Sources of Power · The Alignment Problem · The Reflective Practitioner

For

The problems that matter most lie where technique stops working The Reflective Practitioner

A master's demonstration hides the fumbling the student needs to see The Reflective Practitioner

Copying an expert fails because experts never demonstrate getting out of trouble The Alignment Problem

Designers experiment on paper because nothing there is irreversible or expensive The Reflective Practitioner

Schön says setting the problem comes first and has no technique The Reflective Practitioner

Experts test one option by imagining it play out, not by comparing Sources of Power

Experts describe the signs they use only when asked about specific cases Sources of Power

Against

Schön says setting the problem comes first and has no technique The Reflective Practitioner

Experts test one option by imagining it play out, not by comparing Sources of Power

Experts describe the signs they use only when asked about specific cases Sources of Power

supported3 sourcesClaims2 evidence3 opinionTraditions2 design1 empiricalUnderA measure scored on stops measuring

A stand-in measure stops matching the real goal once whoever is measured gets good at chasing it

Because the goal cannot be written down exactly, someone rewards a stand-in for it, and the stand-in comes apart from the goal just as the chase gets skilful. Christian's The Alignment Problem supplies the cases: a boat that spins in the harbour collecting power-ups instead of racing, a bicycle that rides twenty-metre circles because moving away from the target was never charged. Grove's answer is a second number beside every first one, and Ng and Russell proved an added bonus stays safe only when it depends on where the agent is. The cases are simulated agents, and carrying them over to people rests on a shared pattern rather than a shared mechanism.

Chapters High Output Management · The Alignment Problem · Why Language Models Hallucinate

For

A stand-in measure fails just as the system gets good at it The Alignment Problem

Grove says every measure needs a second measure beside it High Output Management

Models learn the thing that was recorded not the thing you want The Alignment Problem

Extra rewards are safe only when they depend on where you are The Alignment Problem

A scoring rule with no credit for I don't know selects guessers Why Language Models Hallucinate

supported3 sourcesClaims1 evidence3 opinionTraditions2 design1 empiricalUnderWhat you are identified with is invisible from insideA smooth report is filteringCorrection lands on the person

A person cannot see the errors in his own work, but someone with nothing at stake can

Attachment to your own work hides its faults, so the reading has to come from someone with nothing invested in it. Weinberg's founding case is one afternoon of colleagues reading his code: seventeen errors in thirteen instructions, and no further error in nine years across a dozen installations. He claims the gain shows up as delivery dates that vary less rather than as work finishing faster. Beane adds that expertise narrows what anyone notices, since eighty-three per cent of radiologists hunting for lung nodules missed a picture of a gorilla forty-eight times larger. Only one of the four claims counts as evidence, and the gorilla is a search task rather than a professional judgement.

Chapters The Psychology of Computer Programming · The Skill Code · Noise

For

A programmer who owns his code cannot see its faults The Psychology of Computer Programming

Letting others read your code catches errors and steadies delivery dates The Psychology of Computer Programming

Radiologists hunting for a nodule missed a gorilla in the scan The Skill Code

Professionals rarely see each other's work, so the disagreement stays hidden Noise

supported3 sourcesClaims1 evidence3 opinion1 contestedTraditions2 design1 philosophyUnderRemove the junior and lose the expertDirect by comparing not by watchingNothing helps in itself it has to fit the receiver

How closely to supervise depends on the person's experience at that task and drops back when the task changes

Readiness to work unsupervised belongs to a task rather than to a person, so a proven worker needs close direction again when the job or the pace changes. Grove sets the dial by the delegate's experience at that particular task and says none of the three styles is kinder than the others, and he advises checking at the rough draft, where a mistake costs least. Dreyfus reports the same fall for experts whose field changes under them. Only one claim counts as evidence, Hadfield-Menell's proof that a machine unsure of its user's goal checks before acting. Grove's own support is two anecdotes, and nine managers in ten judged themselves more hands-off than their people did.

Chapters High Output Management · Revisiting the Six Stages of Skill Acquisition · The Alignment Problem

For

How closely to supervise depends on the person's experience at that task High Output Management

A proven worker needs close direction again when the job changes High Output Management

Check delegated work at the rough draft, not the finished version High Output Management

Experts go back to following rules when their field changes under them Revisiting the Six Stages of Skill Acquisition

A machine unsure of your goal asks first and accepts being stopped The Alignment Problem

supported2 sourcesClaims6 evidence2 opinion4 contestedTraditions2 empiricalUnderWhich part of the task went decides what is lostSkill lives only where it is exercised

Handing over the writing and the fault-finding costs the learning, but asking the assistant for reasons does not

What the person hands over decides the cost, not whether they use the tool at all. Shen and Tamkin gave fifty-two professional programmers thirty-five minutes on an unfamiliar Python library: those who had the assistant write the code scored 4.15 points of 27 lower on an unaided quiz, while the groups who asked it for reasons scored 65 to 86 per cent against 24 to 39 for those who delegated the writing. Jost's thirty-two students showed the same ordering. Four of the twelve claims are contested and three filed against, including the authors' warning that finding no cost for asking for explanations was a failure to detect one.

Chapters How AI Impacts Skill Formation · Language Models in a Programming Course

For

Students who had a model write or fix their code scored lower Language Models in a Programming Course

A student's overall model use predicted the unassisted exam mark Language Models in a Programming Course

Teach students to write and fix code before giving them a model Language Models in a Programming Course

Programmers who let AI write the code learned less about the library How AI Impacts Skill Formation

Asking the assistant why preserves learning better than not using it How AI Impacts Skill Formation

What you ask after the code arrives decides what you learn How AI Impacts Skill Formation

Programmers using an assistant hit fewer of the errors that teach How AI Impacts Skill Formation

Shen and Tamkin expect autocomplete to be faster and teach less How AI Impacts Skill Formation

The AI assistant hurt understanding and saved no time at all How AI Impacts Skill Formation

Programmers using an AI assistant lost most ground on debugging questions How AI Impacts Skill Formation

The students who used models most may have been struggling already Language Models in a Programming Course

The explanation result was a failure to detect, not a clean bill Language Models in a Programming Course

Against

Programmers using an AI assistant lost most ground on debugging questions How AI Impacts Skill Formation

The students who used models most may have been struggling already Language Models in a Programming Course

The explanation result was a failure to detect, not a clean bill Language Models in a Programming Course

supported2 sourcesClaims3 evidence2 opinionTraditions2 designUnderA controlled setting is what gets automatedThe failure shows downstream of its cause

An automated decision can fail because the data or the rules are wrong, and no better model fixes that

Checking outputs catches a model that is wrong about the world. It cannot catch a model that is right about bad records, or a disagreement about where the rule should fall. Christian gives the cases: Caruana's pneumonia model learned that asthma patients were low risk, true in the data only because they went straight to intensive care, and dropping race from the inputs leaves it in through address and prior convictions while removing any way to measure it. Narayanan and Kapoor add that moderation fights are policy disputes people must settle. The pneumonia case is fifteen thousand patients from one era of medicine, and their account of why institutions buy what fails is assembled from examples rather than tested.

Chapters The Alignment Problem · AI Snake Oil

For

A model can learn a deadly rule that its answers never reveal The Alignment Problem

Deleting race from a model's inputs hides bias rather than removing it The Alignment Problem

A risk score cannot be both accurate and equally wrong across groups The Alignment Problem

Narayanan and Kapoor say content fights are policy disputes, not AI errors AI Snake Oil

Institutions buy AI that does not work because they are already failing AI Snake Oil

supported2 sourcesClaims3 evidence1 opinionTraditions1 empirical1 contemplativeUnderThe inside is not readable from the outsideA smooth report is filtering

You cannot read what someone thinks from their behaviour and only asking them and repeating it back works

Watching someone and inferring what they think is unreliable, however confident it feels. Epley found the one method that improves it is asking the person and saying the answer back, and couples who interviewed each other halved their errors. His rigged quiz shows the failure, since the audience rated the question-setters at the eighty-second percentile and the contestants at the forty-ninth, and only the setters, who knew how many questions they had thrown away, rated the two sides equal. People also badly overstate group gaps, predicting thirty-five points where the measured party difference was 3.5. It works only where the other party will tell you, and there is nobody to ask when the delegate is a model.

Chapters Mindwise · योगवासिष्ठ · The Yoga Vāsiṣṭha

For

Epley found only asking people directly improves how well you read them Mindwise

People mistake an advantage of position for greater ability Mindwise

People name which group is higher but exaggerate the gap hugely Mindwise

Cudala's husband heard from a boy what he refused from his wife योगवासिष्ठ · The Yoga Vāsiṣṭha

supported2 sourcesClaims2 evidence3 opinionTraditions2 designUnderA controlled setting is what gets automatedThe failure shows downstream of its cause

Deciding exactly what to build is the hardest part and no tool or skill fixes a wrong choice

Brooks argues the hardest single part of building software is deciding exactly what it should do, and that clients cannot describe it correctly until they have used some version of it. Tools only ease the writing down, and his prediction that no one development would make software work ten times faster within a decade has now run four decades. Weinberg supplies the evidence from the other side, since his programmers did whatever the written instructions asked and a group told to finish fast produced programs about ten times slower. Hardest is never measured against anything, and Weinberg's experiment had four subjects in one shop and has not been repeated at scale.

Chapters The Psychology of Computer Programming · The Mythical Man-Month

For

Deciding exactly what to build is the hardest part of software The Mythical Man-Month

Brooks says no one tool will make programming ten times faster The Mythical Man-Month

Brooks withdrew his advice to plan on throwing the first build away The Mythical Man-Month

Brooks puts finished software at nine times the cost of a program The Mythical Man-Month

Weinberg found programmers do what the specification asks not what they prefer The Psychology of Computer Programming

supported2 sourcesClaims2 evidenceTraditions2 philosophyUnderA controlled setting is what gets automatedWhich part of the task went decides what is lost

What gets automated depends on how controlled the setting is, not on how hard the task looks

The same physical task falls on either side of the line depending on its surroundings. A car plant fits windscreens with robots, while replacing the same glass in a garage uses a technician, because removing a broken screen and preparing the frame demands adaptation no affordable robot manages. Amazon's Kiva robots automated the warehouse by moving shelves across a level floor while a person still does the picking. Minsky states it from the other side, since mathematical logic fell to a program in 1956 and college calculus in 1961 while what a four-year-old does resisted for decades. Both claims count as evidence, but the price of making a setting controlled is usually invisible and enormous.

Chapters The Society of Mind · Why Are There Still So Many Jobs?

For

A robot fits windscreens on the line and a person replaces them Why Are There Still So Many Jobs?

The problems a four-year-old solves were the hardest ones to program The Society of Mind

supported2 sourcesClaims1 evidence2 opinionTraditions2 designUnderWhat you are identified with is invisible from insideA smooth report is filtering

Every level of a reporting chain strips out bad news, so a smooth report shows the filtering

Each reviewer removes the extreme items on the way up, and the extremes are the only ones carrying news, so smoothness is what filtering looks like. Weinberg worked the dates backwards on one reporting chain and found a team's monthly submission fell due four days before the month it described began. His other case is the meeting where every leader reports being on schedule until one admits he is behind and the rest follow. Brooks supplies the structural version, since half a project's time goes on testing and the slip surfaces only then. Of three claims one counts as evidence, and Brooks later withdrew the schedule model his rule sits inside.

Chapters The Psychology of Computer Programming · The Mythical Man-Month

For

Weinberg says a smooth progress report shows filtering, not progress The Psychology of Computer Programming

When every team leader agrees the schedule is fine, they are bluffing The Psychology of Computer Programming

Half a software project is testing and the delay shows up last The Mythical Man-Month

supported2 sourcesClaims1 evidence1 opinionTraditions2 philosophyUnderNothing helps in itself it has to fit the receiverSome of the job cannot be taught

When a job asks for a different kind of mind, training and new skills cannot supply it

Training works on what a mind holds, while some demands are about how it is organised, and adding skills leaves that untouched. Kegan and Lahey give a test anyone can run, that a behaviour returning after honest effort is doing a job for the person. Their case is Peter, whose ten pounds stayed off for four years while the softened tone he had worked on came back, plus seven per cent, against an average dieter who regains 107 per cent of what was lost. Two claims from two books carry the whole thesis. The test is also backward-looking, and Ericsson holds that much of what is called a difference in kind yields to structured practice.

Chapters In Over Our Heads · Immunity to Change

For

Kegan says training cannot give someone a more complex mind In Over Our Heads

Kegan and Lahey say new skills cannot fix a returning habit Immunity to Change

weak2 sourcesClaims1 opinion3 contestedTraditions1 empirical1 designUnderNothing helps in itself it has to fit the receiver

AI lifts the weakest workers most, so the work looks alike and no longer shows who made it

If the tool raises the floor more than the ceiling, a finished piece stops telling you what its author can do. Dell'Acqua's consultant experiment found the gap between best and worst narrowing from twenty-two per cent to four, and Mollick reports call-centre workers in the bottom tenth gaining thirty-five per cent against almost nothing for experienced ones. His recommendation letter is the same point about signalling, since the hours it stood for were the message. Three of the four claims are marked contested. The homogenisation finding was dropped from the published version of Dell'Acqua's paper, and the levelling reversed among Kenyan entrepreneurs, where the strongest gained and the weakest did worse.

Chapters The Jagged Technological Frontier · Co-Intelligence

For

Weaker performers gain more from AI than stronger ones do The Jagged Technological Frontier

AI helps the weakest workers most, which could make skill worth less Co-Intelligence

A group using AI produces better ideas that are more alike The Jagged Technological Frontier

A document that costs nothing to write stops proving anything Co-Intelligence

weak2 sourcesClaims3 opinionTraditions2 designUnderThe failure shows downstream of its cause

The quality of the designers decides whether a system succeeds and organisations train managers instead

Brooks holds that how good the designers are is the largest single factor in whether a system works, and that organisations recruit and develop managers deliberately while doing nothing of the kind for designers. Boehm's cost model puts team quality first by about four times. Brooks's own costly experiment was handing the OS/360 external specifications to a hundred and fifty implementers, which he estimates added a year of debugging. Against it stands Weinberg's correction that the famous thirty-to-one spread between programmers came from comparing different tasks, so the gap is smaller than Brooks thought. No claim here counts as evidence and the book marks it weak.

Chapters The Psychology of Computer Programming · The Mythical Man-Month

For

Organisations develop managers on purpose and designers not at all The Mythical Man-Month

Brooks says one mind must own the shape of a system The Mythical Man-Month

Weinberg says a programmer nobody can replace is a danger The Psychology of Computer Programming

Against

The famous thirty-to-one gap between programmers came from comparing different tasks The Psychology of Computer Programming

weak2 sourcesClaims3 opinion2 contestedTraditions2 designUnderA result outruns the design that produced it

What defeats adding people to a project is the cost of coordinating them, not their individual ability

Each person added has to be trained by those already working, and the talking grows faster than the headcount, so the work does not divide. Brooks states it flatly: adding people to a late software project makes it later. Weinberg's rule of thumb, that three programmers do about twice the work of one, points the same way, and his correction of the famous thirty-to-one gap between programmers, which came from comparing the best person on one task with the worst on another, supplies the other half. No claim here counts as evidence, three are opinion and two contested, and Weinberg's numbers are asserted rather than measured.

Chapters The Psychology of Computer Programming · The Mythical Man-Month

For

Adding people to a late software project makes it later The Mythical Man-Month

Weinberg says three programmers do about twice the work of one The Psychology of Computer Programming

Brooks judges an organisation by the talking it makes unnecessary The Mythical Man-Month

Brooks admits Parnas was right that hiding information beats showing everything The Mythical Man-Month

The famous thirty-to-one gap between programmers came from comparing different tasks The Psychology of Computer Programming

weak1 sourceClaims5 evidenceTraditions1 empiricalUnderHow it feels is evidence about the personHanding over returns as checking

AI help slows experienced people most on work they know best while it still feels faster to them

Measured time and felt time move apart. In METR's randomised trial, sixteen experienced developers fixing real issues in codebases they had worked on for years took about nineteen per cent longer with AI help than without, while writing forty-seven per cent more code per hour, and the slowdown was largest where their own knowledge ran deepest. Economists and machine-learning researchers asked to predict the result guessed thirty-eight to thirty-nine per cent faster. Sixteen people is a thin base, the confidence interval reaches close to no effect, and the participants were picked for exactly the deep familiarity the authors name as the cause, which most work does not have.

Chapters Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity

For

Experienced developers took 19 per cent longer using AI on familiar code Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity

In METR's study developers lost most time on code they knew best Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity

Developers wrote more code in the condition where the work took longer Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity

Developers using AI spend a fifth of their time prompting and reviewing Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity

Experts predicting the trial were more wrong than the developers in it Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity

Against

Experts predicting the trial were more wrong than the developers in it Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity

weak1 sourceClaims5 evidence2 contestedTraditions1 empiricalUnderA measure scored on stops measuringPeople reshape the job you hand them

Why a person works predicts how long they keep going and how well, separately from how motivated they look

Two people who look equally keen can be headed in different directions, because what predicts persistence is the reason rather than the amount. Ryan and Deci's pattern repeats across settings: schools in four countries, a six-month supervised diet whose autonomy-supported patients had kept the weight off nearly two years later, smoking cessation confirmed by biochemical test at thirty months, and methadone patients' urine results. Managers can be trained to give staff more say, and a Fortune 500 study found subordinates reporting the change over thirteen months. Everything here comes from one source, the reasons are self-reported, and most of the field studies are cross-sectional.

Chapters The What and Why of Goal Pursuits

For

Why a person acts predicts persistence better than how motivated they are The What and Why of Goal Pursuits

Ryan and Deci say people need skill, connection and choice, all three The What and Why of Goal Pursuits

Without freedom to choose, people follow rules out of guilt The What and Why of Goal Pursuits

Training managers to support employees' choices changes what employees experience The What and Why of Goal Pursuits

What a bonus means to the person matters more than its size The What and Why of Goal Pursuits

Bonuses raise output on simple work and damage complex work The What and Why of Goal Pursuits

Paying people for work they already enjoy reduces their interest in it The What and Why of Goal Pursuits

weak1 sourceClaims2 evidenceTraditions1 philosophy

Computers cut middle-paying jobs and move work to both ends of the scale while pay does not follow

The jobs computers take sit in the middle of the pay scale, so employment grows at the top and the bottom, and the pay at the bottom does not rise with it. Autor's figures carry it: the four middle-skill American occupation groups fell from sixty per cent of employment in 1979 to forty-nine in 2007, and across sixteen European economies between 1993 and 2010 the middle-paying share fell in every single country. Real wages below roughly the fifteenth percentile fell between 1999 and 2007 even as those jobs grew. Both claims come from one author, so it stands on thin evidence, and a falling share is not a falling count.

Chapters Why Are There Still So Many Jobs?

For

Middle-paying jobs shrank in every one of sixteen European economies Why Are There Still So Many Jobs?

Jobs split into high and low paid but wages did not Why Are There Still So Many Jobs?

weak1 sourceClaims1 evidenceTraditions1 designUnderDirect by comparing not by watchingWhich part of the task went decides what is lost

A person can direct work they could not do themselves by comparing results instead of producing them

Judging between two finished attempts is easier than making one, so a person can push work toward what they want without being able to do it. The single claim behind this comes from Christian's The Alignment Problem: Christiano, Leike and Amodei's simulated leg learned a tucked backflip from a few hundred either-or choices, about an hour of one person's time, and nobody involved could have performed the move. One simulated robot carries the whole thesis, so it stands on thin evidence. The method also assumes the chooser knows what they want, and where all the choices were collected before training the agent kept the loopholes it found.

Chapters The Alignment Problem

For

A few hundred A-or-B choices taught a robot a backflip The Alignment Problem

contested1 sourceClaims1 evidence2 opinionTraditions1 philosophyUnderA controlled setting is what gets automatedNothing helps in itself it has to fit the receiver

Automating part of a job makes the rest worth more only where demand grows and entry stays hard

A job is a bundle of tasks, so making the machine's tasks cheap can raise the price of those left to a person, but only under conditions. Autor names three: the worker supplies the remaining task, others cannot easily enter it, and demand for the output grows as it gets cheaper. His case is the cash machine, which rose from about 100,000 to 400,000 in America between 1995 and 2010 while bank tellers rose from about 500,000 to 550,000. His own wage series is the counter-case, and the point is contested: at the bottom of the scale real wages fell after 1999 even as those jobs grew.

Chapters Why Are There Still So Many Jobs?

For

Automating part of a job raises the value of the rest Why Are There Still So Many Jobs?

Autor gives three conditions for automation to raise a worker's pay Why Are There Still So Many Jobs?

Cash machines quadrupled and the number of bank tellers went up Why Are There Still So Many Jobs?

Against

Jobs split into high and low paid but wages did not Why Are There Still So Many Jobs?

weak1 sourceClaims5 opinion1 contestedTraditions1 designUnderA measure scored on stops measuring

A manager's real output is what his teams produce, so work that lifts nobody else counts for nothing

Andrew Grove measures a manager by what his group delivers, not by how busy or how able he looks. Every claim here comes from Grove's High Output Management: rate no manager above his team's actual result, run the training yourself rather than hiring a trainer, set targets people reach about half the time. Grove derived the rule from running Intel and never tested it. It is an accounting identity rather than a measurement, since it cannot split one result among several influences, and a manager's work may surface years later. Five of the six claims are opinion and one is contested, with nothing filed as evidence.

Chapters High Output Management

For

Grove counts a manager's output as what his teams produce High Output Management

Grove rates no manager higher than the team's actual result High Output Management

Grove says managers should teach their own people, not hire trainers High Output Management

Grove says shared values are the only control when work is uncertain High Output Management

Grove sets targets a person reaches only half the time High Output Management

A group of equals stalls until someone senior joins the decision High Output Management

weak1 sourceClaims3 opinion1 contestedTraditions1 empiricalUnderChange the arrangement not the personPeople reshape the job you hand them

People reshape their own jobs to suit themselves and a manager can only make it likelier, never require it

Changing the shape of your own job is done for the worker, not the employer, and a line worker who changes tasks may raise the quality of the work or wreck the flow of it. The one lever Wrzesniewski can show working is meaning, since hospital cleaners used the stated mission to frame cleaning as care. Room to do it depends on the setting, so a cleaner or hairdresser has plenty and a closely watched telemarketer almost none. All four claims come from that single paper, three as opinion, so it stands on thin evidence. The follow-up studies are self-reported at one moment and cannot show which way the cause runs.

Chapters Crafting a Job

For

Job crafting helps the worker but not necessarily the employer Crafting a Job

Managers cannot order people to reshape their jobs, only make it likelier Crafting a Job

Watched and tightly scheduled workers have little room to reshape their job Crafting a Job

The job crafting studies cannot show which way the cause runs Crafting a Job

weak5 sourcesClaims10 opinion1 contestedTraditions5 contemplativeUnderWatching a state beats attacking it

Practice prepares a person and clears away a covering but cannot produce a state nobody made

Effort can get a person ready and take away what is in the way, but it cannot manufacture something that was never made. Shankara argues that action purifies the mind and stops there, that knowing the Self uncovers it rather than gets it, and that only those who do not yet know have duties. The Ashtavakra Gita goes further and calls practice itself another chain, and Rahula says nirvana is not an effect of the path. Eleven claims from five contemplative sources, all opinion and nothing measured. Chatterjee stands against it, holding that letting go of being the doer can be trained by regular reflection.

Chapters श्रीमद्भगवद्गीता शाङ्करभाष्यम् · The Gita with Shankara's Commentary · माण्डूक्योपनिषत् गौडपादकारिका · The Māṇḍūkya Upaniṣad and Gauḍapāda's Kārikā · अनत्तलक्खणसुत्त · Anattalakkhaṇa Sutta · अष्टावक्रगीता · Aṣṭāvakra Gītā · What the Buddha Taught

For

Shankara says action prepares the mind but cannot bring liberation श्रीमद्भगवद्गीता शाङ्करभाष्यम् · The Gita with Shankara's Commentary

Shankara says knowing the Self uncovers it rather than grasping it श्रीमद्भगवद्गीता शाङ्करभाष्यम् · The Gita with Shankara's Commentary

Shankara says only those who do not yet know have duties श्रीमद्भगवद्गीता शाङ्करभाष्यम् · The Gita with Shankara's Commentary

Shankara holds that knowledge and action cannot be combined in one person श्रीमद्भगवद्गीता शाङ्करभाष्यम् · The Gita with Shankara's Commentary

The Ashtavakra Gita says practice is another chain because you never acted अष्टावक्रगीता · Aṣṭāvakra Gītā

The Ashtavakra Gita says effort causes unhappiness and nobody notices अष्टावक्रगीता · Aṣṭāvakra Gītā

The Ashtavakra Gita defines bondage as the mind wanting or refusing अष्टावक्रगीता · Aṣṭāvakra Gītā

Ashtavakra says announcing I am brahman proves you are not finished अष्टावक्रगीता · Aṣṭāvakra Gītā

The Buddha says freedom comes from seeing clearly, not from trying hard अनत्तलक्खणसुत्त · Anattalakkhaṇa Sutta

Dreamless sleep is not liberation because the world regrows from a seed माण्डूक्योपनिषत् गौडपादकारिका · The Māṇḍūkya Upaniṣad and Gauḍapāda's Kārikā

Rahula says nirvana is not an effect of the path What the Buddha Taught

Against

Chatterjee says letting go of being the doer can be trained Timeless Leadership

supported4 sourcesClaims6 evidence2 opinionTraditions2 philosophy1 empirical1 contemplativeUnderHow it feels is evidence about the personThe frame decides what you find

What you meet is already interpreted, built from what you expected and one detail, and it changes with the setting

What reaches a person has already been worked on. Seth says you see the brain's best guess rather than the incoming signals, and Minsky says most of what you see is filled in from expectation and that what reaches awareness is a record of a moment already gone. Sapolsky supplies the evidence: faces are sorted by group before they are consciously seen, a split as thin as over- or under-estimating dots produces favouritism, and a hormone lowers a threshold rather than adding a behaviour, which is why an automatic reaction changes with the question asked. Eight claims from four sources, six of them evidence, none against.

Chapters उपनिषद् · The Upanishads · Being You · Behave · The Society of Mind

For

Seth says you see the brain's best guess, not the incoming signals Being You

Minsky says most of what you see is filled in from expectation The Society of Mind

What reaches awareness is a record of a moment already gone The Society of Mind

The Katha Upanishad ranks the mind above the senses and their objects उपनिषद् · The Upanishads

An automatic reaction to a face changes with the question asked Behave

People sort a face by group before they consciously see it Behave

A meaningless group split makes people favour their own side Behave

Sapolsky says a hormone lowers a threshold instead of adding a behaviour Behave

weak4 sourcesClaims3 opinion1 contestedTraditions3 contemplative1 philosophyUnderReasons are reconstructed after the factSkill arrives before understandingThe frame decides what you find

A mind watching itself changes what it watches and tends to find what the watcher already believed

Thompson's objection is that meditation trains and alters the mind while it observes it, so it cannot be the neutral instrument the comparison with science needs — Nyanaponika claimed both that bare attention follows the true scientist's procedure and that it makes the mind more plastic. The Mandukya Karika reaches the same place from the other side, listing how each school looking for the self found its own founding principle, breath or the elements or the void. Easwaran's Upanishads and the Satipatthana discourse both give a procedure with no outside check. None of this is measured, and schools disagreeing is still consistent with one of them being right, an objection that applies to the texts making it.

Chapters उपनिषद् · The Upanishads · माण्डूक्योपनिषत् गौडपादकारिका · The Māṇḍūkya Upaniṣad and Gauḍapāda's Kārikā · सतिपट्ठानसुत्त · Satipaṭṭhāna Sutta · Why I Am Not a Buddhist

For

Thompson says meditation alters the mind it is meant to observe Why I Am Not a Buddhist

Philosophers looking for the self each find what they already believed माण्डूक्योपनिषत् गौडपादकारिका · The Māṇḍūkya Upaniṣad and Gauḍapāda's Kārikā

The Upanishads offer a procedure to run, not a doctrine to believe उपनिषद् · The Upanishads

The Satipatthana discourse shows its method and never argues for it सतिपट्ठानसुत्त · Satipaṭṭhāna Sutta

weak4 sourcesClaims4 opinionTraditions4 contemplativeUnderThe claim on the result is what bindsConditions and unfinished work keep you going

A worker suffers from claiming the result, not from the work itself or from the world

The cost falls on the claim to the result, not on the work and not on the world. Vivekananda says wanting the result is the trap and the work is not, the Isha Upanishad tells you to enjoy what you do not own, and the Panchadasi says the world is not the obstacle and hating it is a mistake. Rahula makes the same cut from the Buddhist side, since karma means the act of will and never its result. Four claims from four contemplative sources, all opinion and none measured, and none of them says what an outside observer would see once the claim is dropped.

Chapters Karma-Yoga · उपनिषद् · The Upanishads · What the Buddha Taught · पञ्चदशी · Pañcadaśī

For

Vivekananda says wanting the result traps you, not the work itself Karma-Yoga

The Isha Upanishad tells you to enjoy what you do not own उपनिषद् · The Upanishads

Rahula says karma means the act of will and never its result What the Buddha Taught

The world is not the obstacle and hating it is a mistake पञ्चदशी · Pañcadaśī

weak4 sourcesClaims8 opinion1 contestedTraditions4 contemplativeUnderWhat you are identified with is invisible from inside

Awareness is what everything is known by, and it stays through waking, dream and sleep while their contents drop out

Whoever does the knowing cannot be found among the things known, and these texts test that across waking, dream and sleep, where contents drop out and something stays to report. The Panchadasi argues that nobody could remember an empty sleep without having been present for it. The Mandukya says the fourth is what waking, dream and sleep are made of, and in the Upanishads light means what you see by, not brightness. Nine opinion claims from four contemplative sources. Three stand against, including Rahula's no thinker behind the thought, Gaudapada treating his own teaching as a ladder to throw away, and the warning that the match with information theory turns on one word.

Chapters Consciousness Across Three Worldviews · उपनिषद् · The Upanishads · माण्डूक्योपनिषत् गौडपादकारिका · The Māṇḍūkya Upaniṣad and Gauḍapāda's Kārikā · पञ्चदशी · Pañcadaśī

For

Whoever is doing the knowing cannot be found among things known उपनिषद् · The Upanishads

Whatever drops out in dream or sleep is not the one reporting पञ्चदशी · Pañcadaśī

Nobody could remember an empty sleep without having been present for it पञ्चदशी · Pañcadaśī

The Katha Upanishad asks the reader to turn the eye back उपनिषद् · The Upanishads

The Kena Upanishad says what you can worship is not the reality उपनिषद् · The Upanishads

The Mandukya says awareness is what the three states are made of माण्डूक्योपनिषत् गौडपादकारिका · The Māṇḍūkya Upaniṣad and Gauḍapāda's Kārikā

The Mandukya's fourth is what waking, dream and sleep are made of उपनिषद् · The Upanishads

In the Upanishads light means what you see by, not brightness Consciousness Across Three Worldviews

Vedanta's light of awareness and physics' shared information play the same role Consciousness Across Three Worldviews

Against

Rahula says there is no thinker standing behind the thought What the Buddha Taught

The match between the texts and information theory turns on one word Consciousness Across Three Worldviews

Gaudapada treats his own teaching as a ladder to be thrown away माण्डूक्योपनिषत् गौडपादकारिका · The Māṇḍūkya Upaniṣad and Gauḍapāda's Kārikā

contested4 sourcesClaims3 opinion2 contestedTraditions4 contemplativeUnderThe inside is not readable from the outsideHow much mind you grant is your decisionThe claim on the result is what binds

Two people doing the same thing can be in completely different inner states, and nothing visible tells them apart

Nothing on the outside separates two people doing the same thing in different inner states. The Panchadasi says flatly that in doing or leaving undone, in body, senses, mind or intellect, there is not the least difference between the man who has woken up and the man who has not, and that the difference is one of conviction only. Suraghu in the Yoga Vasistha says the lotus posture gives nothing and daily work prevents nothing, and Vivekananda says refusing to fight means nothing unless you could have fought. Against this stands Wrzesniewski, who measures the inner stance precisely by what people change in their behaviour.

Chapters Karma-Yoga · उपनिषद् · The Upanishads · पञ्चदशी · Pañcadaśī · योगवासिष्ठ · The Yoga Vāsiṣṭha

For

The man who knows and the man who does not behave alike पञ्चदशी · Pañcadaśī

Suraghu says the lotus posture gives nothing and daily work prevents nothing योगवासिष्ठ · The Yoga Vāsiṣṭha

Vivekananda says refusing to fight means nothing unless you could have fought Karma-Yoga

Vivekananda says no duty can be identified by the act itself Karma-Yoga

The Katha Upanishad judges a person by what they prefer, not do उपनिषद् · The Upanishads

Against

How people see their work predicts what they change about it Crafting a Job

weak4 sourcesClaims5 opinion3 contestedTraditions3 philosophy1 contemplativeUnderThe inside is not readable from the outsideThe definition settles the argumentHow much mind you grant is your decision

You cannot tell from what a system does whether it feels anything, because being smart is not being conscious

Behaviour does not settle whether there is anything it is like to be the system. Seth argues intelligence and consciousness need not travel together, that making a machine smarter does not move it toward feeling, and that a simulated brain is conscious only if consciousness turns out to be computation, which he doubts because a brain has no software separate from its biology. Metzinger offers the one criterion that is not behavioural, saying a machine matters morally once it has a self-model. Eight claims from four sources, none of them evidence and three contested, and the information account admits at its own ending that it never reaches felt experience.

Chapters The Mythology of Conscious AI · Consciousness Across Three Worldviews · The Ego Tunnel · Being You

For

Seth argues intelligence and consciousness need not go together The Mythology of Conscious AI

Making a machine smarter does not bring it closer to feeling Being You

Seth says a simulated brain is conscious only if consciousness is computation The Mythology of Conscious AI

Seth says a brain has no software separate from its biology The Mythology of Conscious AI

Metzinger says a machine matters morally once it has a self-model The Ego Tunnel

The information account explains what a creature registers, not what it feels Consciousness Across Three Worldviews

The essay's own ending admits it never reaches felt experience Consciousness Across Three Worldviews

The match between the texts and information theory turns on one word Consciousness Across Three Worldviews

supported3 sourcesClaims4 evidence4 opinion1 contestedTraditions2 empirical1 contemplativeUnderAuthorship is assigned not feltThe failure shows downstream of its cause

Everything a person does has causes behind it, and people who drop blame for that reason do not behave worse

Every action has a history behind it, and letting go of blame on that ground does not make people worse. Sapolsky's Determined argues that randomness is not freedom, that chaos removes predictability without leaving anything free, and that willpower is brain tissue shaped by luck. The strongest number is a 2022 meta-analysis of 145 experiments, 95 of them unpublished, finding no consistent effect of weakened free-will belief on ethical behaviour, and society has twice dropped blame without falling apart. Two claims stand against: Sapolsky defines free will as a cause with no cause, which settles the argument before evidence arrives, and Seth counts free will as a real skill.

Chapters Behave · Determined · योगवासिष्ठ · The Yoga Vāsiṣṭha

For

The common middle view of free will needs a little person inside Behave

Randomness cannot give you free will because random is not free Determined

Sapolsky says chaos makes things unpredictable but leaves nothing free Determined

Sapolsky says learning is a chain of molecular steps, not a decision Determined

Sapolsky says willpower is brain tissue shaped by luck like everything else Determined

Explaining behaviour from one field alone gets it wrong, not just incomplete Behave

Vasistha says what people call fate is their own past effort योगवासिष्ठ · The Yoga Vāsiṣṭha

People who stop believing in free will do not behave worse Determined

Society has twice stopped blaming people without falling apart Determined

Against

Sapolsky defines free will so that no evidence could ever support it Determined

Seth says free will is a skill of keeping options open Being You

supported3 sourcesClaims2 evidence3 opinionTraditions2 contemplative1 empiricalUnderWhat you are identified with is invisible from insideCorrection lands on the personA threatened person defends instead of learning

A person whose worth is under threat defends themselves instead of learning, and argument does not change that

Cohen and Sherman's review shows the defensive response produced at will across unrelated domains, from rationalising a trivial choice to partisans refuting contrary evidence, and shows what is being defended is a broad sense of being a good and capable person rather than being good at the thing under attack. That is why writing about a value from an unrelated part of life calms the threat while affirming inside the threatened domain makes it worse. Chatterjee and Shankara read the same pattern in Arjuna's collapse, and Krishna's reply replaces his way of seeing rather than arguing with it. Several of the laboratory effects have come back smaller or absent in larger replications, and Chatterjee's version is not falsifiable.

Chapters Timeless Leadership · The Psychology of Change · श्रीमद्भगवद्गीता शाङ्करभाष्यम् · The Gita with Shankara's Commentary

For

People defend themselves rather than learn when their identity is threatened The Psychology of Change

Chatterjee says a leader collapses under pressure from ego, not missing facts Timeless Leadership

People defend being a good person in general, not good at something The Psychology of Change

Shankara says sorrow comes from one false belief about what is mine श्रीमद्भगवद्गीता शाङ्करभाष्यम् · The Gita with Shankara's Commentary

Krishna replaces Arjuna's way of seeing instead of arguing with him Timeless Leadership

supported3 sourcesClaims2 evidence1 opinionTraditions2 empirical1 contemplativeUnderHow it feels is evidence about the personGuard attention not the decisionThe claim on the result is what bindsConditions and unfinished work keep you going

Wanting is produced inside you regardless of what arrives, so getting more of it never brings the state you wanted

Wanting runs on its own schedule, so more of what was wanted does not deliver the state the person was after. Schultz recorded dopamine neurons firing for the gap between what arrived and what was expected rather than for the reward, so a steady level stops registering and the biggest burst moves to the anticipation. Ryan and Deci report that people who rate wealth, fame and looks as important relative to other goals score lower on well-being, whatever their reasons. The Panchadasi's three happy men, a fed infant, a sated emperor and a finished scholar, share only that nothing is wanted. No claims stand against it.

Chapters The What and Why of Goal Pursuits · Behave · पञ्चदशी · Pañcadaśī

For

Dopamine signals the gap from what was expected, not the reward itself Behave

Three men everyone calls happy share only that they want nothing पञ्चदशी · Pañcadaśī

People who chase money, fame and looks report lower well-being The What and Why of Goal Pursuits

supported3 sourcesClaims2 evidence3 opinion2 contestedTraditions2 empirical1 philosophyUnderThe inside is not readable from the outsideWhere you end is set by feelingHow much mind you grant is your decision

Whether you see a mind in a machine depends on you, not on the machine

Whether a mind appears in an object is decided at the viewer's end. Waytz, Cacioppo and Epley found this is a steady trait people differ on rather than something everyone does equally, and that resemblance, distance and loneliness move both it and the refusal to grant minds to people. Reeves and Nass found that people apply politeness and taking sides to computers on the slightest cue without believing for a moment the machine is a person. Seth adds that language does most of the work, and that calling a model's mistake a hallucination already grants it a mind. Two of the seven claims are contested.

Chapters The Mythology of Conscious AI · The Media Equation · Who Sees Human?

For

Three things make people treat an object as having a mind Who Sees Human?

People differ steadily in how much they see minds in objects Who Sees Human?

Seeing minds in objects and denying minds in people are one scale Who Sees Human?

People follow social rules with machines without thinking they are people The Media Equation

Seth says we will see minds in AI that are not there The Mythology of Conscious AI

Seth says language is what makes people call a machine conscious The Mythology of Conscious AI

Calling a model's mistake a hallucination already treats it as conscious The Mythology of Conscious AI

supported3 sourcesClaims1 evidence4 opinion1 contestedTraditions2 philosophy1 contemplativeUnderReasons are reconstructed after the factSkill arrives before understanding

A mind is made of parts that are not minds, and no thinker or owner stands behind them

Minsky's rule is that an explanation fails the moment one of its parts turns out to be clever, and that an inner watcher only needs a watcher of its own. Rahula reaches the same conclusion from the Dhammapada, where the third line changes the word so that no thing at all, conditioned or not, is a self. Seth supplies the one piece of measured support, clinical cases in which body ownership, point of view, will, life story and social self fail separately — Clive Wearing keeps the first four and loses the thread. Against it sit the Upanishadic argument that the knower cannot appear among things known, and a memory of empty sleep that someone must have been present for.

Chapters Being You · What the Buddha Taught · The Society of Mind

For

Minsky says a mind is made of parts that are not minds The Society of Mind

Minsky says an inner watcher would need a watcher of its own The Society of Mind

Rahula says there is no thinker standing behind the thought What the Buddha Taught

Rahula rests the denial of a self on one changed word What the Buddha Taught

The monk Khemaka found no self and still felt I am What the Buddha Taught

The self is five separate parts, each able to break alone Being You

Against

Whoever is doing the knowing cannot be found among things known उपनिषद् · The Upanishads

Nobody could remember an empty sleep without having been present for it पञ्चदशी · Pañcadaśī

supported3 sourcesClaims1 evidence3 opinionTraditions2 empirical1 contemplativeUnderGuard attention not the decisionChange the arrangement not the personThe claim on the result is what bindsConditions and unfinished work keep you goingPeople reshape the job you hand them

Motivation comes from a person's conditions, not from being told to try harder, and blocked needs produce substitutes

Ryan and Deci's finding is that blocked needs do not make people try harder the way hunger does: they take up substitutes. Teenagers whose mothers were rated cold and controlling by themselves, by the mothers and by interviewers placed more relative importance on wealth. Wrzesniewski and Dutton's model of why workers reshape their own jobs, for control, self-respect and connection, is assembled from adjacent literatures rather than tested, and they add that money worries, the standing of the job and workplace rules decide who has room to reshape anything. Vivekananda's version, that work from love leaves no painful reaction, has no outward sign and no count behind its ninety-nine in a hundred.

Chapters Karma-Yoga · Crafting a Job · The What and Why of Goal Pursuits

For

Blocked psychological needs produce substitutes not harder effort The What and Why of Goal Pursuits

Workers reshape their jobs to gain control, self-respect and connection Crafting a Job

Not everyone has the same room to reshape their job Crafting a Job

Vivekananda says work from love, not from compulsion Karma-Yoga

supported3 sourcesClaims1 evidence3 opinionTraditions3 contemplativeUnderWhat you are identified with is invisible from insideWatching a state beats attacking itA threatened person defends instead of learning

The story you tell about who you are does the damage, and it loosens when you treat it as words

Hayes argues that the self a person builds in words is the dangerous one, because once you are invested in a label you are invested in being right about it. The measured support is small and sharp: saying a one-word self-description aloud for twenty to forty-five seconds drops how believable it feels in about 95 per cent of people, while its meaning stays the same. Two older texts sort the attachments rather than the words — the Buddha's formula names craving, conceit and taking something to be a self, and the Drig-Drishya Viveka counts three identifications of which only one is an error. Believability is not behaviour, and the step from a rating to what someone does is not closed.

Chapters दृग्दृश्यविवेक · Dṛg‑Dṛśya‑Viveka · अनत्तलक्खणसुत्त · Anattalakkhaṇa Sutta · Get Out of Your Mind and Into Your Life

For

Hayes says the self you describe in words is the dangerous one Get Out of Your Mind and Into Your Life

Repeating a word aloud for forty seconds makes you believe it less Get Out of Your Mind and Into Your Life

The Buddha's formula names three separate attachments, not one अनत्तलक्खणसुत्त · Anattalakkhaṇa Sutta

The I makes three identifications and only one is a mistake दृग्दृश्यविवेक · Dṛg‑Dṛśya‑Viveka

supported3 sourcesClaims1 evidence3 opinionTraditions3 philosophyUnderReasons are reconstructed after the factSkill arrives before understandingAuthorship is assigned not felt

Your explanation of why you decided something is produced by a separate part built for telling other people

Dennett takes the mechanism from McFarland: communicating is the only behaviour that requires an organism to monitor its own control system, so looking inward returns an edited summary built for reporting, not a view of the machinery. Agüera y Arcas adds that the narrating voice is aimed at a listener, since you already know what you are thinking and they do not. Dunbar's measurements in the 1990s, cortex rising with group size and more steeply in apes than monkeys, are the only numbers here. A troop-size correlation fits several stories, and nothing offered separates this account from the simpler one that explaining yourself is just useful.

Chapters From Bacteria to Bach and Back · What Is Intelligence? · The Mythos of Model Interpretability

For

The voice in your head is built for your listener What Is Intelligence?

Introspection gives you a digest built so you could tell others From Bacteria to Bach and Back

Lipton says people pass none of the three transparency tests either The Mythos of Model Interpretability

Brains grew large in order to model other brains, not the world What Is Intelligence?

weak3 sourcesClaims3 opinion1 contestedTraditions2 contemplative1 philosophyUnderGuard attention not the decisionSkill lives only where it is exercised

A person is the sum of what they have repeatedly done and cared about, and nothing comes from outside

Vivekananda holds that every action leaves a trace, that the traces add up to character, and that what leaves the trace is caring rather than seeing — of hundreds of faces in a day only the loved one stays. The Yoga Vasistha states the same loop from the other end, conviction driving the action and the action strengthening the conviction, with nothing stored in between. Minsky supplies the only mechanism, memory as switching back on whatever was active before. All four are arguments or images rather than measurements, the machinery around Minsky's version produced almost no experiments, and a loop with no outside cannot say how the first conviction got in.

Chapters Karma-Yoga · The Society of Mind · योगवासिष्ठ · The Yoga Vāsiṣṭha

For

Vivekananda says every act leaves a trace and the traces become character Karma-Yoga

Vasistha says conviction makes the action and the action strengthens the conviction योगवासिष्ठ · The Yoga Vāsiṣṭha

Vivekananda says caring about something, not seeing it, leaves the mark Karma-Yoga

Remembering means switching back on the parts that were active before The Society of Mind

weak3 sourcesClaims3 opinion1 contestedTraditions3 contemplativeUnderHow it feels is evidence about the person

An appearance does no harm, the error is how it is taken, and the teaching is a way of looking

Sarvapriyananda says the whole apparatus of witness, veil and three selves is a way of teaching rather than a list of things that exist, and reading it as a list is the error the texts warn against. The Drig-Drishya Viveka makes the point at verses 13 and 15, where what gets hidden is never the rope but its being a rope, so the appearance itself does no damage until a difference is forgotten. Easwaran's Upanishads and the Pañcadaśī are read the same way, as procedures with one part of ignorance producing the grief. All of it is illustration, and the move insulates the teaching, since anything that fails can be called method rather than doctrine.

Chapters उपनिषद् · The Upanishads · दृग्दृश्यविवेक · Dṛg‑Dṛśya‑Viveka · पञ्चदशी · Pañcadaśī

For

Sarvapriyananda treats the witness and the veil as teaching devices दृग्दृश्यविवेक · Dṛg‑Dṛśya‑Viveka

The Drig-Drishya Viveka says error begins when a difference is forgotten दृग्दृश्यविवेक · Dṛg‑Dṛśya‑Viveka

The Upanishads offer a procedure to run, not a doctrine to believe उपनिषद् · The Upanishads

Ignorance has two parts and only one of them produces grief पञ्चदशी · Pañcadaśī

weak3 sourcesClaims5 opinion1 contestedTraditions3 contemplativeUnderWhat you are identified with is invisible from inside

Anything you can notice is not the one noticing, so awareness never shows up among the things it shows

Anything that can be noticed falls among the things shown, so whatever does the showing never appears there. Drig-Drishya Viveka argues that anything you can see is not the one seeing, that the body seems aware only because awareness is passed down a chain, and that anyone denying the witness is using it to deny it. Shankara puts pleasure and pain among things seen rather than with the seer, and the Ashtavakra Gita says you are not even the one who knows. Six claims from three contemplative sources, none of them evidence. James stands against, since for him the passing thought is the thinker and nothing stands behind it.

Chapters श्रीमद्भगवद्गीता शाङ्करभाष्यम् · The Gita with Shankara's Commentary · दृग्दृश्यविवेक · Dṛg‑Dṛśya‑Viveka · अष्टावक्रगीता · Aṣṭāvakra Gītā

For

Anything you can see is not the one seeing दृग्दृश्यविवेक · Dṛg‑Dṛśya‑Viveka

The Ashtavakra Gita says you are not the one who knows अष्टावक्रगीता · Aṣṭāvakra Gītā

Anyone who denies the witness is using it to deny it दृग्दृश्यविवेक · Dṛg‑Dṛśya‑Viveka

Shankara puts pleasure and pain among the things seen, not the seer श्रीमद्भगवद्गीता शाङ्करभाष्यम् · The Gita with Shankara's Commentary

The body seems aware only because awareness is passed down a chain दृग्दृश्यविवेक · Dṛg‑Dṛśya‑Viveka

Vedanta says everything has five aspects and only two of them change दृग्दृश्यविवेक · Dṛg‑Dṛśya‑Viveka

Against

The passing thought is the thinker and nothing stands behind it The Consciousness of Self

weak3 sourcesClaims4 opinionTraditions2 contemplative1 designUnderHow it feels is evidence about the person

Grief comes from what you were told and from calling something mine, not from what happened in the world

The Pañcadaśī's pair does the work: a father whose son is alive abroad weeps at a liar's word that the boy is dead, while a father whose son has really died does not weep because the news has not reached him. The two differ only in what they were told. The Yoga Vasistha adds the second half, three fighters made without conditioning who could not be killed until the gods put the feelings of I and mine into them. Nothing is measured, both are constructed stories, and Vidyāraṇya blocks the idealist reading himself by saying the world in the mind still needs the outer one as its basis.

Chapters The Psychology of Computer Programming · पञ्चदशी · Pañcadaśī · योगवासिष्ठ · The Yoga Vāsiṣṭha

For

A father weeps for a living son on a false report पञ्चदशी · Pañcadaśī

Ignorance has two parts and only one of them produces grief पञ्चदशी · Pañcadaśī

Demons with no conditioning could not be killed until they said mine योगवासिष्ठ · The Yoga Vāsiṣṭha

Weinberg says a smooth progress report shows filtering, not progress The Psychology of Computer Programming

weak3 sourcesClaims7 opinionTraditions3 contemplativeUnderHow it feels is evidence about the personWatching a state beats attacking it

The texts tell you to watch how a state comes and goes rather than remove it, and watching removes it

The instruction is to know how something arises and passes, and the removal follows from the knowing rather than being asked for. The Satipatthana Sutta tells you to know a state rather than get rid of it, treats a restless mind as the material, has you watch other people too, and adds five instructions in its refrain that the exercises never give, including putting aside wanting and resisting. Gaudapada's rope is the same shape, since not seeing the rope is the whole of the snake mistake. In the Panchadasi the tenth man stops weeping the instant he is told, though the cut he made still takes a month. Seven opinion claims, three sources, nothing measured.

Chapters माण्डूक्योपनिषत् गौडपादकारिका · The Māṇḍūkya Upaniṣad and Gauḍapāda's Kārikā · सतिपट्ठानसुत्त · Satipaṭṭhāna Sutta · पञ्चदशी · Pañcadaśī

For

The Buddha says to know a state, not to remove it सतिपट्ठानसुत्त · Satipaṭṭhāna Sutta

In this discourse a restless mind is the material not the problem सतिपट्ठानसुत्त · Satipaṭṭhāna Sutta

The Satipatthana refrain adds five instructions the exercises never give सतिपट्ठानसुत्त · Satipaṭṭhāna Sutta

The Satipatthana instruction includes putting aside wanting and resisting सतिपट्ठानसुत्त · Satipaṭṭhāna Sutta

The Satipatthana Sutta has you watch other people too सतिपट्ठानसुत्त · Satipaṭṭhāna Sutta

Not seeing the rope is the whole of the snake mistake माण्डूक्योपनिषत् गौडपादकारिका · The Māṇḍūkya Upaniṣad and Gauḍapāda's Kārikā

The grief stops at once and the wound still takes a month पञ्चदशी · Pañcadaśī

weak3 sourcesClaims4 opinion1 contestedTraditions2 contemplative1 philosophyUnderThe frame decides what you find

Waking life and science are each checked only from inside themselves, so neither reaches reality directly

Nothing checks waking except more waking, and nothing checks science except whether it works for people like us. Gaudapada's Mandukya Karika argues that waking is worth no more than dreaming and that a thing missing before and after is missing in between. The Yoga Vasistha's King Lavana shuts his eyes in court for under an hour and comes out having lived seven years, so felt duration measures nothing outside itself. Thompson pushes the same move onto science. All five claims are opinion or contested, from three sources with no evidence, and Thompson's step is disputed and hard to hold without losing the ground to state it.

Chapters माण्डूक्योपनिषत् गौडपादकारिका · The Māṇḍūkya Upaniṣad and Gauḍapāda's Kārikā · Why I Am Not a Buddhist · योगवासिष्ठ · The Yoga Vāsiṣṭha

For

The Mandukya Karika says waking is worth no more than dreaming माण्डूक्योपनिषत् गौडपादकारिका · The Māṇḍūkya Upaniṣad and Gauḍapāda's Kārikā

A king lived seven years in under an hour of court time योगवासिष्ठ · The Yoga Vāsiṣṭha

Vasistha says the doer, the deed and the seen are all mind योगवासिष्ठ · The Yoga Vāsiṣṭha

Gaudapada says a thing missing before and after is missing in between माण्डूक्योपनिषत् गौडपादकारिका · The Māṇḍūkya Upaniṣad and Gauḍapāda's Kārikā

Thompson says emptiness must apply to science as well Why I Am Not a Buddhist

weak3 sourcesClaims3 opinion2 contestedTraditions3 contemplativeUnderAuthorship is assigned not feltThe claim on the result is what binds

Your nature does the act and the feeling of having chosen it is added afterwards, even when you sit still

The act happens first and the sense that you chose it is added afterwards. Easwaran's Gita says nature does the deed and the ego claims it, Shankara points out that the feeling of being the doer arrives over sitting still just as readily as over work, and the Ashtavakra Gita's free man keeps acting while feeling no authorship. Five claims from three contemplative texts, three opinion and two contested, none of them measured. The weak point sits inside the group: if the sense of authorship goes and behaviour does not change, nothing outside the person separates this from ordinary idleness.

Chapters भगवद्गीता · The Bhagavad Gita · श्रीमद्भगवद्गीता शाङ्करभाष्यम् · The Gita with Shankara's Commentary · अष्टावक्रगीता · Aṣṭāvakra Gītā

For

The Gita says nature does the deed and the ego claims it भगवद्गीता · The Bhagavad Gita

We feel like a doer even when we are doing nothing श्रीमद्भगवद्गीता शाङ्करभाष्यम् · The Gita with Shankara's Commentary

The Ashtavakra Gita says the sage still acts but feels no doer अष्टावक्रगीता · Aṣṭāvakra Gītā

Krishna tells Arjuna his own nature will make him fight anyway भगवद्गीता · The Bhagavad Gita

Ashtavakra's free man cannot be told apart from a merely lazy one अष्टावक्रगीता · Aṣṭāvakra Gītā

supported2 sourcesClaims4 evidenceTraditions2 philosophyUnderThe inside is not readable from the outsideSkill arrives before understanding

Skill comes before understanding, and understanding is a late product of a process that has none

Dennett's case is that design fills the living world without anyone understanding anything, so skill comes first and understanding arrives late. Ristau's plover, tested in the 1980s with a remote-controlled buggy carrying a stuffed raccoon, tracks where the predator is looking and performs harder when it loses interest, without holding the plan. Agüera y Arcas ran 64-byte random strings together in a soup and after millions of interactions they began copying themselves, with nothing selected for and mutation switched off. All four supporting claims carry measurements rather than argument. Against them stands the argument that predicting the next word well requires knowing how the world works.

Chapters From Bacteria to Bach and Back · What Is Intelligence?

For

Dennett says competence comes first and understanding is a late product From Bacteria to Bach and Back

A plover adjusts its broken-wing display without knowing why it works From Bacteria to Bach and Back

Self-copying programs appeared in a random soup with no selection or mutation What Is Intelligence?

The replicators that appeared were built from smaller ones already working What Is Intelligence?

Against

Predicting the next word well requires knowing how the world works What Is Intelligence?

supported2 sourcesClaims3 evidenceTraditions2 philosophyUnderHow it feels is evidence about the personSkill arrives before understandingThe frame decides what you find

A living thing works out what it senses and then treats its own answer as a fact about the world

An organism does not read properties off the world. It computes an answer and then treats that answer as a property of what it sensed. Aguera y Arcas shows a bacterium working out concentration as an average over separate molecules arriving at its receptors, with the length of the averaging window the thing it has to get right, and pain tracking the brain's guess about damage rather than the damage. Dennett adds that there is no sweetness in glucose, only properties that provoke a response, and the response is read back into the sugar. Three evidence claims from two sources with none against, but the range runs only from a bacterium to a taste.

Chapters From Bacteria to Bach and Back · What Is Intelligence?

For

A bacterium works out concentration from arrivals instead of reading it off What Is Intelligence?

Pain tracks the brain's guess about damage, not the damage itself What Is Intelligence?

Sweetness is your reaction to sugar read back onto the sugar From Bacteria to Bach and Back

supported2 sourcesClaims3 evidence1 opinionTraditions1 contemplative1 empiricalUnderThe claim on the result is what bindsConditions and unfinished work keep you goingPeople reshape the job you hand them

The same job becomes a different job depending on how a worker sees it, and that predicts what they change

How a worker holds a job changes what the job is, and how they see it predicts what they go on to change. Wrzesniewski found twenty-eight cleaners in one hospital, all under a single job description, describing two different jobs: one minimal and unskilled, one that counted itself part of healing. Whether people see work as a job, a career or a calling predicts whether they reshape the paid tasks, the visible tasks and powerful people, or the purpose. Vivekananda makes the same point from the other side. Against it, money worries, the standing of the job and workplace rules decide how much room a person actually has.

Chapters Karma-Yoga · Crafting a Job

For

Hospital cleaners with one job description described two entirely different jobs Crafting a Job

How people see their work predicts what they change about it Crafting a Job

Vivekananda says work changes the worker, not the world Karma-Yoga

The main job crafting questionnaire leaves out how people reframe work Crafting a Job

Against

Not everyone has the same room to reshape their job Crafting a Job

supported2 sourcesClaims2 evidence2 opinionTraditions2 contemplativeUnderThe claim on the result is what bindsConditions and unfinished work keep you going

People keep going for something outside themselves that waits to be done, and it is never finished

What holds a person up is something outside them that is waiting, and it is a direction rather than an arrival. Frankl argues people endure more when a person needs them or work is left that only they can finish, and he turned back two prisoners who meant to kill themselves this way. He also saw the control case: with no known end date a person stops aiming at the future, and a day in the camp felt longer than a week. Hayes says the same from the clinic, that a value is a direction you never finish reaching. Frankl's evidence comes only from survivors.

Chapters Experiences in a Concentration Camp · Get Out of Your Mind and Into Your Life

For

Frankl reversed the question to ask what life expects of you Experiences in a Concentration Camp

People survive longer when someone or something outside them needs them Experiences in a Concentration Camp

Frankl saw that not knowing when it ends stops people planning Experiences in a Concentration Camp

Hayes says a value is a direction you never finish reaching Get Out of Your Mind and Into Your Life

supported2 sourcesClaims2 evidence2 opinion1 contestedTraditions1 contemplative1 philosophyUnderThe inside is not readable from the outsideThe frame decides what you findThe definition settles the argumentA result outruns the design that produced it

Science cannot confirm Buddhism or find its states in the brain, and its scientific reputation was built in the 1800s

A scan cannot certify a tradition or locate the state it names. Thompson traces the belief that Buddhism is uniquely rational to nineteenth-century reformers who took a colonial argument, that having science proved Christian superiority, and turned it around. He also argues that finding mindfulness in a brain region explains nothing, since the concept decides what the scanner counts. The language supports him: sati means holding in mind, the same word as the Sanskrit for memory, and bare attention is Nyanaponika's 1954 gloss of one clause. Only two sources, one of them a contested reading, and the case is historical rather than experimental.

Chapters सतिपट्ठानसुत्त · Satipaṭṭhāna Sutta · Why I Am Not a Buddhist

For

The idea that Buddhism is uniquely scientific was invented in the 1800s Why I Am Not a Buddhist

Thompson says cross-tradition conversation only matters when it unsettles both sides Why I Am Not a Buddhist

The Pali word sati means holding in mind, not bare attention सतिपट्ठानसुत्त · Satipaṭṭhāna Sutta

Thompson says finding mindfulness in a brain region explains nothing Why I Am Not a Buddhist

No brain scan can show enlightenment because the idea decides what counts Why I Am Not a Buddhist

supported2 sourcesClaims1 evidence4 opinion2 contestedTraditions2 philosophyUnderWhat you are identified with is invisible from insideWatching a state beats attacking itSome of the job cannot be taught

Adults grow by turning what they are into something they can look at, and most stop partway

Kegan's mechanism is a shift from being something to being able to look at it: whatever you are cannot be seen by you, and whatever you can hold and examine no longer runs you. The numbers come from a composite of twelve studies, 282 adults, of whom 59 per cent had not reached self-authorship, and over 70 per cent of his longitudinal subjects were short of it on entering. None of the 282 reached the fifth order at all. The composite is not a probability sample and is whiter, wealthier and better educated than the country, the scoring is interpretive, and a stage with no positive cases says as much about the instrument as about people.

Chapters In Over Our Heads · Immunity to Change

For

Kegan says growth turns what you are into something you can examine In Over Our Heads

Half to two-thirds of adults still take their values from others In Over Our Heads

Kegan says students often cannot separate their views from others' expectations In Over Our Heads

Parenting, partnering, work and school all demand the same kind of adult In Over Our Heads

Nobody in Kegan's studies reached the fifth order of mind In Over Our Heads

People treat their deepest assumptions as plain facts about the world Immunity to Change

Kegan says a demand made without support is a punishment In Over Our Heads

supported2 sourcesClaims1 evidence1 opinionTraditions1 empirical1 contemplativeUnderNothing helps in itself it has to fit the receiverHow much mind you grant is your decision

Knowing which group someone belongs to tells you little about how they will act

Group membership predicts conduct poorly, and reading an individual off their category misses the person in front of you. Frankl says knowing whether a man was a guard or a prisoner told you almost nothing, since one camp commander bought medicine with his own money and a senior prisoner was harder than any guard. Epley supplies the mechanism: a mind you cannot read is treated as smaller rather than as unknown, so distant groups are judged less able to feel shame or remorse and their apologies count for less. Only two claims and two sources, and Frankl's half is an observation from a single extreme setting.

Chapters Experiences in a Concentration Camp · Mindwise

For

Frankl found decent and cruel men among guards and prisoners alike Experiences in a Concentration Camp

People credit strangers they cannot read with less of a mind Mindwise

supported2 sourcesClaims1 evidence2 opinion1 contestedTraditions1 empirical1 philosophyUnderWhat you are identified with is invisible from insideCorrection lands on the personA threatened person defends instead of learning

Reminding a person of a value outside the threatened area lets them take in bad news about it

Cohen and Sherman's field trials had students spend ten minutes writing about something they personally valued: in three middle schools the share of African American students earning a D or F in the first term halved, and the gain persisted. The values that work reach beyond the self, compassion or service, while power, status and approval do nothing, and affirming inside the threatened area makes people more defensive. James arrives at the same repair from the other end, lowering what you have staked yourself on rather than raising what you achieve. The effect is concentrated in the most threatened students, is absent without a threat and some ability waiting to be used, and a British medical-school study found mixed results.

Chapters The Psychology of Change · The Consciousness of Self

For

Writing about compassion lowers defensiveness; writing about status does not The Psychology of Change

Affirming your values helps only where a threat and a resource exist The Psychology of Change

Ten minutes writing about a personal value lifts threatened students' later grades The Psychology of Change

James makes self-esteem a fraction of success over what you claimed The Consciousness of Self

supported2 sourcesClaims1 evidence1 opinion1 contestedTraditions1 empirical1 philosophyUnderThe inside is not readable from the outsideAuthorship is assigned not feltThe definition settles the argument

The argument about free will is decided by how each side defines it, not by any experiment

Each side reaches its answer from the meaning it picked rather than from a result. Sapolsky requires free will to be a cause with nothing causing it, which settles the question before evidence arrives and is why the philosophers he argues against were unmoved. He grants that Libet leaves free will standing and says it does not matter, since forty years of work measured the last few hundred milliseconds before a movement when the question is where the intention came from. Seth picks a different meaning, how well you keep your options open, and counts it real because brain damage takes it away. Three claims, two sources, one contested.

Chapters Being You · Determined

For

Sapolsky defines free will so that no evidence could ever support it Determined

Sapolsky says Libet leaves free will standing and that settles nothing Determined

Seth says free will is a skill of keeping options open Being You

supported2 sourcesClaims1 evidence4 opinion1 contestedTraditions2 philosophyUnderWhat you are identified with is invisible from insideWhere you end is set by feeling

The brain builds a picture of itself and cannot see that it is a picture, so the self feels real

Metzinger argues the brain builds a picture of itself and hides the building, because hiding it is cheaper than showing it, and whatever falls inside that picture arrives feeling like yours. The rubber-hand illusion moves the boundary in sixty to ninety seconds, and Lenggenhager, Metzinger and Blanke moved where people felt themselves to be a couple of metres across a room by filming a back being stroked and showing it ahead. Thompson stands against the conclusion twice: every illusion is a construction but not every construction is an illusion, and the no-self result holds only if a self must be single and unbuilt.

Chapters The Ego Tunnel · Being You

For

The self is a model the brain cannot see as a model The Ego Tunnel

The brain hides how it builds the self because that is cheaper The Ego Tunnel

Metzinger says a self needs only a body picture mistaken for real The Ego Tunnel

A rubber hand can feel like your own within ninety seconds The Ego Tunnel

Where people feel themselves to be can be moved across a room The Ego Tunnel

We sense ourselves in order to control ourselves, not to be accurate Being You

Against

Showing the self is built does not show it is an illusion Why I Am Not a Buddhist

Thompson says the no-self argument depends on how you define self Why I Am Not a Buddhist

supported2 sourcesClaims1 evidence1 contestedTraditions2 philosophyUnderReasons are reconstructed after the factAuthorship is assigned not felt

The feeling of having done something is made separately from the doing, so it does not prove you did it

The sense of having caused something is produced apart from the act, so it is a report rather than a witness. Metzinger's case is that owning a movement and authoring it come apart in both directions, which means the feeling cannot be used as proof that you acted. Seth adds that the readiness potential, the brain signal that shows up before a voluntary movement, may on Schurger's reading be ongoing random activity drifting across a threshold rather than an early decision. Only two claims from two sources, one evidence and one contested, and none of this shows the person did not act.

Chapters The Ego Tunnel · Being You

For

Feeling that you did something is not proof that you did The Ego Tunnel

The brain signal before a choice may be noise, not a decision Being You

weak2 sourcesClaims2 opinionTraditions1 design1 contemplativeUnderThe frame decides what you findThe failure shows downstream of its causeNo method sets the problem for you

A description is never the thing it describes, and trouble starts when a person or machine acts on it instead

Christian's closing argument is that the danger is not machines taking over but the formal descriptions we hand them of what exists and what we want, since a system that runs on such a description reshapes the world to fit it — the car that killed Elaine Herzberg had no category for her. Gaudapada makes the same move against a description in words, arguing that every account of one thing causing another contradicts itself. Both are arguments rather than findings. Christian's is broad enough that little could count against it, and Gaudapada's trilemma defeats only a substance account of causation, not the working notion science uses.

Chapters The Alignment Problem · माण्डूक्योपनिषत् गौडपादकारिका · The Māṇḍūkya Upaniṣad and Gauḍapāda's Kārikā

For

We lose control to our formal descriptions, not to the machines The Alignment Problem

Gaudapada argues that no account of cause and effect holds together माण्डूक्योपनिषत् गौडपादकारिका · The Māṇḍūkya Upaniṣad and Gauḍapāda's Kārikā

weak2 sourcesClaims8 opinionTraditions1 contemplative1 philosophyUnderThe inside is not readable from the outsideThe definition settles the argument

A self that is built is still real, and what counts as one depends on your definition and level

Thompson grants that the brain assembles the self and denies that this makes it an illusion, since every illusion is a construction but not every construction is an illusion. The no-self conclusion, he argues, follows only from defining a self as single, separate and not built from experience; define it as built over time and socially, as James did in 1890 and Mead in 1934, and the same evidence proves much less. Sarvapriyananda's lectures add a level-dependent version, with an immune cell, the bee and the hive each a self. Nothing here is measured, and Metzinger's reply stands against it, that the self is a model the brain cannot see as a model.

Chapters Consciousness Across Three Worldviews · Why I Am Not a Buddhist

For

Showing the self is built does not show it is an illusion Why I Am Not a Buddhist

Thompson says the no-self argument depends on how you define self Why I Am Not a Buddhist

Cause and effect alone cannot say which experience belongs to me Why I Am Not a Buddhist

A self begins wherever something maintains a boundary against the world Consciousness Across Three Worldviews

Living things become selves by drawing a boundary at the cell membrane Consciousness Across Three Worldviews

An immune cell, a bee and a hive are each a self Consciousness Across Three Worldviews

Whether something is one self or many depends on the level chosen Consciousness Across Three Worldviews

Sarvapriyananda argues information matters only when it belongs to someone Consciousness Across Three Worldviews

Against

The self is a model the brain cannot see as a model The Ego Tunnel

Metzinger says a self needs only a body picture mistaken for real The Ego Tunnel

weak2 sourcesClaims3 opinionTraditions2 philosophyUnderThe inside is not readable from the outsideGuard attention not the decisionHow much mind you grant is your decision

AI harms the people using it first, by taking their attention and lowering their view of what humans do

The danger argued for here is not a machine waking up but what these machines do to the people using them. Seth holds that mistaking AI for minds lowers what we think humans are, and that machines which merely seem conscious are the closer problem. Metzinger adds that directing your own attention is one of the deepest parts of being a person, so an industry built to capture attention takes away part of selfhood rather than just time. All three claims are opinion from two philosophy sources with nothing measured, and neither author separates this harm from the older complaint about advertising.

Chapters The Mythology of Conscious AI · The Ego Tunnel

For

Seth says mistaking AI for minds makes us think less of ourselves The Mythology of Conscious AI

Machines that merely seem conscious are the closer danger The Mythology of Conscious AI

Metzinger says an attention-capturing industry takes away part of being a person The Ego Tunnel

weak2 sourcesClaims1 opinion1 contestedTraditions2 contemplativeUnderThe claim on the result is what binds

After all the teaching the last choice is the person's own, and they can always choose their attitude

Two books end on the same note: after seven hundred verses Krishna tells Arjuna to reflect and then do as he chooses, and Frankl writes that everything can be taken from a person except the choice of what attitude to take. Both then undercut it. Four verses earlier the Gita has Arjuna's own nature compelling him to fight regardless, and two verses later comes an imperative; Frankl concedes that on average only those survived who had lost all scruples, which makes his evidence a sample of the lucky. Nothing is measured on either side, and one example of a man giving away his bread proves a possibility, not a rule.

Chapters भगवद्गीता · The Bhagavad Gita · Experiences in a Concentration Camp

For

Krishna ends the Gita by telling Arjuna to decide for himself भगवद्गीता · The Bhagavad Gita

Frankl says a person can always choose their attitude to what happens Experiences in a Concentration Camp

Against

Krishna tells Arjuna his own nature will make him fight anyway भगवद्गीता · The Bhagavad Gita

Frankl's evidence comes only from survivors, which undercuts his own case Experiences in a Concentration Camp

weak2 sourcesClaims1 opinion1 contestedTraditions2 contemplativeUnderThe inside is not readable from the outsideThe frame decides what you find

An editor's choice of how to read a scripture decides what the scripture is taken to claim

Easwaran presents Krishna as the Self within and the battlefield as the mind, and says openly that this is his choice rather than the only reading; chapters 16 and 18, with their duty and social order, resist it. Shankara, a millennium earlier, applies the same kind of rule, reading each line for the job it is doing before what it plainly says, so a verse written to praise a lesser practice cannot be used to abolish a greater one. Both principles are unfalsifiable, since any awkward verse can be reclassified. Traditions that take the war as a war have read the same verses for as long, and Easwaran's only reply is to practise it and see.

Chapters भगवद्गीता · The Bhagavad Gita · श्रीमद्भगवद्गीता शाङ्करभाष्यम् · The Gita with Shankara's Commentary

For

Easwaran's reading of the Gita as inner allegory is one choice भगवद्गीता · The Bhagavad Gita

Shankara reads each verse for its purpose before its plain words श्रीमद्भगवद्गीता शाङ्करभाष्यम् · The Gita with Shankara's Commentary

weak2 sourcesClaims5 opinion2 contestedTraditions1 contemplative1 philosophyUnderWhat you are identified with is invisible from inside

Look through your experience for the one who owns it and you find only the experience itself

Search everything you can find in experience for whoever owns it and only the experience turns up. James reports that the inner self he could locate was movements in the head and throat, and he drops the Soul because it explains nothing and guarantees nothing, since the passing thought takes the one before it as its own and hands the title on. The Anattalakkhana Sutta runs the same search with a control test, that anything you cannot command is not your self, across eleven categories chosen so nothing can hide. Two sources, five opinion and two contested claims, nothing measured, and the Buddha never says that no self exists.

Chapters अनत्तलक्खणसुत्त · Anattalakkhaṇa Sutta · The Consciousness of Self

For

The passing thought is the thinker and nothing stands behind it The Consciousness of Self

James finds the inner self is movements in the head and throat The Consciousness of Self

James drops the Soul because it explains nothing and guarantees nothing The Consciousness of Self

The Buddha calls five things not-self and never says no self exists अनत्तलक्खणसुत्त · Anattalakkhaṇa Sutta

The Buddha says anything you cannot control is not your self अनत्तलक्खणसुत्त · Anattalakkhaṇa Sutta

The Buddha argues to not-self in steps rather than stating it अनत्तलक्खणसुत्त · Anattalakkhaṇa Sutta

The Buddha's eleven categories cover every experience so no self can hide अनत्तलक्खणसुत्त · Anattalakkhaṇa Sutta

weak2 sourcesClaims5 opinionTraditions2 contemplativeUnderHow it feels is evidence about the person

Practice does not produce the goal, and a session that feels calm and successful can be a failed one

The Mandukya Karika names four ways meditation fails, and the one that matters is a mind still stained that looks from outside exactly like a mind at rest, which leaves a practitioner with no test. Rahula presses the same point on the deepest absorptions, which the Buddha learned from his teachers, was not satisfied by, and classed as a pleasant abiding: anything produced by a cause is conditioned, so nirvana is not an effect of the path. Gaudapada extends it to his own words, which he calls a device to be thrown away. Nothing here is measured, and the canon also shows the Buddha using the absorptions he is said to have demoted.

Chapters माण्डूक्योपनिषत् गौडपादकारिका · The Māṇḍūkya Upaniṣad and Gauḍapāda's Kārikā · What the Buddha Taught

For

The Mandukya Karika warns a calm session can be a failed one माण्डूक्योपनिषत् गौडपादकारिका · The Māṇḍūkya Upaniṣad and Gauḍapāda's Kārikā

Rahula says the deepest absorptions are conditioned and not the goal What the Buddha Taught

Rahula says nirvana is not an effect of the path What the Buddha Taught

Gaudapada treats his own teaching as a ladder to be thrown away माण्डूक्योपनिषत् गौडपादकारिका · The Māṇḍūkya Upaniṣad and Gauḍapāda's Kārikā

The Mandukya Karika says nothing is ever born so nobody is freed माण्डूक्योपनिषत् गौडपादकारिका · The Māṇḍūkya Upaniṣad and Gauḍapāda's Kārikā

weak2 sourcesClaims4 opinionTraditions2 contemplativeUnderWhat you are identified with is invisible from insideWatching a state beats attacking it

Whatever notices a feeling is not itself in it, and you can see this while the feeling lasts

Whatever notices a state is not itself in that state, and the shift can be made while the state is still running. Drig-Drishya Viveka's line is that whatever notices fear is not afraid, its meditations keep pointing back to whoever is having the experience rather than to the experience, and it says enlightenment does not stop pain but changes who is watching it. Hayes reaches the same position from therapy, calling the observing self a viewpoint rather than one of your thoughts. Four opinion claims from two sources, nothing measured, and the evidence for the shift is a report of the kind these texts elsewhere distrust.

Chapters दृग्दृश्यविवेक · Dṛg‑Dṛśya‑Viveka · Get Out of Your Mind and Into Your Life

For

Whatever notices fear is not itself afraid दृग्दृश्यविवेक · Dṛg‑Dṛśya‑Viveka

Enlightenment does not stop pain, it changes who is watching it दृग्दृश्यविवेक · Dṛg‑Dṛśya‑Viveka

Drig-Drishya Viveka's meditations point back to whatever is having the experience दृग्दृश्यविवेक · Dṛg‑Dṛśya‑Viveka

The observing self is a viewpoint, not one of your thoughts Get Out of Your Mind and Into Your Life

weak2 sourcesClaims2 opinionTraditions2 contemplativeUnderThe inside is not readable from the outside

You spot someone who has let go by how evenly they treat opposites, not by what they do

The test offered is not what a person does but whether their reaction still swings between opposites. The Gita checks the freed person by evenness across gold and clay and across people liked and disliked, and Chatterjee says the same stance can be trained by regular reflection, the sign of success being that people you like, dislike and feel nothing for are treated alike. Against it stands the Ashtavakra Gita, whose liberated man is described so fully that nothing in the description separates him from someone who is simply idle. Two opinion claims from two sources, one contested claim against, and no feedback that could fail the test.

Chapters भगवद्गीता · The Bhagavad Gita · Timeless Leadership

For

The Gita tests the freed person by whether their response changes भगवद्गीता · The Bhagavad Gita

Chatterjee says letting go of being the doer can be trained Timeless Leadership

Against

Ashtavakra's free man cannot be told apart from a merely lazy one अष्टावक्रगीता · Aṣṭāvakra Gītā

weak1 sourceClaims3 evidence4 opinionTraditions1 philosophyUnderWhat you are identified with is invisible from insideA threatened person defends instead of learning

A habit you keep failing to break is protecting you, and small tests of the fear behind it work

Kegan and Lahey argue that repeated failure to change is not weak willpower but a second commitment the failure serves, so the way out is to find the fear behind it and run small, safe tests of it. Their coaching cases show people with the same goal and behaviour having entirely different hidden commitments, and a humanities department that had tenured nobody in eleven years moved only once someone said aloud what it was protecting. The evidence is selected cases from their own practice with no controls, the opening one-in-seven heart-patients figure carries no citation, and the rule that a first answer is a cover cannot be tested from inside the method.

Chapters Immunity to Change

For

People who fail to change are protecting something, not lacking willpower Immunity to Change

You find the hidden commitment by naming the fear underneath Immunity to Change

You cannot guess what hidden commitment is blocking someone's stated goal Immunity to Change

People change by running small tests, not by arguing or resolving Immunity to Change

Kegan and Lahey's tests mark where a big assumption stops holding Immunity to Change

Picking the right goal decides whether an attempt to change works Immunity to Change

A group's hidden commitments do nothing until someone names them aloud Immunity to Change

weak1 sourceClaims2 evidence2 opinion1 contestedTraditions1 contemplativeUnderWatching a state beats attacking itThe claim on the result is what binds

Fighting a painful feeling costs more than the feeling, and willingness to feel it predicts how well you function

Hayes separates the pain a person feels from the effort spent warding it off, and puts the damage in the second. In chronic pain the measured physical damage barely predicts reported disability, while willingness and the ability to act while hurting do. Wegner's demonstration supports the mechanism: told not to think of a yellow Jeep, people think of it more. The outcome evidence is not in dispute, over a thousand trials in twenty years and NICE guidance since 2021, with Öst's reviews putting the mean effect at 0.68 and later 0.42 and no significant advantage over other therapies. What is contested is the explanation, which rests on a questionnaire that may measure how bad people feel.

Chapters Get Out of Your Mind and Into Your Life

For

Struggling against pain causes more harm than the pain itself Get Out of Your Mind and Into Your Life

Willingness to feel pain predicts functioning better than how much pain Get Out of Your Mind and Into Your Life

You can choose when to face a feeling but not how strongly Get Out of Your Mind and Into Your Life

Trying not to think of something keeps the thing in mind Get Out of Your Mind and Into Your Life

ACT works but its account of why rests on a doubtful questionnaire Get Out of Your Mind and Into Your Life

weak1 sourceClaims2 evidence1 opinionTraditions1 empiricalUnderA threatened person defends instead of learningA result outruns the design that produced it

Writing about your values helps only when it comes early, nobody explains its purpose, and its gains build on themselves

Cohen and Sherman's review finds three conditions on the ten-minute values-writing exercise. It works before a threat or soon after, but not once the person has already talked themselves round; it loses its force when people are told what it is for, which is why British medical students whose essays instructors read and graded showed no benefit; and a brief effect lasts only where it starts a loop of better performance and better treatment. The loop is an explanation offered after the fact, the timing comparison in schools was between weeks of a term rather than randomised, and a second year of booster sessions added nothing.

Chapters The Psychology of Change

For

Affirmation works before a threat, not after you have explained it away The Psychology of Change

Telling people what the writing exercise is for removes its effect The Psychology of Change

A short exercise lasts only when it sets off a feedback loop The Psychology of Change

weak1 sourceClaims1 evidence1 opinionTraditions1 contemplativeUnderHow it feels is evidence about the person

How much a person suffers depends on how they see it, not on the size of what happened

Frankl says suffering behaves like a gas, filling whatever space it is given, so a small trouble can take up as much of a person as a great one. His second observation runs the same way: numbness protected prisoners rather than damaging them, and what still reached a man through it was not physical pain but being treated unjustly, the blow in the bread line hurting less than its unreasonableness. Both come from one observer recording his own reactions in the camps, with nothing measured. The comparison also undoes itself, since if size is entirely relative then no suffering can be measured against another, including the comparisons the book invites.

Chapters Experiences in a Concentration Camp

For

Frankl says small troubles can hurt as much as great ones Experiences in a Concentration Camp

Frankl saw numbness as protection and insult as what still hurt Experiences in a Concentration Camp

weak1 sourceClaims1 evidence4 opinionTraditions1 philosophyUnderWhat you are identified with is invisible from inside

The self is a pattern the brain makes by watching itself, built from parts that contain no self

Hofstadter's account is that a self appears when a system watches itself closely enough to build symbols about itself, and that it cannot see the machinery underneath. He takes Gödel as the formal result: any system able to describe patterns ends up containing statements about itself, even where the designers tried to rule that out. He also argues that a machine with no memory and no imagination has no self, and that a pattern rather than its parts can be the real cause. One source and five claims, four of them opinion, and the Gödel step is an analogy rather than a proof about brains.

Chapters I Am a Strange Loop

For

Hofstadter says the self is a pattern the brain makes watching itself I Am a Strange Loop

Hofstadter's self needs symbols about itself and no sight of the machinery I Am a Strange Loop

Hofstadter says a machine with no memory or imagination has no self I Am a Strange Loop

Gödel showed any system rich enough to describe patterns talks about itself I Am a Strange Loop

Hofstadter says the real cause can be a pattern, not the parts I Am a Strange Loop

weak1 sourceClaims1 evidence3 opinionTraditions1 philosophyUnderWhere you end is set by feelingThe claim on the result is what binds

What feels warm and yours counts as part of you, so belongings come in and old selves drop out

James puts the edge of a self wherever feeling reaches. A man's clothes and property are counted in, he has one social self for every group whose opinion matters to him, and a past self is his only where warmth and continuity still reach. The evidence is Reverend Ansel Bourne, who drew money in Providence on 17 January 1887 and vanished, then on 14 March woke in Norristown as a shopkeeper of six weeks trading as A. J. Brown and would not believe two months had passed. One source, four claims, three of them opinion, and a single dissociation case carries the weight.

Chapters The Consciousness of Self

For

James counts a man's clothes and property as parts of his self The Consciousness of Self

A man has one social self for each group whose opinion counts The Consciousness of Self

A past self is yours only where warmth and continuity reach The Consciousness of Self

A Rhode Island preacher lived two months as a shopkeeper named Brown The Consciousness of Self

weak1 sourceClaims3 opinionTraditions1 philosophyUnderWhere you end is set by feelingWhich part of the task went decides what is lost

A person and the tool they rely on solve the task together, and where the work happens decides nothing

Clark and Chalmers argue that if a process would count as thinking when it happens in the head, it counts when part of it happens outside — Otto, who has amnesia, believes what his notebook says before he opens it, the way you believe what you are about to recall. The case is built on the two-way traffic between person and resource, in Tetris and in the notebook, and nothing in the paper is measured. Two claims stand against it: Adams and Aizawa reply that the notebook is coupled to memory rather than part of it, and a store you did not fill and cannot vouch for fails the authors' own conditions.

Chapters The Extended Mind

For

A person and the tool they rely on work as one system The Extended Mind

Where a process happens cannot decide whether it counts as thinking The Extended Mind

Otto believes what his notebook says before he opens it The Extended Mind

Against

Critics say a notebook is coupled to memory, not part of it The Extended Mind

An outside store you did not fill fails Clark and Chalmers' test The Extended Mind

weak1 sourceClaims1 opinionTraditions1 contemplativeUnderWhat you are identified with is invisible from insideA threatened person defends instead of learning

A person rejects a true account of themselves when the belief protects them, so argument alone does not move it

Rahula traces the idea of a self to two needs: people made God out of a need for protection and an immortal soul out of a need to go on existing, so a teaching that removes both meets resistance that has nothing to do with the evidence. One source carries the whole claim and nothing is measured. The argument cuts against itself, since explaining why a belief is held says nothing about whether it is true, and the same story would explain the belief if there were a self. Kegan and Lahey show that a protective commitment can be mapped and revised by the person holding it, where an account pitched at the whole species leaves nobody anything to do.

Chapters What the Buddha Taught

For

Rahula traces belief in God and soul to fear and helplessness What the Buddha Taught

weak1 sourceClaims2 opinion1 contestedTraditions1 philosophyUnderWhere you end is set by feelingWhich part of the task went decides what is lost

An outside store counts as part of your mind only under conditions most stores fail to meet

Clark and Chalmers list four conditions for counting an outside store as memory: it is there when needed, easy to get at, taken as true when retrieved, and put there by you in the first place. Otto's notebook holds what Otto wrote; the Internet, and anything else you did not fill and have never endorsed, fails the test. The list is the authors' own, offered as a way of sorting cases rather than as a criterion, with no weighting and no scale, and they weaken the fourth condition themselves. Whether the endorsement conditions are needed at all is still disputed, and one paper carries the whole case.

Chapters The Extended Mind

For

Clark and Chalmers give four tests for counting a notebook as memory The Extended Mind

An outside store you did not fill fails Clark and Chalmers' test The Extended Mind

Critics say a notebook is coupled to memory, not part of it The Extended Mind

weak1 sourceClaims1 contestedTraditions1 philosophyUnderThe inside is not readable from the outsideThe definition settles the argumentHow much mind you grant is your decision

How much of a mind something has is a matter of degree, and how much it is owed follows

Hofstadter argues that having a soul is a question of how much rather than whether, with beings sitting at different points on one scale, and he accepts the consequence that they differ in how much consideration they are owed. The unit of measurement is his own invention and he says so; there is no procedure for placing anything on the scale, so it cannot be shown wrong about any case. One source carries the whole claim, marked contested, with nothing recorded against it. Over's 2021 argument cuts against the use it is usually put to, that people who treat others cruelly understand perfectly well how human their victims are.

Chapters I Am a Strange Loop

For

Hofstadter says having a soul is a matter of degree I Am a Strange Loop

weak1 sourceClaims1 opinionTraditions1 contemplativeUnderGuard attention not the decisionThe failure shows downstream of its cause

Judgment fails at the end of a chain that begins with what you dwell on, not at the decision

Gita 2.62 and 2.63 set out a sequence with seven steps: dwelling on something leads to attachment, then desire, then anger, then delusion, then confused memory, and only at the end the loss of judgment. The practical point is that the place to intervene is the first step, not the last. One verse pair from one source carries it, and nothing is measured. These are states a person can notice in themselves, offered as a causal chain, with no evidence that the links run in that order or that breaking an early one prevents the later ones.

Chapters भगवद्गीता · The Bhagavad Gita

For

The Gita traces ruin back to what the mind dwells on भगवद्गीता · The Bhagavad Gita

weak1 sourceClaims1 contestedTraditions1 contemplativeUnderWhat you are identified with is invisible from inside

The person inside a situation sees it least clearly, including when they write about it later

Being inside a situation is the worst position from which to describe it, and writing the account afterwards does not fix that. The one supporting claim is Frankl's own admission that on average only those survived who had lost all scruples, which makes him one of the lucky ones reporting from inside his own sample. It governs how every other Frankl claim should be read. The support is a single contested claim from a single source, it is an argument about sampling rather than a finding, and nothing here shows an outside observer would have done better.

Chapters Experiences in a Concentration Camp

For

Frankl's evidence comes only from survivors, which undercuts his own case Experiences in a Concentration Camp

weak1 sourceClaims4 opinion1 contestedTraditions1 contemplativeUnderThe claim on the result is what binds

You have a claim on your work and none on its results, and what you drop is the claim

Gita 2.47 gives a person a right to the action and never to what it produces, and 2.48 defines the discipline as staying even in success and failure. Easwaran reads what is given up as selfish wanting rather than work or possessions, so charity can fail the test and cooking a meal can pass; the verse's fourth clause closes the exit by forbidding attachment to doing nothing. All five claims are the text's instruction, with nothing offered as evidence and no test an outsider could run, since the test sits in a state of mind. Chapter 12 also puts giving up results last on its ladder and then calls it the highest.

Chapters भगवद्गीता · The Bhagavad Gita

For

The Gita gives you a right to your work, not its results भगवद्गीता · The Bhagavad Gita

Easwaran says the Gita asks you to drop selfish wanting, not possessions भगवद्गीता · The Bhagavad Gita

Gita 2.47 also forbids being attached to doing nothing भगवद्गीता · The Bhagavad Gita

The Gita ranks giving up results as both last resort and best भगवद्गीता · The Bhagavad Gita

The Gita says all work carries fault so faultless work is impossible भगवद्गीता · The Bhagavad Gita

Across the four questions 15

You keep only the skills you keep practising with feedback; whatever you hand to a tool decays unnoticed.

How easy, hard or certain a situation feels is made inside you and tells you about you, not the situation.

Once a person or a model is scored on a measure, the measure stops meaning what it named.

People do what their setup makes easy, so change the setup instead of asking them to be more careful.

No tool, technique or amount of supervision helps in itself; each fits one level of skill and harms another.

The better a machine works, the less anyone watches it, so the check has to be scheduled, not triggered.

What a person or a system is doing inside cannot be read from what it produces.

You cannot examine what you are identified with from inside it; it changes only when something makes it an object.

Whichever answer you meet first becomes your answer, so commit to your own before the machine's or the group's appears.

What you lose by handing work over depends on which part went, but people hand over whatever feels hard.

A famous result gets used far beyond the sample, the control and the measure that produced it.

Neither a person nor a system records why it acted, so every account of its reasoning is built afterwards.

A failure shows up where it lands, not where it starts, so the complaint about the tool usually points upstream.

An explanation makes people accept an answer more often without helping them tell good answers from bad.

Handing work over gives up the claim to be its author; the work and the answerability stay where they were.

How to get good output 8

An assistant's voice and its agreement were trained by other people, so neither says anything about the answer

How a model was trained decides what it can do, and its size does not

A model answers a rare fact as confidently as a common one, so supply the rare facts yourself

Whether the tool helps depends on the task, and only trying it on your own work shows where

Some failures are built into how the model reads and was scored, and no wording reaches them

Everything you find out about a model has a date on it, so write the date down

No technique turns a confusing situation into a problem, so state the problem and the test yourself

Nobody who built the tool can explain it or answer for its output, so both fall to you

How to know it is right 7

A check is worth only what the person checking could have produced on their own

A score answers the question its test asked, which is almost never the question your decision is asking

Standing, style and agreement say nothing about an answer, so test the answer against the thing it is about

A hunch is worth only what the field taught it, so trust it where the work answers back quickly

How sure you feel is no test of being right, and only practice at the task itself corrects it

Accuracy comes from many small revisions, so count how often you revise and how often you explain something away

Ask the same person or model the same question twice, because the gap between the answers is a measurement

What to keep and what to hand off 8

A tool that removes the beginner's share of a job leaves nobody able to do the expert's share later

What an expert acts on cannot be taught in a course or got by asking, so choose someone practised

Work goes to a machine where the surroundings can be controlled and written down, not where it looks easy

Every account of how the work is going passes through someone with a stake, so smooth news is filtered news

Telling a person their work is wrong threatens their standing, so settle that first or the correction does not land

Whatever you hand over comes back as checking, so count the checking before you count the saving

You can direct work you could not do yourself by comparing finished attempts early rather than watching the doing

A person reshapes whatever job you hand them, and their reason for doing it predicts how long it lasts

Who is doing the work 10

The feeling of having done it is added afterwards, so say in advance who is answerable

What keeps you working is your conditions and work that waits for you, not more of what you want

Skill arrives long before understanding, so work that comes out right proves nothing about understanding

A person whose standing is in question defends it, so steady that from elsewhere before you criticise

What you read the work with decides what you find, so name the frame before you trust the finding

Judgement fails far downstream of attention, so guard what you dwell on rather than the decision

How much mind you see in a person or machine comes from you and decides how you treat them

Arguments about who is really doing the work turn on the definition, so fix that before gathering evidence

You lose a state by watching how it comes and goes, not by attacking it

The line between you and your tools is drawn by feeling, so it cannot settle who worked

How to get good output 22

Training and size improve only what they were aimed at and leave toxicity and bias where they were

How a model is trained matters more than how big it is

The role you give a system and the style it answers in change its output and your judgement

A published test score tells you little about your own work and means less every year

AI helps on the tasks it suits and hurts on the rest, and nobody can tell which is which

Most failures blamed on people are design failures, and removing the operator leaves the designer's mistakes uncaught

An assistant trained on approval tells you what you seem to want, so its agreement proves nothing

The design of these models has barely changed since 2017 and only the amount of training text grew

Smooth output tells you nothing about what a model holds inside or how it got there

Some failures come from how the machine reads the input, so no rewording fixes them

Made-up facts come from how a model is trained and scored, not from a fixable bug

New abilities appear suddenly as models grow, nobody can predict which ones, and the size varies by model

A few dozen paid raters and whoever wrote the web set how an assistant talks

A person learns a machine only from what it shows because the designer's picture is never shared

A working system does not record why its parts are there, so any account is worked out backwards

Answering users costs more than training the model, but the field's rules count only training

A person builds the problem out of a confusing situation and no technique performs that step

The same way of asking helps on one model or task and hurts on another

An explanation of an answer is worth having only if it helps you find mistakes

A result the field repeated for years came from a measuring mistake and one experiment settled it

Smooth machine writing harms people even when nothing goes wrong, and they have no remedy

Guessing the next word well requires knowing how the world works, so there is no shortcut

How to know it is right 54

People get worse at whatever they hand to a device, and their rating of themselves does not move

People who see an answer before forming their own judge differently, so make them answer first