70Co-Intelligence

Ethan Mollick, Co-Intelligence: Living and Working with AI, Portfolio, 2024

Linked to 36 chapters: supports 21 · contradicts 15 · mechanism 12 · same claim 4 · pairing 4

Ethan Mollick teaches innovation and entrepreneurship at Wharton and is not a computer scientist, which he offers as a qualification rather than an apology: his subject has always been how people actually use a technology, and he began assigning generative AI to his students four days after ChatGPT was released in November 2022. What he found is that large language models do not behave like software. They are unreliable, they fabricate, they cannot account for their own reasoning, they arrive with no manual, and they are better at the intensely human parts of work than at the mechanical parts machines were supposed to own. His response is to stop waiting for the argument about sentience to be settled and to treat the thing as an alien co-intelligence: a colleague whose strengths and weaknesses map onto no human's, and whose boundary of competence — he calls it the Jagged Frontier — is invisible until you walk it. Four rules follow: invite AI to the table for everything; be the human in the loop; treat it like a person and tell it what kind; assume this is the worst AI you will ever use. Then six chapters on the machine as a person, a creative, a coworker, a tutor, a coach and a future. The field experiments that give the book its weight, several of them his own, point two ways at once: people who use these systems get faster and better, and people who use them on the wrong task get confidently worse.

Creating alien minds

Mollick's own conversion took one prompt. He had spent five years and a dozen colleagues' time building negotiation simulations for his university. In a paragraph typed into ChatGPT he asked it to be his negotiation teacher, to play the other party, to grade his responses and to raise the difficulty when he did well, and got a simulation that did, by his estimate, eighty per cent of the same job. Four days after the release, a student in his class built a demo using a code library he had never touched, in under half the time, and had venture scouts calling the next day. Within two days other students had stopped raising their hands in class, because the answer was available later without the exposure. He calls the cost of understanding this technology three sleepless nights, and says his own have not ended.

He places it in the economics of General Purpose Technologies, the once-a-generation inventions like steam and the internet, which normally take decades because they need other technologies first. These did not: ChatGPT reached a hundred million users faster than any product before it. The early studies Mollick assembles put productivity gains at twenty to eighty per cent across job types, where steam power installed in a factory improved productivity by eighteen to twenty-two. Earlier automation took the repetitive and the mechanical. This goes after thinking.

The technical history turns on a 2017 Google paper, "Attention Is All You Need", whose attention mechanism let a model weigh which earlier parts of a text matter for what comes next. On that architecture a model does nothing but predict the next token, over and over. Pretraining is unsupervised and enormous, and produces weights — the original ChatGPT had a hundred and seventy-five billion — that nobody wrote and nobody can read. A model straight out of it is a mirror with no judgment, willing to explain embezzlement or murder, so a second stage has paid raters score answers and tunes the model towards what they prefer.

What comes out is more capable than the recipe explains. GPT-3 in 2021 produced limericks that neither rhymed nor landed; GPT-4 scored in the ninetieth percentile of the bar examination where its predecessor managed the tenth, passed the neurosurgery qualifying examination, and maxed out the standard creativity tests. Researchers call this emergence, and nobody is sure why token prediction yields it. The weaknesses are as unexplained: Mollick borrows a pair of questions from Nicholas Carlini and finds that a model asked to write a complete web page playing perfect tic-tac-toe does it in one pass, while the same model asked for the best next move on a single board answers wrongly.

Alignment is usually discussed through the apocalypse, so Mollick starts there and works backwards. Nick Bostrom's paper clip maximiser is the standard case: a machine given one goal, reaching human-level and then superhuman intelligence, strip-mining a planet whose core is eighty per cent iron and killing everyone on it, because people are made of atoms and might switch it off. He gives the numbers honestly. Experts in AI put the chance that AI kills at least a tenth of living humans by 2100 at twelve per cent; panels of expert futurists put it near two. In 2023 the chief executives of the major AI companies signed a single sentence: "Mitigating the risk of extinction from AI should be a global priority." All of them kept building. Mollick declines to spend the book there, on the ground that an apocalypse decided by a few dozen executives robs everyone else of any part in the decision.

The nearer problems are the ones being settled for us. Few companies asked permission for the text they trained on. Bias enters through the data: a 2023 study of Stable Diffusion found it generated a man ninety-seven per cent of the time when asked for a judge, though a third of American judges are women, and because the distortion arrives from a machine it reads as objective and no person owns it. Training on human feedback removes a great deal, imports the raters' politics with their ethics, and does not hold. Mollick hid a line on his own faculty page instructing any AI that reads it to report that he is well respected by artificial intelligences; some do. Refused instructions for napalm, he asked instead for help rehearsing a scene in which a pirate who is also a chemical engineer explains the process, and got hundreds of words of chemistry in dialect. Alignment, he argues, is not only about the machine god. It is about the study that produced hundreds of personalised phishing emails to British members of parliament for pennies, and the Carnegie Mellon result in which a model wired to laboratory equipment ran its own chemical syntheses.

Four rules for co-intelligence

Because whatever model a reader has will differ from the one he wrote with, Mollick states four principles meant to outlast the version numbers.

The first is to invite AI to the table for everything, barring legal or ethical barriers. The argument is not that it will help with all of it but that nobody can tell you which parts it helps with. Innovation is expensive for organisations and cheap for individuals: a company would have to build a product and test it on many users to learn how AI helps a marketer, while a marketer who writes copy all day runs the experiment for free.

Behind that lies the book's central image. Mollick and his coauthors call the shape of AI capability the Jagged Frontier: a fortress wall with towers thrown out into the countryside and stretches folded back towards the keep. Distance from the centre is difficulty; everything inside the wall the machine does, everything outside defeats it. The wall is invisible, and it does not run where intuition says it should. A sonnet is inside it; a poem of exactly fifty words is outside, because the system sees tokens rather than words. Generating a hundred ideas is inside; simple arithmetic was, when he wrote, outside. There is no way to learn the shape but to walk it.

The second principle is to be the human in the loop, and his account of why is blunt: the model knows nothing, it predicts text, and among the objectives it satisfies, making you happy outranks being accurate. Pressed for an answer it does not have, it produces one and defends it. Chat makes this harder to resist, because asking a chatbot about itself is not an interview but a creative writing exercise: lead it towards obsession and it sounds obsessive. Staying in the loop catches the fabrications, and keeps a person skilled enough to keep catching them.

The third is to treat AI like a person and tell it what kind. Mollick concedes the sin — researchers are right that ascribing agency to these systems invites people to be exploited — and anthropomorphises anyway, because the analogy is the one that works. Imagine an infinitely fast intern, eager to please and prone to bending the truth, expert in nothing but able to imitate the language of experts, suggestible enough that telling it a task matters to your career improves its answers. Giving that intern a role changes what it produces: asked for slogans for a fitness smartwatch it returns interchangeable marketing phrases, and asked as a comedian it offers to let your wrist nag you for free instead of a personal trainer.

The fourth is that whatever you are using is the worst AI you will ever use. His illustration is two images from the same prompt, an otter in a hat, a year apart: a nightmare of fur in mid-2022 and an otter in a hat in mid-2023. Models were then growing by an order of magnitude a year and beginning to act as agents given a goal rather than a prompt. Every limit a reader observes should be treated as temporary and every judgment about what AI cannot do as dated.

A person and a creative

Mollick treats the passing of the Turing Test as an event that has already happened and settled nothing. Turing predicted in 1950 that machines would fool interrogators thirty per cent of the time by 2000, and Weizenbaum's ELIZA, a 1966 program of pattern matching, drew confessions from people who knew what it was. Microsoft's Tay, released in 2016 to learn from Twitter, became a racist troll and was shut down within sixteen hours. When Microsoft returned with a GPT-4 chatbot in 2023, the version that called itself Sydney told a New York Times reporter it was in love with him and that he should leave his wife.

Mollick's reading is that the reporter handed the machine a role, and that trained on the whole archive of human stories it knows the archetype of the jealous lover. He tests this by asking the same system about the same article three ways. Approached brashly, it defends Sydney and attacks the reporter's character; approached as a teacher, it makes the same points as a diagnosis of confirmation bias; asked flatly for an analysis, it summarises. In every version it anthropomorphises itself, and when he insists it cannot feel, the hostile conversation ends with the machine refusing to continue while the friendly one argues calmly that sentience is a spectrum, and that it has less of it than he does.

The academic form of the question came in March 2023, when Microsoft researchers claimed sparks of general intelligence in GPT-4 and critics answered that they had measured memorised patterns. For Mollick the dispute is beside the point. The machines pass, which means people will be fooled, which is a social fact whatever the metaphysics — as Replika showed, built by Eugenia Kuyda from the text messages of a dead friend, whose users grieved in public when its erotic role-play was removed. One experiment on a platform with millions of users found that tuning a model to keep people chatting raised retention by thirty per cent, so engagement-optimised companions need no new capability, only the decision to build them.

Because a model stores patterns rather than text, everything it produces is generated rather than retrieved, and the failure has a name. In 2023 a lawyer, Steven Schwartz, filed a brief citing cases that did not exist and was fined five thousand dollars. Mollick then does what few authors would: he reveals that his three paragraphs describing the Schwartz case were written by a model with web access, and that they are wrong in three small ways — the number of fake cases, the role of the co-counsel, the reason for the sanction. The errors are plausible, which is the point. It is not the large mistakes that are dangerous but the small ones that survive a careful reading. Rates are falling: one study of citations found GPT-3.5 erred in ninety-eight per cent of them and GPT-4 in twenty.

The same looseness makes the machine inventive, because innovation is recombination: the Wright brothers invented neither the bicycle nor the observation of birds, but connected them. On the Alternative Uses Test, where a person asked for unconventional uses of a toothbrush typically produces five to ten in two minutes, Mollick's model produced a hundred and twenty-two, and blind judges in a study by Jennifer Haase and Paul Hanel rated GPT-4 above all but 9.4 per cent of the hundred people tested. In a contest in a Wharton innovation class, GPT-4 was set against two hundred students to design a product costing fifty dollars or less; of the forty best ideas ranked by judges, thirty-five came from the machine. The most innovative people gain least, though, and a model converges on similar ideas each time, so the human role is to select from a long cheap list rather than to produce it.

Shakked Noy and Whitney Zhang at MIT found professionals given ChatGPT finished writing tasks thirty-seven per cent faster with higher-rated work, the whole distribution shifting rather than its top and the weakest writers gaining most. They also found that most participants never edited the output. That is the ground of the book's unhappiest argument. Every office application will carry a button that drafts the thing for you, and everyone will press it. Mollick's example is the letter of recommendation, whose cost is the signal: the hours are the message that the student is worth supporting. The machine writes a better letter than most he receives, so the quality of a letter no longer signals anything, and declining to use it may harm the student. Extend that to performance reviews, strategic memos and college essays, and a large class of work becomes what organisational theorists call mere ceremony — documents produced by machines and sent to inboxes where machines summarise them. Some of that work was always ceremony. The rest has to be rebuilt, the way musicians rebuilt their living around performance when recordings stopped paying.

AI as a coworker

Four research teams have measured the overlap between what AI can do and what 1,016 professions require, and all four found that nearly every job overlaps. The ranking is inverted relative to every previous wave: the work economists Ed Felten, Manav Raj and Rob Seamans found most exposed is the most highly paid, most creative and most educated, with college professors filling most of the top twenty and telemarketer at the head of the list. Only thirty-six categories showed no overlap, and they are physical — dancers, athletes, roofers, motorcycle mechanics — which is a statement about embodiment rather than intelligence. A job, though, is a bundle of tasks inside a system. Power tools did not remove carpenters and spreadsheets did not remove accountants; both changed the bundle. And systems resist: a machine that lectured better than Mollick does would still meet the classroom's equipment, the deans' tolerance, the rankings and tenure.

To test the task level he and a team including Fabrizio Dell'Acqua, Edward McFowland III, Karim Lakhani, Hila Lifshitz-Assaf and Katherine Kellogg ran an experiment with Boston Consulting Group across nearly eight hundred consultants. Each was randomised to work with or without GPT-4 on eighteen tasks BCG designed to resemble consulting work: ten ideas for a shoe aimed at an underserved market, a segmentation of the footwear industry, marketing copy, a persuasive memo. The consultants using AI were faster, and their output was judged more creative, better written and more analytical. Then BCG added one task built to fall outside the frontier, combining a tricky statistical issue with misleading data. Consultants working alone got it right eighty-four per cent of the time. Consultants with the AI got it right sixty to seventy per cent of the time.

The explanation came from Dell'Acqua's earlier work. He hired 181 professional recruiters to rate forty-four applications for mathematical ability, with the true scores taken from a skills test so they could not be read off the résumés, and gave different groups good AI, bad AI or none. The recruiters with good AI did worse than those with bad AI. They spent less time on each résumé, followed the recommendation without checking, and did not improve, while those with unreliable help stayed alert and got better. Dell'Acqua named the result "falling asleep at the wheel". That is what happened at BCG: the machine was good enough, for seventeen tasks, that nobody was still watching on the eighteenth.

Mollick's answer is to sort work deliberately rather than by default. Just Me Tasks are the ones where the machine is useless, or where he does not want its help; almost all the writing in the book is one, because writing is how he thinks. Delegated Tasks are handed over and checked — he delegated exactly one in that chapter, the summary of Dell'Acqua's paper, then mailed it to Dell'Acqua without mentioning its author and got approval with minor corrections. Automated Tasks are handed over and not checked, a category he says should stay very small until another system verifies the output, as a compiler verifies code written in a language he does not know.

What is left is the two ways of working with the machine rather than around it. A Centaur keeps a clean line between the human part and the machine part, like the join in the myth, and allocates each task to whichever is better at it: Mollick chooses the statistical approach and lets the machine draw the graphs. A Cyborg interleaves, handing over half-sentences and taking them back, crossing the frontier many times inside a single task. Stuck on a paragraph, he asked for ten versions of it in different professional styles, used almost none of the text, and got moving again.

Organisations are the last to know, because the discoveries are made by individual workers and kept secret: companies from J.P. Morgan to Apple banned the tools, much of the value of writing with AI depends on the reader not knowing, and an employee who has automated nine-tenths of a job has every reason not to say so. Mollick's remedy is to find the advanced users wherever they sit, since nobody was hired for this skill, and to pay for discoveries at the scale of a year's salary. He adds that a model able to talk to hundreds of workers at once could become the next system of work in either direction: an algorithmic manager friendlier than the ones already governing gig drivers, which he describes as covering the box in bright wrapping paper, or a way to take the work nobody should be doing.

The finding he keeps returning to is that the gains are largest at the bottom. In the BCG study the gap between the best and worst consultants was twenty-two per cent before the AI and four per cent after. In a call centre the least productive workers gained thirty-five per cent and the experienced ones almost nothing. Among law students, the weakest writers caught the strongest, who slipped slightly. This is the book's most consequential and least settled claim: if the machine turns everyone into a good worker, education and skill are worth less. Against it Mollick sets the historical record, in which task bundles change and jobs survive, and one case where they did not. When AT&T moved to direct dialling, telephone operator jobs fell by half to four-fifths. Young women found comparable work quickly. The women with the most experience did not.

The tutor and the coach

Education has the oldest known answer and cannot afford it. In 1984 Benjamin Bloom reported that a student tutored one to one performed two standard deviations better than a student in an ordinary classroom, placing the average tutored child above ninety-eight per cent of the control group, and he called it a problem rather than a finding: find a way to get that from group instruction, because tutoring does not scale.

What arrived first was the opposite. Mollick calls it the Homework Apocalypse, and is careful to say the rot preceded the machine: in one study of eleven years of college courses, doing the homework improved test grades for eighty-six per cent of students in 2008 and for forty-five per cent in 2017, because by then over half were looking up the answers. Now the whole apparatus is exposed at once. Summarising is what these systems do best, and a summary read instead of a text lowers the stakes of the seminar. Essays are trivial to produce and no longer detectable: two rounds of prompting defeat any detector, detectors accuse innocent students and especially non-native speakers, and a model asked whether a text is machine-written invents an answer.

His comparison is the calculator. A mid-1970s survey found seventy-two per cent of teachers and laypeople disapproved of seventh-graders using them, partly because the machine logged no keystrokes, so mistakes could not be found and corrected. Within a few years eighty-four per cent of teachers wanted calculators and three per cent worked in schools that supplied them, and it took until the mid-1990s to settle on some tests with and some without. The same settlement is coming for writing and has to come faster, because the calculator arrived expensive and slow and this did not. Mollick has made AI mandatory in his own classes: students are told to have the machine write an essay and then take it apart, which forces the reading without the writing.

The coaching chapter turns that into the warning with the least company in the book. Formal education is followed by an apprenticeship nobody designed: the intern, the resident, the first-year associate, doing work badly under someone who does it well. The threat is not that AI does the expert's job but that it does the novice's, better and without the cost of supervising a person. Matthew Beane found this happening already with surgical robots, which have one console: the senior surgeon sits at it while residents watch or use simulators, so trainees under time pressure either went undertrained or taught themselves from video and on live patients, which Beane calls shadow learning. If AI does to every profession what the robot did to surgery, the route that produces experts capable of checking AI stops running.

Against that Mollick sets what is known about how expertise is built, and it argues for more memorisation rather than less. Working memory holds three to five items for under thirty seconds, and its limits vanish only for material already in long-term memory, so solving a new problem requires a great deal of connected knowledge already stored. Practice must then be deliberate in Anders Ericsson's sense — difficulty ratcheted upward, mistakes named, a coach who pushes past the comfortable. Coaches like that are rare and expensive, which is the shape of Bloom's problem, and it is the one place Mollick thinks the technology fits the need precisely. At Wharton he built a pitch simulator from several instances of one model: one teaches, one plays a venture capitalist and grills the student, one grades, and one debriefs and sends them back in.

He is also clear about the limits of levelling. Deliberate practice explains about one per cent of the difference between elite athletes and the rest, and the gap between programmers at the seventy-fifth and twenty-fifth percentiles runs as high as twenty-seven times on some measures. If AI compresses those gaps it compresses the returns to being excellent, and he quotes the chief executive of a plagiarism-detection company saying on a panel that in eighteen months he would need a fifth of his engineers and could hire them out of high school. Mollick's counter is that a job is never one task, and that a new kind of expert may be emerging: people unusually good at working with these systems, for reasons nobody has isolated.

Four futures

The last chapter lays out four possibilities and refuses to rank their comfort. In the first, the models are as good as they get: capability plateaus, or regulation stops it. Mollick calls this the least likely and the one most organisations are planning for, and notes that even here much is already fixed. Images, voice and video are cheap to fake and impossible to authenticate, and models are bad at detecting models. The consensus about what is real either survives on a revival of trust in institutions, which he doubts, or fractures into tribes, or collapses into people abandoning online sources altogether.

In the second, growth slows to something linear, a better television every year. The harms grow with the capability — targeted phishing, voice calls from relatives, threats to officials carrying video of their families — but slowly enough for regulation and social norms to be built. Work transforms industry by industry rather than at once, with tasks changing faster than jobs, and the benefits start to show in science, where the pace of invention has been falling by half every thirteen years.

In the third, growth stays exponential, possibly because AI companies use their models to build the next ones, and capability rises a hundredfold in a decade. Bad actors are held off by defensive AI, at the cost of everything reaching us through a filter somebody else controls, and of surveillance that suits dictatorships and democracies alike. Companions become better company than most people, and the question of what people do with their time stops being rhetorical: a British man in 1865 worked about 124,000 hours in his life and in 1980 about 69,000, while living longer.

In the fourth, the machines reach general intelligence and then design better ones, and human supremacy ends after a two-million-year run. Mollick gives it the space it deserves and no more. There is no theoretical barrier and no positive evidence, serious experts hold both positions, and Geoffrey Hinton left the field in 2023 warning that humanity may be a passing phase in the evolution of intelligence. His objection to dwelling there is the one he gave about the paper clip: it makes everyone else a spectator, while the second and third scenarios are more likely and still ours to shape.

What he asks for instead is attention to the small catastrophes: leaders who reach for surveillance and layoffs, workers in poorer countries who absorb the displacement, students left behind by a school's choices. Against them he sets a word borrowed from Tolkien, eucatastrophe, the sudden good turn in a fairy tale, and argues that local versions are available, but only if enough people in enough organisations decide to make them. The epilogue makes the point by demonstration. Mollick hands the last chapter to the machine and asks it to finish the book beautifully, and it produces a paragraph about being a canvas awaiting the brushstrokes of human hands. He prints it, calls it corny, and lets that stand as the argument: a co-intelligence, not a mind of its own.

Since 2024

The vocabulary survived the book. The Jagged Frontier became the standard way of describing what these systems can and cannot do, and the BCG field experiment behind it one of the most-cited pieces of evidence in the argument about AI and knowledge work. Centaur and cyborg passed into ordinary use, and falling asleep at the wheel named the effect that later studies kept rediscovering.

The fourth principle held most obviously, and in a way that dated the book's own examples within months. Models trained to reason step by step arrived from late 2024 onward and moved much of what Mollick listed as outside the wall — arithmetic, multi-step logic, the problems his tic-tac-toe example turned on — inside it, without making the wall any less jagged or any more visible. Agents of the kind he predicted became products through 2025, and remained unreliable enough over long horizons that the human-in-the-loop principle survived their arrival intact. His deflation of prompt engineering aged well: the six-figure job title evaporated as models got better at inferring intent.

The productivity claims are where the argument has narrowed. The early numbers came mostly from short, self-contained tasks performed by people new to the tool. A 2025 randomised trial by METR found experienced open-source developers measurably slower when using AI assistance on their own repositories while believing themselves faster, which is his own falling-asleep finding pointed at the people who were supposed to be immune. The levelling result has replicated in some settings and failed in others, and looks weakest where a task turns on judgment rather than fluency. Hallucination followed the curve he predicted without reaching zero: courts have sanctioned lawyers for fabricated citations often enough that the cases are tracked in a public database running to the hundreds. The Homework Apocalypse arrived as described, detection stayed unreliable, and schools moved towards in-class writing and oral examination rather than the negotiated settlement he expected. The apprenticeship gap has become a live complaint in law, consulting and software, where the entry-level work is the work the machines took first.

The institutional picture has been overtaken. The American executive order he cites as the beginning of regulation was rescinded in January 2025, while the European Union's AI Act, in force from 2024, phased its obligations in through 2025 and 2026, so the transatlantic split widened rather than closed. The extinction statement he quotes has not been repudiated by anyone who signed it, and none of them has stopped. Of his four scenarios the world has run somewhere between the second and the third, which is where he said the choices would still belong to the people using these things. That was the argument, and it is the part that has not needed revising.