77The Society of Mind

Marvin Minsky, The Society of Mind, Simon and Schuster, 1986

Linked to 23 chapters: supports 6 · contradicts 5 · mechanism 9 · same claim 11 · pairing 3

In the late 1960s Minsky and Seymour Papert set out at the Massachusetts Institute of Technology to build a machine that could play with children's building-blocks. It had a mechanical hand with pressure sensors in its fingertips and a television camera for an eye, and a laboratory of students spent years writing the programs it needed: one to find the edges of a block, one to check by touch that a block was where it looked, one to confirm that what had been planned inside had happened outside. The robot worked badly, and the manner of its failure was the discovery. Problems that educated adults call hard — mathematical logic in 1956, college calculus in 1961 — had yielded to programs holding no more than a hundred or so facts. Problems a four-year-old solves without noticing took until the 1970s and were not solved then. Papert had come to MIT in 1963 after five years with Jean Piaget in Geneva; the two ran the laboratory jointly for a decade, tried in the middle 1970s to write a book about societies of mind, and gave up because the ideas were not ready. This is that book, finished alone ten years later and built to match its own thesis: 270 essays, each exactly one page long, each too small to carry an argument, cross-connected so that the argument is in the arrangement. His daughter Juliana drew the illustrations. What the pages claim together is that a mind is assembled out of parts that are not minds, and that everything a person does arises from how those parts are organised — including the belief that a single person is doing it.

The world of blocks

The book begins by breaking one skill into pieces. A child builds a tower, and Minsky supposes an agent called Builder is in charge. Builder cannot lift anything; it only turns on Begin, Add and End. Add cannot lift anything either; it calls Find, Get and Put, and below those lie the muscle-motor agents that move a finger joint. Nothing in the chain resembles a small person, and that is the rule he sets himself at the outset: whenever an agent has to do anything complicated, replace it with a society of agents that do simpler things, so that no explanation smuggles in the thing it was meant to explain. He names the nineteenth-century chess automaton that Poe exposed as concealing a human player. A theory of mind fails the same way when any of its parts turns out to be clever.

What makes the decomposition bearable is a distinction he uses throughout. Watched from outside, with no idea of its workings, Builder gives the impression of knowing how to build towers. Opened up, it contains nothing that could be called knowledge, only a few switches arranged to turn each other on. Both descriptions are correct and neither is dispensable. This is also his answer to holism. He stages a dialogue in which a Holist proves that no box can hold a mouse, since no single board contains any mouse-tightness and six boards containing none cannot contain any between them. Containment is what the arrangement does: each board bars escape in one direction, and the combination bars them all. Words like life and mind were invented for how large assemblies behave, and it is a mistake to look for them in the smallest parts. Nor does this make psychology a branch of physics: the laws of thought depend on how brain cells are connected, which is why, he says, psychology will need hundreds of small theories where physics needed a dozen principles.

The next step is conflict. Add an agent called Wrecker, whose specialty is knocking things down, and the child's mind holds two agents that want opposite things at the same moment. Minsky refuses to settle this by negotiation, on the ground that agents this small cannot know enough to bargain.

The longer a quarrel persists among an agent's subordinates, the weaker that agent becomes among its own competitors, until another takes control and the quarrelling pair are dismissed. A dispute between Builder and Wrecker weakens Play-with-Blocks, which weakens Play, and then Eat or Sleep — which have grown stronger the whole time — carry the child away. Small distractions are easy to resist while work is going well; once trouble starts inside the work, the least disturbance can take you off it. There need be no topmost agent where the process stops, and the organisation is not a clean tree: to decide whether a line of blocks is two or three, See needs Move to shift the front one and Move needs See to check the arm's path. Loops like that require memory, which is why memory becomes the book's constant subject.

The Self

Minsky reserves the capital letter for what people mean when they say some part of them is really them — the part that thinks and wants and decides, and stays the same throughout. He declines to define it. Forcing definitions on things we do not understand does more harm than good, he says, and you can know what a tiger is without defining one. The useful question is not what Selves are but what our ideas about Selves are for, and asked that way the single idea becomes several: self-images, beliefs about what we are capable of and disposed to do, which we draw on whenever we make a plan; and self-ideals, beliefs about what we would like to be and ought to be, formed early and mostly beyond reach of consciousness.

The function he assigns them is conservative. A long project — learning a trade, raising a child, writing a book — will conflict with other ambitions, so it is not enough to set an agency working on it; something must stop us from turning that agency off again next week. Without ideals that change slowly, no plan could survive its author's next mood. The myth that a Self is a licence to escape ordinary cause and law has it backwards: our ideals are the chains we forge to keep from wrecking our own plans.

Why, then, is self-control so indirect? His example is an evening spent failing to concentrate. Bored and sleepy, he imagined a competitor, Professor Challenger, about to solve the same problem; anger at the prospect of being beaten kept him working another hour, and the strange part is that the problem was not one Challenger had ever cared about.

The reason for the detour is that the direct paths have been removed. A creature that could switch off hunger or pain would not survive, and a mind whose agency for Work could switch on Anger would spend its life fighting, so the connections were taken out in the course of evolution and fantasy supplies the missing ones. You cannot become angry by deciding to; you can imagine something that makes you angry. The conscious versions of the trick are no better: bribing yourself works only once your agencies have learned one another's dispositions, and like bargains struck with other people the bargains often fail. If self-control were easy, he observes, we would accomplish nothing at all.

Against this he sets the image of a single ruling Self, which explains nothing. Put a Voyeur-Puppeteer inside the head to watch an inner screen and decide what to do, and that inner Self needs a screen of its own, and a Self to watch it, without end; a thing with no parts supplies no pieces to explain anything with. The image persists because it is convenient: bodies occupy one place at a time, responsibility requires individuals, and we get confused when the same agencies are asked to do two similar jobs at once, so thinking seems to arrive in a single stream. Traits work the same way. Because friendship depends on being predictable, we teach ourselves to behave in accord with the descriptions others give us, and imagined traits make themselves real. Behind all of it lies his objection to the soul conceived as a spark that cannot grow: if souls live forever and never learn, they have traded all change for changelessness, a destiny indistinguishable from death. A painting's art is not in any of its daubs but in the network of relations among them, and minds are the same. "The value of a human self lies not in some small, precious core", but in the wide crust it spends a lifetime constructing.

Insight and introspection

We drive a car without knowing how its engine works, and Minsky's claim is that we drive our bodies and our minds the same way. Turning while walking means falling deliberately toward the inside of the turn and catching yourself on the next step, managed by hundreds of programs specialists cannot yet describe; what reaches consciousness is turn that way. The words we think with are signals in the sense that a brake pedal is a signal — their effects are assigned to them, not inherent in them, and the designer could as easily have wired the pedal to the steering.

This puts a limit on introspection that no effort can lift. Programmers debugging a program must stop it first, and machines have interruption hardware to freeze the program under inspection and a private memory to restart it afterwards. A brain running many processes at once cannot do that: freeze one and you have changed what you were studying, freeze them all and you cannot watch them interact. What a mind can do instead is an architecture.

Divide the brain into A and B. Connect A to the world; connect B so that A is B's world. B can then notice that A seems confused, or is repeating itself, or is buried in detail, and can inhibit it, stop it, or push it toward a higher-level view — all without any idea what A's activity means. Minsky is candid about the costs. A B-brain that mistakes the addition of a long column of numbers for a repetitive loop produces a person who complains of being bored; two halves watching each other too closely make the whole thing unstable; and there is no reason to stop at two.

The rest of the chapter catalogues further obstacles. Signals take time to cross a mind, so when you meet a friend the agency for Places, which hears from Faces, and the agency for Quotes, which hears from Voices by way of Names, will disagree about whether you saw him or heard him first; each agent lives in a slightly different world of time. And the blindness is protective. A mind able to reach into its own pleasure systems could produce the satisfaction of success without any accomplishment, and that would be the end of everything. The self-ideals formed in infancy are the hardest agents of all to reach, which is a mercy: an ordinary change of mind can be reversed, but change your ideals and nothing is left to turn you back.

Intelligence is not a thing to be defined: it is the name we give to whichever problem-solving processes we do not yet understand, so the word empties as the science advances, like the phrase the unexplored regions of Africa. What can be described are methods, and the one he thinks most useful is a difference-engine, studied in the late 1950s by Allen Newell, C. J. Shaw and Herbert Simon: it holds a description of a wanted situation and contains subagents aroused by particular differences between that and the actual one, each acting to reduce the difference that woke it. Persistence plus a description of what is wanted is all that having a goal amounts to, and he refuses to argue about whether such a machine really wants anything. Learning by reward he considers close to circular, since you must be able to do a thing before you can be rewarded for doing it — and a hard problem means reusing an agent in different contexts, so the same agent gives good advice at one moment and bad at another, and a reward arriving at the end reinforces both.

A theory of memory

The theory that answers to all this is the simplest machine in the book. Whenever you solve a problem or have an idea worth keeping, a wire-like agent attaches itself to whichever agents were active at that moment. Activate it later and it switches the same agents back on, putting you roughly into the state you were in before, which should make the next similar problem easier. He calls it a Knowledge-line, or K-line, and credits Kenneth Haase with the image that explains it: smear your hands with red paint before repairing a bicycle, and every tool you touch ends up marked red; next time, take out the red tools first. Use a different colour for each kind of job and a tool may carry several marks, which is to say an agent may hang from many K-lines.

States are the units the theory works in. A mind is in one total state and many partial ones at once, which is how it thinks several things simultaneously, and the arrangement explains a familiar failure. Imagining a small white ball is easy because the agencies involved do not overlap, while imagining a round square forces the agents for round and for square to compete for the same shape-describing agency; if the conflict is not settled, the principle of noncompromise removes both and leaves an undefined shape.

The refinement that makes this useful is that the attachments are not equally firm. A K-line holds tightly at a certain level of detail and weakly above and below, and those weak edges are where default assumptions live. Hearing that Jack is flying a kite arouses string, which nobody mentioned, and a colour and a shape drawn from the kites you have seen, and those weak connections give way the moment you are told the kite is green. The lower fringe supplies structure — detail that helps only until it becomes so specific that it matches nothing new, since past a certain magnification you are no longer seeing a kite but this particular piece of paper. The upper fringe supplies function, the goals the memory once served, which is why a child who learned to stack blocks can use the middle of that skill to build a house while discarding the top and the bottom. A memory without weakened edges would be a disaster: relive a moment in full detail and you would erase the present self that asked to remember it.

K-lines can attach to other K-lines rather than to agents, which is cheaper and more orderly. Recognising that Jack was flying a kite already used the lines for Jack, for flying and for kites, so the new memory needs three connections rather than thousands. What is lost is perceptual detail; what is gained is that memories come out shaped like the recognitions that made them. A society of K-lines can then grow as a layer beside the society of agents it records, with a B-brain governing the pair by choosing which band stays active, understanding nothing inside.

Papert's principle

The best evidence for the whole scheme, Minsky thinks, comes from watching children fail an experiment. Piaget showed a child two matching rows of eggs and egg cups and asked which there were more of; every child said they were the same. He then spread the eggs apart, in full view, and asked again. Five-year-olds said there were more eggs; seven-year-olds said they were the same, because they were the same eggs. Pour the water from a short wide jar into a tall thin one and the answers divide the same way. The change happens without anyone teaching it.

The tempting explanation is that older children have acquired a concept of quantity, and Minsky rejects it. Describe the experiment without performing it, or perform it behind a screen, and many of the younger children answer correctly. They already have the ideas; what they lack is knowledge of when to apply them. He models this with three agents. Tall says the taller thing holds more, Thin says the thinner thing holds less, and Confined says the amount is the same because nothing was added or taken away. A young child settles the disagreement by priority — Tall decides if Tall is aroused, otherwise Thin, otherwise Confined — which serves well in general life and fails in exactly this case.

What the older child has is two new administrators. Appearance reports more when Tall is active and less when Thin is active, and says nothing when a thing looks both taller and thinner; History reports what Confined says. Group Tall with Thin and let them cancel, and Confined takes over; group Tall with Confined instead and nothing improves. Papert proposed this in the 1960s, and Minsky names the generalisation after him: some of the most consequential steps in mental growth consist not of new skills but of new administrative ways to use skills already possessed. Drill can make a child appear to understand conservation, but the structures that result transfer only to situations almost identical to the ones they were learned in. He tells a story against himself: a visiting student of Papert and Piaget tried the water jars on his five-year-old daughter Julie, who answered that it looked like there was more in the tall jar but that he should ask her brother Henry, who had conservation already.

The same chapters supply his account of learning a meaning. A child playing with blocks and a toy car builds a structure that traps the arm: to push the car through you must let go and reach around, which Minsky calls the Hand-Change phenomenon. Curious about what causes it, the child starts revising a description.

Two standing blocks and a lying block will not do, because a structure with the standing blocks touching fits that description and produces no Hand-Change; so the description acquires the condition that the standing blocks must not touch. That is still not enough, because a structure in which they do not hold up the lying one also fits; so they must support it. Then a version with a wedge on top does produce Hand-Change, so the description must become neutral about the shape of the top. Minsky names the three operations — enforcement, prevention and tolerance — and calls the result a uniframe, a description built to cover several different things at once. The scenario reworks a program written by Patrick Winston in 1970, with one change he treats as the point: Winston's program needed a teacher to say which structures were arches, and here the teacher is replaced by the child's concern to account for a strange event. People do not learn well unless they are interested, he says, and older theories lost that by trying to simplify. Nothing that all arches share can be stated in the language of blocks, so the escape is to change languages: draw an imaginary line and every arch becomes a body with two supports. Where no single description can be found — money, furniture, chairs — we accumulate examples instead, and when a rule mostly works we keep it and list the exceptions, since a rule patched for every penguin becomes unusable.

Consciousness and memory

Halfway through the book Minsky states the thesis the machinery was assembled to support: consciousness is not knowledge of what is happening now, but the use of records of what happened a moment ago. There is nothing strange about an agent wired to detect an event inside the brain — most agents detect brain events already. The ones that matter are those that use and alter our most recent memories, and every puzzling feature of awareness follows from the limits on those records. We become less conscious of one thing as we grow more conscious of another because the records displace each other, and thoughts seem to flow in a stream for the same reason. We cannot say how we get our ideas because while solving a hard problem the short-term memories are busy solving it and have no room to record what they themselves did.

From this he derives a companion illusion. Whenever a question can be answered without noticeable delay, it seems as though the answer had been in the mind all along, which is what gives perception its air of immediate contact. Seeing a telephone brings with it how it would feel in the hand, what it weighs, what its other side looks like, none of which is on the retina; the vision agencies begin arousing memories before their own work is done, and you cannot tell what you saw from what you were led to recollect. The same limits explain why we cannot remember infancy. A memory is not a stored object but a process that makes agents act as they acted before, so to retrieve an early experience you would have to reconstruct how an infant mind reacted to it, and the infant mind is gone. Childhood recollections are reconstructions, and the telltale sign is that they are seen from outside, with the narrator standing in the scene — a view nobody ever had.

The second half builds representations to fill this out. Emotions are types of thought, each grounded in a machine specialised to a need; and since humans learn new goals and not merely new means, genes cannot specify what is good for us, so Minsky proposes that being scolded by a stranger changes which goal you pursue in which situation, while being scolded by someone you are attached to changes which goals you think worth having at all. Language gets a family of agents with a shared suffix.

Frames are the structure all of these hang on, and the idea Minsky had already published in 1974. A frame is a skeleton with blanks, which he calls terminals: a chair-frame has terminals for a seat, a back and legs, and the terminals normally arrive with something already attached.

When someone sits across a table from you, no part of the chair may be visible, and a sitting-frame supplies one by default, because its terminal for what to sit upon has a stereotyped chair attached to it. The alternative, he argues, is not better perception but none: seeing how things actually look would be like watching the dots on an untuned television, and the very idea of an object is a bundle of assumptions that go without saying — that it has boundaries, that it existed before you looked, that its unseen sides are there. Frames also explain the speed of sight, since what you react to on meeting someone is less what you see than what the sight reminds you of — the same mechanism we call stereotyping when we disapprove of it and empathy when we do not.

The last chapter gives the self the same treatment as everything else. Knowing something means having a model of it, where a model is whatever helps answer questions, and we model ourselves no differently, as self-made answering machines. Ask what sort of thing you are, and the model of your model of yourself comes out physical at one end and psychological at the other, and the regress stops not at bedrock but at confusion — because I simply wanted to. Free will he treats as a myth of a third alternative: science offers deterministic law and random chance and no third slot, so we postulate a small region called Spirit or Will whose territory has shrunk in every century, and we cannot give it up because responsibility and self-ideals are built on it. What we should give up instead is the expectation of a single principle. The book ends on eight kinds of diversity — accumulated agents, realms of thought, inherited proto-minds, layers of management, evolutionary vestiges, developmental stages, language and culture, and the censors — and on a sentence that does duty for the whole: "The trick is that there is no trick." An appendix carries the biology the one-page essays could not hold, and its best idea is that genes cannot encode a concept but can lay out an architecture destined to learn one kind of thing: wire a recognizer only to detectors for human odours, voices and faces, and it has no choice but to learn to represent people. The postscript is franker than the book. He made hundreds of assumptions deliberately, on the ground that it was too early to weed hypotheses out, and he removed every possibly and every reference to evidence, asking that the book be read less as scholarship than as an adventure story for the imagination.

Since 1986

The word won. Working systems are now assembled as societies of components with a manager that decides which to call, and the word for a component is agent, in the sense Minsky gave it. So did the claim that competence need not be understood by whatever possesses it — the position Dennett restated thirty years later in From Bacteria to Bach and Back as the inversion that comprehension is composed of competences rather than their source. So did frames. The idea that perception delivers a stereotype with slots, filled from the world where the world supplies something and from expectation where it does not, survived into psychology under the older name schema, and is the ancestor of the account Anil Seth gives in Being You, where the brain builds models it cannot recognise as models — Minsky's illusion of immediate contact, given a mechanism at last. K-lines fared similarly: as neural hardware they were never tested, but the claim that remembering means reinstating a pattern of activity present at the time is now ordinary in memory research.

What has not held up is the specific machinery, and the postscript half-concedes it. Polynemes, pronomes, isonomes, micronemes, B-brains and level-bands were offered without evidence and produced almost no experiments; the coinages are nearly absent from the literature that followed. Its form is part of the reason: each idea is a page that ends before an objection can reach it. So is the developmental psychology it rests on. Piaget's conservation findings were contested within a decade — children pass when the transformation appears accidental rather than deliberate, as in the 1974 study by James McGarrigle and Margaret Donaldson — though the moral was closer to Minsky's own defence than to Piaget's: the children had the competence and were being asked the wrong question.

The largest thing the book got wrong is visible in its own appendix. Minsky and Papert had proved in 1969 that a machine which weighs features cannot see relations among them, and the argument is restated here. In the same year this book appeared, David Rumelhart and James McClelland published the two volumes on parallel distributed processing that restarted the other tradition, and Minsky cites them in the appendix, along with John Hopfield's network of 1982, and concedes that he suspects most of the human brain is composed of distributed learning systems. He then argues that such systems cannot be the whole story, because they lack explicit articulated representations — which is why, he adds, consciousness is so shallow. Forty years on, the machines that do what the blocks robot could not do were built by that other tradition, out of one large learned system rather than a bureaucracy of hand-built parts, and the layers of management that Papert's principle made central are the part nobody has had to construct. The companion prediction failed with it: the Connection Machine, designed by his student Danny Hillis at Thinking Machines Corporation and described here as embodying societies of mind, was a commercial failure, and the company went bankrupt in 1994. His complaint that such systems cannot give an articulated account of what they are doing is still the standard complaint — so he lost the engineering argument while keeping a grip on one end of the psychological one.

Minsky kept working on the same problem. He backed a later effort at MIT to gather commonsense knowledge from volunteers over the internet, on the book's conviction that common sense is an immense accumulation of hard-earned practical ideas, and in 2006 he published The Emotion Machine, which rebuilt the theory around six levels of mental activity. He died in January 2016, and Papert died six months later. The observation the two of them made with the robot arm, that the easy things are the hard ones, acquired a name two years after this book from Hans Moravec and is still cited under it.

Of the people who read the manuscript, the one who pressed hardest was Douglas Hofstadter, whom Minsky records as having evaluated the whole theory and forced several substantial changes. Hofstadter took the opposite road in I Am a Strange Loop, where the self is one pattern that has come to perceive itself rather than a committee with nobody in charge, and where the remark that a dead wife goes on being thought with, in the brains of those who knew her, reaches him secondhand from Minsky. Both books deny that anyone is inside; they disagree about whether what remains is a loop or a bureaucracy, and neither has an experiment that would settle it.