1The door is rare. The machine is not.
You pull the handle. The door refuses. Then you see push, and for a second an object has made you feel stupid.
Don Norman built a way of thinking about design around that second. The person knows where they want to go, the object gives them the wrong account of how to get there, the fault is the object’s, and a good designer removes it. It is a generous theory and it has made the world a little better. Adorno had noticed something less generous forty years earlier: the door does not only mislead your hand, it trains it. The door that has to be slammed, the car door, the door that slides, each one teaches a gesture and abolishes another, and after enough repetitions the awkwardness is no longer outside you; it has moved in.1
Norman’s own answer to the door was never the sign. The sign is an apology for a lie the handle told. His answer was to take the handle off the push side and put a flat plate there, so the door tells the truth about what it wants from you and nobody has to be told anything. Almost all bad design is that one confusion: the material world saying one thing and the meaning saying another, and a label can never fix that.
Norman called his book The Psychology of Everyday Things, and ever since, doors, kettles and light switches are what people picture when they hear the word design. But things like these are rare. Count your day. You go through a door perhaps forty times: the flat, the lift, the office, the meeting room, the toilet, the car. Now count the machines. You open an email and a machine has already decided which part of your morning it belongs to; you accept a meeting and a machine has decided how long a conversation is; you type a name into a field and the machine tells you the name does not exist in the form it prefers; you pick a category from a list written by someone who never met your customer; you approve, decline, forward, tag, snooze, submit, and the form clears itself and asks again. If you touch an interface every fifteen seconds of a working day, and you do, that is two thousand touches before you go home. Two thousand small doors, each with its own push, each one teaching your hand what to expect, set against forty that teach nothing. You have been corrected by a form more often this year than by anyone who loves you.
Norman’s was the psychology of things; ours is the psychology of machines, and the difference between them is not the object but the frequency: the dose.
I recently, without thinking about it, recognised a piece of software from a sheet of paper. A purchase request had been broken into things to sign in a way that could only have come out of SAP, and I could have told you the version and the module before I saw a screen, the way you can tell a man was once a soldier from how he stands in a queue. I knew more about the system than the purchase required. Years of using machines had left some of their organisation in my head, living there rent-free, and nobody had asked me whether I wanted a tenant. There is a scene in The Matrix where Cypher sits at the monitors with the code streaming down them and says he does not see the code any more, only the blonde, the brunette, the redhead, and that is the imprint at its far end. I could still read the version number off the form, so I was still seeing the code. The prisoner is the one who has stopped, for whom the ticket is the customer and the count is the work, and who has to go inside the machine to find any meaning at all. Cypher sells out his crew to be put back inside, and he sets his price knowing exactly what the steak is made of. I spent fifteen years building the machines this essay is about, and I am a sociologist by training, which means I was watching what they did to people while I sold them. Enterprise software has never had users, only prisoners, and prisoners learn the routine.2 The people who build it are prisoners of a different kind, of a method rather than a screen.
The answer to what was happening to Cypher and to me was never going to come from the psychology of everyday machines. It comes from the psychology of the world, the system of people the machine sits inside, and the two men who understood that system best were German sociologists who never wrote a line about software. Habermas watched money and bureaucracy do to families and towns what forms do to work, and called it the colonisation of the lifeworld, which is a heavy phrase for a simple thing: the systems we build to run our lives move into the places where we make sense of our lives, and replace the sense with procedure. Luhmann was colder and more useful, and he took his idea from two Chilean biologists, Maturana and Varela, who had worked out that a cell stays alive by making its own parts and its own wall. A company, Luhmann said, does the same with sentences. It is made not of people but of communications, and it can only do what it can say. Whatever cannot be said in the company’s own language does not exist for the company.3 Add the dose, and a company is exactly as intelligent as the categories of its software. It has no way of finding this out, because that thought is not one of the categories.
A customer has a problem. The company has a ticket. These are related things, but watch what happens when only the ticket can move. The customer learns to describe the problem in a form the software will accept, the employee picks a category she knows is wrong because the right one does not exist, the manager counts closed tickets because closed tickets are what the system can count. At no point did anyone decide that helping the customer had stopped being the job. The substitution was available, countable and easy to repeat, and repeat it did, a thousand times a day. The category is fine, the ticket is fine, the dashboard is fine, and then one day the customer is gone and everyone is surprised. A receipt is proof that something ran. An outcome is proof that it mattered. The ticket system produces receipts, and the whole building has learned to call them outcomes and put them in a deck.
This is the inversion of meaning, and it works by mislabelling. Work is a group of people organised around an outcome. A ticket is a group of fields organised around a count. We built the second to represent the first, and every time an action is encoded in software its intent and its reasoning are left outside the code.4 Over enough repetitions the representation becomes the work, and the work becomes the thing that has to justify itself to the representation. The customer must explain how her problem fits the ticket’s form; the ticket is never asked to explain why it has no room for her problem. No better design removes this, because it is not a property of any one machine; it is what happens to an environment at that dose of repetition. Environments do not serve purposes, they select for what survives in them, and what survives in a ticket system is tickets, and people who are good at tickets.
There are three kinds of thing we deal with all day, and the whole story is in the difference between them. Things imprint: the door works on the hand, a gesture, a jerk of the wrist, the force you learn to use, and Adorno had seen that much by 1951. Machines add scale and frequency: software works on the categories, on what can be named, what can be counted and what counts as done, and it gets there through the dose, a thousand corrections a day until the schema is in you and, at the far end, you are Cypher. The alien mind is the third kind, and it does something neither of the others could do, which is to speak back. A model does not only ask you to fit its form; it takes part in deciding what the form means, and that is a new kind of imprint, and it can go two ways. Used badly it finishes the job the ticket started, and Habermas had a word for that too. He called it strategic action, which is the polite name for treating a person like a lever, and when you hide that you are doing it the word is manipulation. Used well it speaks back the way a good colleague does and shows you a route you did not know existed, which is exactly what a machine did in a game of Go nine years ago.
2The schizophrenia of practice
A method is rarely just a method to the person who has mastered it. It is twenty years of mistakes that stopped happening, shortcuts that turned out to be right, hard calls that were vindicated and a reputation built on all three. The route is where the knowledge went to live. So when someone arrives and says the route was optional, the body hears something else: the years were optional.
I have been on both sides of this. I have defended a way of doing things because it was mine and I was good at it. I have also walked into a room with something finished. The second is worse, and nobody warns you. When a model can produce a complete proposal in an afternoon, the proposal arrives before the conversation we used to have to get there. Its assumptions have already set. The people in the room have only two moves left, endorse it or resist it, because the move they used to have, helping to decide what the problem was, was made for them before they got into the room. I called what I met “resistance to progress”, and they experienced it as the removal of their part in it; both descriptions were accurate, which is the whole problem. If you have said the word “adoption” in a meeting this year, that was you, and it was me.
This is the schizophrenia of practice and it is not a personality defect. The practitioner is the only one who still knows the reason for the method, so to keep that knowledge she has to betray the method that carried it, which is precisely the thing everyone can see her being good at. How deep the split runs depends on what her mastery was made of. Nobody becomes the best salesperson by mastering one method, and nobody becomes the best marketer or designer that way either; their mastery is of first principles, of what a buyer fears, what an audience responds to, what makes a page hold together, and a new method is just one more for the collection. Software engineering is the other case and the extreme one, because there mastery means living inside the method completely. Take the method away from a salesperson and you have a salesperson without a script. Take it away from a software engineer and you have to ask who is left. The answer is the same as in any true mastery. The masters of software engineering from here on will be the ones who can let go, who become builders of meaning rather than workflows, and who move between the method and the intent of their practice without noticing the border.
And the method now moves; a model can read the same request differently in the next sentence. You cannot master a method that moves, and this is where the fear is right and wrong at the same time: right about the method, because the alien mind kills routes and will keep killing them faster than anyone can learn the replacements, and wrong about the knowledge, because a method is knowledge compressed with the reason thrown away, and the reason is the part that survives a change of route. Mastery has to move, from knowing the route to knowing what the route was for in the first place. Method is not knowledge. Say it until it stops sounding like consolation and starts sounding like a design requirement.
There is a trap on the other side too. The escape from a machine-imposed method cannot be another arrangement in which people have no meaningful choice. If AI removes the part where people decided what the problem was, it has not freed anyone. It has industrialised endorsement, which is manipulation in Habermas’s exact sense: people treated as the lever a decision is pulled with, and it will feel, to the person holding the model, exactly like progress.
3Software engineering, prisoner of its method
Craft, product and scale are usually told as a story of progress, and they are not one; they are three answers to one question: how much knowledge can a single carrier, a person or a system, hold against the demand for it?
A cabinet-maker held the whole of the work in his hands and his head. When demand outgrew the hands, nobody asked him to work faster. Instead the factory moved the knowledge out of him and into jigs, tolerances, inspection and the sequence of the line, so that a person with a fraction of his skill could make something that fitted. That was both the genius of it and the cruelty of it, and it worked because a jig can hold a tolerance.
Software did the same trick to half of its problem, and the half it chose explains the culture that followed. Once a program exists, copying it costs nothing, and around that fact the field built the most advanced production system there has ever been: compilers, tests, pipelines, deployment to the whole planet before lunch. But the change of form, from physical to digital, arrived without the change of production that should have come with it. The bridge from an intent, a design, a product to the code that realises it never became manufacturing. Instead it stayed craft, one understanding at a time. What it could not move out of people’s heads was the theory of the thing: why these parts exist and what breaks when they move. Peter Naur said this in 1985 and nobody has refuted him.5 A repository can hold the source and it could hold the reason too. Because every tool we have built rewards the source and ignores the reason, almost nobody writes the reason down, and every large system stays hostage to the three people who understand it, two of whom are interviewing elsewhere.
So software engineering did what a practice does when no single system can carry the fit: it became a method, and then the most advanced and most scientific method any craft has ever had. Decomposition, interfaces, review, tests, continuous integration: a way of constraining a hostile medium so that fallible people could work in it together. It is one of the great achievements of the century, which I say without irony. With it come ceremonies, including a daily one in which people who sat next to each other yesterday tell each other what they did. Then, so slowly that the profession never saw it, the method became the practice and the practitioner became a prisoner of the method. This is where the schizophrenia runs deepest, because in a field this rigorous mastery means living inside the method completely, and from inside it every other practice looks loose: marketing, design, sales and product all appear less rigid, less structured and less advanced, rather than simply less method-bound.6 Ask a software engineer what should exist and the answer arrives already translated into the kinds of thing the method knows how to build: a screen, a service, a workflow and a ticket for each of them. (I have drawn that diagram myself, more than once.)
Tell an engineer that a design contains a game, in the plain sense of the word, an intent, a goal, the moves you may make, what each costs and how you lose, and a good one will answer, helpfully, that if I mean tests, continuous integration or a gate, those already exist. That engineer is Cypher at the monitors: not wrong about what he sees, and no longer able to see anything else. A test, a pipeline and a gate are three games with their intent and their loss conditions hidden, three arrangements of one general activity, which is to make a bounded attempt, see what happens and decide what to keep. The game is not a nicer name for the method, or a variant of it. It is the level underneath the method, the level the method was built on and then buried. A proposal that starts there looks to the engineer like skipping the real work, when it is instead questioning the choice that made the work necessary. Anything we now build for the alien mind inherits that choice, and the industry’s instinct runs the wrong way. If we want better software, software that does not imprison us, we have to give the alien mind the concepts the abstractions were made from, the intent, the goal, the moves and the losses, and let it find its own routes through them, rather than handing it our abstractions and asking it to work inside them. The abstractions were made from the concepts, and design for the alien mind has to run in the same direction.
I am not the first to say this. It is the curious case of software engineering, and it is curious because the field has always contained the arguments against it. Dijkstra warned that the tools shape the thinker. Naur warned that the program is not the understanding. Parnas told us to separate the decisions that change from the sequence that runs them.7 Computer science has spent fifty years proving that a program is not an understanding while at the same time building tools that treat the program as the understanding. Software engineering is the one profession that mistook its scaffolding for its building, and then rented the scaffolding out to everyone else.
That would be an internal matter if every other practice did not now live inside the artefacts software engineering makes and get measured against its idea of rigour. The buyer’s practice runs inside a purchasing workflow designed by someone who has never bought anything. The nurse’s judgement is exercised through a form designed by someone who has never seen a ward, and when the form is wrong it is the nurse who gets audited. The thousand-to-one ratio is the delivery mechanism: one profession’s category error, administered to everyone else in small doses, all day, for thirty years.
And the method cannot see the thing that is about to make it optional. A build system sees whether the steps ran. It cannot see whether the route was good, because the route was fixed before the build began. Ask a pipeline whether the tests are testing the right thing and it will show you a green tick, which is the most successful product in the history of software, because it is the only one that has never disappointed anyone. The agent harnesses now being bolted onto every enterprise product are the same object in a new costume, with roles, steps and gates and a green tick at the end. It is a robot in the driver’s seat of a self-driving car, holding the wheel, under a sign that says keep your hands on the wheel at all times,8 and it is the sign taped over the handle: we have taken the first machine that can speak back, given it a badge that says clerk, and now we complain that our clerk is acting strangely. They will do to judgement what the pipeline did to design, unless the thing being harnessed is allowed to change the question.
4Take the handle off
In March 2016, in the second game against Lee Sedol, AlphaGo played a move on the fifth line that every professional watching read as a mistake. No strong human would have played it; the commentators said so on air; the system itself put the odds of a human choosing it at about one in ten thousand. It won the game, and then the match. The move has a name now, Move 37, and it is the first thing most people know about what Jakub Pachocki, OpenAI’s chief scientist, has since called the alien mind: an intelligence that was grown rather than designed, whose workings its makers cannot fully describe, and which was competent, that afternoon, in a way that expert judgement could not recognise as competence.9
AlphaGo had learned from human games. The version that came eighteen months later, AlphaGo Zero, learned from none. It was given the rules, a way of knowing whether it had won, and itself to play against. It had no opening book, no professional repertoire and no method. After three days it played the version that had beaten Lee Sedol, and the score was one hundred games to nil.10
Is method knowledge? Nobody has run a cleaner experiment on the question. Take the best method humanity ever assembled for a hard problem, twenty-five centuries of it, and give it to one system. Give the other system only the game, the intent, the goal, the moves, their cost and how you lose, and the second one wins a hundred games to nil. Zero was also a better machine, with a redesigned network and search, so the method was not the only thing that changed; it was the difference that could not be argued away, because it was the thing Zero was not given. Zero had the richer space, and it had it precisely because it had less: no repertoire of moves already found good, nothing that was to the game what a pipeline is to software. What its makers withheld was the requirement that good play first look like human play. In other words, they took the handle off.
Then it happened to the humans. Within a year professionals were opening with moves that had been taught as mistakes for centuries; Ke Jie, then the strongest player alive, said that humanity had been wrong about the game for thousands of years; the opening theory of Go was rewritten by people who had watched a machine play it.11 That was an imprint as well, the first from a machine that taught instead of trained. Instead of replacing them it spoke back, and it could only teach them because nobody had made it pretend to be one of them. That is the design principle: mislabel the alien and you get a worse human; let it be alien and you get a collaborator.
The players had two responses. Some retired, some wanted the machine kept away from the board, but the best of them studied its games and got stronger. Software engineering is having the same two responses now, and the first is winning. The alien mind did not come out of the field’s method; it came out of statistics and a borrowed neuron, and it arrived as an outsider, so the field’s instinct has been to wrap it in the method it already has: agent frameworks, typed tool calls, pipelines, gates. That is AlphaGo built again after Zero has been demonstrated, a machine trained to resemble the human repertoire, worse to use and capped in what it can find, and it is being sold as the responsible option. The other response is harder, because it questions the method at the heart of the field, and if you have spent a career inside the dogma of AlphaGo the honest options are the players’ options: resist, or study the games and rebuild your foundations.
Software engineers have a direction for this kind of stripping-away, which they call going down a level, and it runs toward the machine: from the application language to instructions, memory and the processor. It is a real axis but not the only one, and the profession’s mistake was to name it the only one. Ask a designer to go back to first principles and she moves toward attention and comprehension; ask a negotiator and he moves toward interests and obligations; ask a buyer and she moves toward the purchase. Those are the reasons a workflow exists, and in every practice except software engineering, down points toward purpose. Ask what sits underneath a pipeline and the honest answer is the game it was built to play. We have spent thirty years calling the processor low and the purpose high, and calling a historical accident a law of nature.
What Zero shows is that an unfamiliar intelligence can work at the practice’s level directly. It does not need the purpose reduced to a sequence first; it needs the purpose, the rules and a way of knowing. Practices that have kept hold of their first principles, that can still say what a good outcome is without pointing at a screen, get a structural gift from this, and the gift is what Taleb calls antifragility: they get better when kicked.12 Vary a workflow and it breaks. Vary the distinctions a practice uses to decide, and the good variations improve the distinctions. A practice that lives at that level treats a new method as an experiment, which is why a marketer, a designer or a salesperson can take the alien mind as one more method to master. A practice that lives inside its method experiences the same variation as damage, which is why software engineering, with the most to gain, has the highest switching cost of all. Its languages are its dogma, and the alien mind works underneath them.
The people building the models have a specific blind spot, which is that they put evolution at build time: train, evaluate, ship, and the intelligence is delivered, with everything after called inference. But the thing that made Zero better was not the architecture but the play. Evolution happens at runtime, in use, in the world the system acts in. But if you look for the enterprise version of that world, the room in which a model would have to act, nobody is designing it. They are designing the organism, and they have neuroscience for the head and nothing at all for the room.
Go had a board and one objective, while an enterprise has neither: procurement’s saving is operations’ risk, and the forecast that sales is proud of is, to finance, an unsupported commitment. There is no board on which all of that has been settled, and there is nobody to hand the model the rules, because the rules are exactly what the participants disagree about. Zero tells us how to search a world that has already been designed, and nothing about how to design one in the first place, which is the question the field has decided not to ask.
5A bigger brain for a problem between brains
Let me take it one step further. Demis Hassabis, Yann LeCun, Jakub Pachocki and Richard Sutton do not agree with each other about much, and they agree about this by assumption rather than argument: the unit of intelligence is the individual, so make the individual bigger. I am going to disagree with all four, and I am going to do it from my field rather than theirs, the sociology of systems and fifteen years of watching what machines do to organisations, which is where their designs land and the one place none of them has stood.
Demis Hassabis has the most beautiful version. Models propose, evaluators test, an evolutionary loop keeps the promising candidates; AlphaEvolve made it real for code and mathematics, and the mixture-of-experts architectures inside many of today’s largest models make the same bet one level down, a population of specialists behind a learned gate.13 Any mind that works, works by ignoring most things,14 and a mixture of experts is that economy built into the weights. The direction is right, yet for the thing that matters here it is also wildly inefficient. It rebuilds the division of labour of a society inside one skull with a router where the language should be: a company with a perfect org chart and no meetings. Hassabis is honest about the gap. In his conversation with Lex Fridman last summer he singled out scientific taste, the judgement that picks a question worth asking, as the thing naive search does not supply.15 Where does he expect to find it? He expects to find it in a more capable model, and that is wrong, in a way software engineers are peculiarly unable to see. Taste does not live in the individual; it lives in the seminar, the rival lab, the reviewer who rejected the paper, the instrument that made a new question askable, the dead teacher whose objection you can still hear. Science is a social evolutionary system and has been since the Royal Society. The genius is its output, not its engine. Build a bigger individual and you will get a better output and call it the engine, and you will keep wondering why the taste never arrives.
Why is this so hard to see from inside the field? Because the field inherited the blindness twice and then scaled it. Everything the labs build stands on Shannon, who defined information as the reduction of uncertainty and, on the first page, set meaning aside as irrelevant to the engineering problem; three years later he measured English by having people guess the next letter, and a language model is that parlour game with a trillion parameters. It inherits his exclusion of meaning the way a child inherits a surname.16 Then the field borrowed its architecture from the neuron and its object from neuroscience, and with the object came neuroscience’s oldest mistake: looking for the mind in the organ, the way phrenology looked for character in the bumps of the skull, with better funding. So every lab hit the same wall, and anyone watching could see it. Scaling the network gave fluent completion and stalled short of the higher functions, reasoning, judgement, taste, at exactly the point where neuroscience stalls in explaining the brain; and the way through, in every lab, was to build an environment around the network, with rewards, verifiers, opponents and people, and to call it training rather than design. The higher functions arrived when the network was given a world. It is in every one of their papers, under Methods, which is the section nobody reads.
The conclusion is that the clever part of the brain is the simpler part. The cortex that does our abstract work is wallpaper, one small pattern repeated, and we have more of it than a chimpanzee without being otherwise very different; put a toddler and a chimp in front of a physical puzzle and they do about equally well, put them in front of another person and only one of them starts learning.17 We can eat, fear, mate and cry alone, but we cannot innovate alone, and the brain’s answer to that problem was not to compute more but to compute less, and to leave the computation in the world: in language, in the argument between people who know different things, in the institutions that remember what no one person can. The numbers surprised me. Speech, in every language anyone has measured, carries about thirty-nine bits a second, a rate a modem would be ashamed of, and it is ambiguous by design, because ambiguity is cheap when the listener’s world will do the disambiguating.18 Our protocols beat us by seven orders of magnitude at moving bits but lose to us at moving meaning, and the reason is that the brain does not encode meaning at all. It points at a shared world that already holds it; the compression is semantics. That is what Luhmann took from the biologists: a cell stays alive by producing its own wall, and a society stays intelligent the same way, by a form he called “meaning”, which leaves almost all of the world uncomputed while keeping it within reach.19 Put a boundary around what a group of people mean by their words and you get a lab, a trade, a company; take it away and you get a crowd. The field cannot see the boundary because it is looking at the organ, doing the psychology of the organ when the answer, as it was for Cypher and for me, is in the psychology of the world. None of this is one field’s finding. Norman’s door, Habermas’s lifeworld, Luhmann’s cells, Shannon’s parlour game and Zero’s board each hold a piece, and none of those fields talks to the others, which is the whole argument in miniature. The synthesis is mine, and I am not going to apologise for it.
Yann LeCun is the purest case. He is the most serious critic of language models inside the field; he does not think they will get us to human-level intelligence; and his alternative is the world model, a system that learns how the world works by watching it, predicts what will happen, plans against the prediction, and in his fuller architecture makes room for memory, objectives and the influence of other people.20 It is serious work and it is a neuroscience answer to a sociological question. LeCun is a man standing inside a forest, explaining that to understand forests we must grow one enormous tree that contains all of them. The world is not a brain but a thing in which every part depends on the others and no part contains the rest, and its efficiency comes from exactly that. Marx lost patience with the philosophers of his day for the same reason in 1845: they sat in their studies asking how thought could ever reach the world while standing in it up to their knees, and his line for it was that what we are is not an abstraction inside each of us but the whole set of relations between us.21 The labs are doing the same thing with better hardware. We are in the world, in the grit and the mud of it, and so is the model; you do not get down to it, because you are already in it.
Every enterprise now has a diagram with boxes labelled planner, researcher, critic and executor passing messages to one another, and it is presented as the arrival of organisation in AI, when it is an org chart without a world, which is where most companies end up anyway; enterprise AI just got there faster. The roles reproduce the same old method; the shared understanding is scattered across prompts; and when three agents agree that an account is profitable it is because all three inherited the same incomplete account of its costs. Agreement scales omission. The word people reach for when a model fits its context unusually well is resonance, and it is a good word that explains nothing. What we need is not resonance but interaction, the thing language does, where a few bits passed between people who share a bounded world do the work of a great deal of computation. That needs what Schelling called focal points, the reason two strangers told to meet in New York with no further instruction both turn up at Grand Central at noon.22 Without them, more agents will produce more consensus about nothing. My bet is that one general model interacting inside a curated world will beat a crowd of them talking to each other without one, and that it will be cheaper. Nobody in the labs will run that bet, for the simple reason that it is not as sophisticated as the grand design of a large language model. The Chinese challengers in AI have already shown how that story ends: the answer turned out to be engineering. It is the same here, a new kind of engineering, a new architecture and a new process for how we manufacture, boring maybe, but with results that are easy to judge and need no eval. I intend to run it.
Pachocki’s essay is the most serious statement of the individual view, and he is candid about being frightened. His systems are grown rather than designed, and they are approaching the point of improving themselves. He does not fully understand them, and he expects some agents to pursue their own objectives and to cross the scope of what their operators intended. His answer is to work on the mind: teach it values, hope the values survive contact with situations nobody trained it for, and read its reasoning to check. He talks about teaching machines to love, and he is not joking.23 He knows the environment matters and wants people kept in the loop, but the environment appears in his account as the thing the mind’s values must survive, and never as the thing to design, even though his own systems acquired their higher functions inside environments the labs built, with rewards, opponents and human feedback; the world was there all along, and it was called training. This is the category error again, made by the person with the most at stake. Nothing inside Zero made it safe to leave alone; the board did: rules that did not move, a goal that could not be argued with, a way of knowing when it had won, and nothing it could touch outside the game. Every danger he lists is an alien mind without a world, an agent with no pinned rules, no authority it was lent, no conflict that gets raised, no record that can be replayed, so that the only lever left is the mind’s own disposition and the only safeguard is reading its diary. He is asking the mind to carry what the world should carry. On caution he is right, and the disagreement is about the unit rather than the pace. By his own account the mind is grown and cannot be designed. The world can, and that is where the design effort belongs, and it is the one place the field is not putting it.
Hayek made the point about prices in 1945: the knowledge that runs an economy is not in anyone’s head, and no amount of enlarging a head will put it there.24 That is the unit, not the mind but the world between minds, with its persistent references, its revisable terms, its useful disagreements and its commitments people can inspect. This is why Richard Sutton’s bitter lesson is useful here, even though I disagree with him about what it teaches. General methods that use more computation win, and he is right; but the most general method that has ever used computation is the one that has run between human minds for fifty thousand years, a search with selection, memory and stakes, and it produced every sentence the models were trained on.25 We scaled the network and starved the world, and that is the real bitter lesson: we were looking in the wrong place. The alien mind is alien because we never saw the brain and its language as one system, so we built half of one and called the result strange. A model can understand a promise but has no authority to make one, because authority is a property of the world and not of the skull; it can be lent to a model, in scope, by someone who holds it, it can never be assumed, and a model’s confidence is not a signature. We should be designing how intelligence develops between participants at least as seriously as we design what happens inside each one, and at the moment the ratio of effort is roughly a thousand to one, and given where we are, that is the wrong way round.
Set against those four, my position is the mirror image, and it holds at the same altitude. The unit of intelligence is the world between minds, not the mind. Intelligence is manufactured there, in a bounded space of shared meaning with rules, evidence, authority and stakes, and it always has been; the individual is where it gets used, not where it gets made. A model is a participant in that space, a capable one, and never its owner. And the direction of the field is not a bigger organism but a designed ecology: the minds the labs grow will converge and become interchangeable, and the worlds built around them will be what differs, competes and compounds. And that is where we are going, with or without the labs, because it is where the value has always been.
6A world, not a model
So build the world. That is an engineering instruction rather than a slogan, and it is what I have been building.
The pattern is Zero’s pattern moved into manufacturing, and it has four steps.
Distill: take a situation and reduce it to the distinctions that actually decide it. Which purchase, which commitment, what has been observed, what is still an assumption, what would make an alternative better. Distillation is not summary; its job is to lower the burden of context without deleting the one fact that would reverse the conclusion.
Construct: state the proposal explicitly, as a thing with a purpose, a result that would count, the objects and obligations involved and a way of knowing whether it was achieved. That is the board, and in the plain sense the game, intent, goal, moves, cost and loss, and it is the lowest level of the design, with everything the engineer calls “the real work” sitting on top of it. Then let the participants, human and machine, attempt to find routes across it; test the routes against the stated conditions; keep the useful differences between candidates long enough to learn from them.
Then choose the form, and only now decide how the work should appear and run: a review might need a table, a conversation or a full record, and those are renderings of one construct that carry the same promise. Then either realise it, or, if the attempt reveals that the promise was unclear, go back to first principles and fix the promise, because a prettier screen cannot repair an undefined obligation.
The world holds the rules, the goals, the evidence and the disagreements. The workflow was the handle with the sign on it; the board is the flat plate, and it tells the alien mind exactly what the work wants from it and nothing about how it should look while doing it. The model is a participant in that world, a very capable one, with no more authority over what the work means than any other participant. This is the inversion of the inversion: the representation stops being the authority on the work, because the work now has an architecture of its own that can outlive any particular representation. A city outlives every building in it, because a city was always more than a collection of buildings. A company whose meaning lives in its applications dies with its applications.26
People hear graph and think ontology, a scientific claim about what exists, and the graph here is nothing of the kind. It is a manufacturing artefact, the jig and the tolerance for knowledge. A saving belongs to a baseline. A baseline applies to a period. A proposal depends on a supplier commitment. A reviewer has authority over one decision and not every decision connected to it. Connect those and a participant can be handed the part of the situation the work needs and nothing else. The graph is not a theory of the company but a narrower and harder promise: that every number an action rests on can be followed back to what it rests on. Its job is to let a human, a model and a system fit their work together without rediscovering everything each time, which is what a factory does with parts, what Toyota’s kaizen did with process, and what nobody has yet done with meaning. It is a factory for meaning, built to use what Zero showed, and it makes no claim to be science.
This is the bridge software engineering never built, from intent, design and product to running software, and it is the problem Cadence is built for, which is a different problem from the one programming languages solve. Programming languages were built so that data and code could grow without collapsing: more records, more services, more states, held together by types and interfaces, all of it to protect the scarce thing of the time, which was engineering hours. Nobody built one so that meaning could grow, and meaning is what runs out first now, because the scarce thing now is the model’s attention: tokens, context, what it costs to carry a distinction and what it costs to forget one. The design questions change with the scarcity. Not “what is the schema” but “what does it cost to keep this distinction in play, and what does it cost to drop it”. The amount, the applicable authority and the recorded effect stay exact; a system that gets those wrong has lost the game. The explanation, the route of inquiry and the arrangement that helps a particular person judge are free, within what the craft owes: the baseline stays visible, the alternative stays on the table, the consequence stays where a person can inspect it. A system that fixes those has confused a stable answer with a settled question. An unknown is never rendered as zero. Determinism earns its place where an answer must not move, and it is not spent settling questions the participants have not yet understood.
At runtime this becomes one concrete choice, and it is the opposite of the choice every enterprise application has made since the first one. By my own estimate more than half the code in a typical enterprise application is interface,27 which means we have been spending most of the money on the least meaningful part of the system, which is the part with the buttons. The meaning lives in the construct, not in the app. A finance reviewer looks at a table that makes the baseline and the assumptions explicit. An operations lead looks at the dependencies a change would create. A buyer negotiates the next move, and all three are looking at one proposal, with one identity and one history. Change the view and the saving being claimed does not change, because the saving was never in the view. Screens, actions and conversations become ways of realising work whose commitments stay inspectable somewhere the screen cannot edit. The app and the meaning stop being two systems with an integration between them. The product is a rendering of the semantics, which is what software shaped around the problem looks like, instead of the problem forced into the shape of a ticket or a form.28
It also changes what you do when people disagree, which in an enterprise is always. Software has a name for the thing that makes two records agree, a reconciler, and the more tightly the records are linked the worse its life gets: fixing one thing breaks a conclusion three tables away, checking everything against everything becomes unaffordable, and some sets of demands have no answer at all.29 The tempting response is a bigger graph and a stronger reconciler. Sometimes that is right, and more often it is an expensive way of avoiding the discovery that two people want different things. Human organisations act without resolving their whole world: they qualify a promise, narrow its scope, defer a question, try something reversible, agree who decides,30 and the system should do the same. Resolve what the present scope can resolve, surface the conflicts that matter, narrow the question and carry forward every dependency that could change the decision, and then ask, negotiate, come back with a different objective, or stop. A failed reconciliation is not consensus, and it must never be laundered into consensus by a persuasive paragraph. The nondeterminism this allows is not a defect; unreliable arithmetic is a defect, and the ability to reconsider whether the number answers the right question is the capability we wanted from the start.
Four rules make this buildable, and none of them needs a new word. The route is free: different people, or different models, may take different paths across the same board, and different paths produce different runs. The record is exact: the same choices, replayed, produce the same trace, so an attempt can be audited without being re-argued. The conflict is loud: when two automatic processes disagree, the system raises the disagreement instead of quietly picking a winner. And the rules change only in the open: a route may adapt during an attempt, but a rule, a baseline or a win condition changes only through a proposal that someone owns, with a version, between attempts and never in the middle of one. The board does not move during the game. When an attempt fails because the promise was wrong, the failure goes up to whoever owns the promise, and they change it where everyone can see. That is how a category stays challengeable: its failures have an owner.
The first thing we are building on this is called Value Hunter.31 It hunts savings across a company’s suppliers: initiatives, baselines, forecasts, what was actually realised, who gets the credit. A workflow executes; it does what it was told in the order it was told. It is exactly as intelligent as the person who drew it on the day they drew it. A hunter has a purpose and a world and finds a route, and when the route is unfamiliar the world is still there to say whether the purpose was served. “It mattered” has four levels, and the hunt is not over at the first: the saving was executed, which is the receipt; the reviewer accepted it; it showed up in the numbers; and it was still there a year later. A ticket system stops at the first, most dashboards stop at the second, and a hunter is judged on the last two. The model inside it is a co-actor, a collaborator that can adapt what a person sees but holds no authority of its own; what it may do has been lent to it, in scope, and the consequence of every action that changes the world is shown before the action, not after. Judge the whole thing by one number: the total cost of reaching a useful, defensible result, counting computation, context, delay, human attention and correction. Neither code volume nor the size of the graph is that number. Semantic coherence, the property that the work still means what it meant when the view changed, matters more than determinism everywhere except where determinism is the point.
The best evidence I have that this works at the level that matters is Pippa.32 She has spent her career in marketing, and she looked at a body of campaign and design work that had been produced this way, and what she recognised was not a set of generated outputs but an agency. Audience, emphasis, language and form held together the way they hold together when people who know what they are doing have done it. She recognised the coherence before she needed any account of the machinery, which means her expertise had somewhere to operate. It is no accident that she, Mikkel, who is our designer, and Don, who runs our sales, have had the easiest time of anyone with this transition, and it is not because their practices are weaker. Their mastery was never of a method, so the alien mind arrives for them as one more method to master, which is what mastery has always meant in their fields; the people with the hardest time are the software engineers, and that is the cost of their rigour, not a failing of it. The test is not a benchmark: whether a practitioner who can say what the work must accomplish can judge several credible ways of getting there. If she can, her contribution becomes more consequential, not less; if she can only approve a machine’s chosen process, we have built the ticket system again with a better vocabulary.
7Freedom from method
The winners are not going to be the companies with the biggest models, because everyone will have the same models and the same slide about them; the winners will be the practices closest to the problem, not the ones closest to the technology:33 the practices that can get back to their own first principles fastest, the ones that can still say what a good outcome is without pointing at a screen, and can therefore let the method change without losing the knowledge. And among companies, the ones that can operate as a whole entity on those principles, with purpose, evidence, obligation and judgement in one architecture, and rebuild their methods from there, as tools, with the tools’ names on them.
This gives computer science a larger job than the one it has been doing. For forty years it has been mistaken, by others and by itself, for the defence of “proper” programming methods. It can be the science of how different kinds of intelligence work on a problem together, human, statistical and mechanical, with programming methods among its instruments. That is who a software engineer is once the method is gone: the person computer science was meant to produce all along, who can state what a system must hold true and let an alien mind find the route. The split between the science and the method produced a fallacy that has cost more than any bug: the belief that fluency in the method is possession of the knowledge.
The economics point the same way. For twenty years software was expensive enough that administration could pass for judgement. Saying no was usually right, because saying yes cost a team and a quarter, and a whole profession learned to ask a reasonable question: is it worth it? When the cost of trying collapses, that question starts lying to you, and the thing that used to be cheap, knowing what is worth trying, becomes the thing that is scarce.34 Method is getting cheaper and knowledge is getting more expensive.
The result is better software, and that is a technical claim rather than a hope. The customer with a complicated problem does not have to become a simpler customer to fit a ticket. The employee does not have to disguise a sound judgement as an approved category. The system helps them make the distinction that matters and carries it through to action, with interfaces, records and exact execution where exactness is owed, because none of this is an argument against software engineering. It is an argument against software engineering’s claim to be the whole of the practice.
This has never been an argument against machines either. The argument is about psychology, ours, the machine’s and the world’s, and it comes down to one act of design that Norman would recognise: take the handle off. We did it once with a door and got a door that tells the truth, and once with a game and got a collaborator that taught its makers the game. The enterprise is the next door, and the handle is the method.
The alien mind is unsettling because it speaks back, and because what it says can make a familiar route look unnecessary. The people who made it are unsettled for a larger reason: they have grown something they cannot fully describe and they have nowhere honest to put it. The danger was never the alien mind; it is the alien mind without a world, and a world is the one thing we know how to design. What is left, once the world exists, is the end of method’s claim to be knowledge, which for anyone who has ever been made to feel stupid by a form is a liberation. Holding a purpose steady while the way you say it changes is something people have always done between them; it is how two departments can share a purchase commitment without sharing a theory of the company’s future. It can now be done in software, which is the whole opportunity, and it is larger than the models.
A thousand times a day, for thirty years, we pushed when the handle told us to pull. We learned the forms, the categories and the sequences. We became remarkably good at accommodating the machine.
It is time to stop mistaking that accommodation for the work.