From the neuron to the agent: what we learned when we tried to speak to machines

Every technique for working with a language model is a scar. Somebody tried something, it failed in a specific and instructive way, and the next technique is the record of that failure. If you want to understand where we are now, with agents and harnesses and context engineering, you have to walk the scars in order. Skip one and the rest look like magic, and magic is a bad thing to build a company on.

So start at the bottom.

The neuron, and the problem of knowing anything at all

A neuron in a neural network is not mysterious. It takes a handful of numbers, multiplies each by a weight, adds them up, and decides whether the total is big enough to pass along. That is the whole thing. The word "neuron" flatters it. It is closer to a dial on a mixing board, and a modern language model is a mixing board with a few hundred billion dials.

Nobody sets those dials by hand. You show the board a sentence with the last word hidden, ask it to guess, measure how wrong it was, and nudge every dial a fraction of a hair in the direction that would have made it less wrong. Then you do that again, a few trillion times, with a large slice of everything humans have written. The dials settle into positions that make the guesses good.

Notice what that means, because the whole story turns on it. The model does not contain facts the way a database contains rows. It contains dial settings that make certain continuations likely. Nobody wrote a rule that says Paris is the capital of France. The dials ended up in a position where "the capital of France is" tends to be followed by "Paris," because that was true across enough text. It is knowledge in the sense that a groove worn into a path is knowledge. The path knows where people walk.

Attention: nothing means anything alone

The early versions of these networks read text the way a nervous person reads a contract, one word at a time, trying to hold everything previous in mind. By the end of a long sentence the beginning had faded. Ask about the subject of a paragraph and the model would answer about whatever it had seen most recently.

The transformer, published in 2017, dropped that approach. Instead of reading in sequence, every word gets to look at every other word in the passage at the same time and ask one question: how much do you matter to me right now? The word "bank" looks at "river" and "money" and weights one of them heavily. That mechanism is called attention, and the name is unusually honest.

There is something almost philosophical in it, and I do not think that is an accident. A word means nothing by itself. It means something only in relation to everything around it. Old poets knew this. The transformer made it arithmetic.

What the transformer did not change is the task. The task is still, and only, predict the next token. Every capability you have seen since, the essays and the working code, is what that one task looks like when it is done extraordinarily well over an enormous amount of text. The model does not know. It predicts. Hold on to that distinction, because every technique from here on is an attempt to get something reliable out of a thing that only ever guesses.

Prompting: the first time we asked properly

Once the models were good enough to be useful, the first failure showed up immediately. The model could complete text beautifully and still not do what you wanted. You asked for a summary and got a continuation. You asked a question and got three more questions in the same style.

Prompting was the answer, and every version of it is the same move in a different costume. You arrange the text in front of the model so that the most probable continuation is the useful one.

A zero-shot prompt just states the task and trusts the model to have seen enough similar tasks. A few-shot prompt puts two or three worked examples above your request, so the pattern is sitting right there and the model has to try hard not to follow it. Role instructions ("you are a senior tax accountant") shift which part of the groove the model settles into. And chain of thought, the most interesting of them, simply asks the model to write out its reasoning before it answers. That works because a model that has written "first I need to check the dates" has changed its own context, and the next prediction is made from a better place.

A small demonstration, because this is where people either get it or do not. Ask a model "Is this contract clause risky?" and paste the clause. You will get a plausible paragraph. Now write: "You are reviewing a supplier contract for a small manufacturing company. Read the clause below. List each obligation it creates for the buyer, then say which of those would be unusual in a standard supply agreement and why. Point to the exact words that create the risk." Same model, same clause. The second answer is worth reading and the first is not, and the difference is entirely in how you set the table.

Prompting taught us something uncomfortable: the quality of the answer is mostly a property of the question. That is true of people too, which is why nobody likes hearing it.

Retrieval: the model does not know, so you must tell it

The second failure was worse than the first because it was invisible. A model trained on text up to a certain date knows nothing after it, and it does not know that it does not know. Ask it about last quarter's results and it produces a confident, well-formed answer with numbers in it. The numbers are inventions. People call this hallucination, and I dislike the term because it suggests a malfunction. It is the system doing exactly what it was built to do, generate plausible text, in a situation where plausible and true have come apart.

The fix is retrieval. Before you ask the model anything, go and find the relevant documents yourself, and put them in front of it along with the question. Now the most probable continuation is grounded in something real. The technique is called RAG, retrieval-augmented generation, and it is the reason a model can answer questions about your company's internal policies without ever having been trained on them.

The mechanism underneath deserves one paragraph. To find "relevant" documents you need a way to compare meaning rather than matching words. Embeddings do that. An embedding turns a piece of text into a long list of numbers, and texts that mean similar things end up with similar lists. Think of it as assigning every sentence a set of coordinates in a space where distance is meaning. A question about "employee time off" lands near a policy document about "leave entitlement" even though they share no words.

Retrieval moved the problem rather than solving it. The model was now only as good as what you handed it, and handing it the right thing turned out to be its own discipline. That problem comes back at the end.

Tools: the difference between describing and doing

The third failure is the one that separates a chatbot from everything that came after. The model could talk about actions but could not take them. Ask it to check a calendar and it would produce text that looked like a calendar check. Words about doing are not doing.

Tool use, or function calling, closes that gap with a trick that is almost embarrassingly simple. You tell the model which functions exist, in a structured format. When the model decides one is needed, it does not write prose. It writes a structured request: call this function with these arguments. Your code intercepts that, runs the real function, and feeds the result back into the context. The model then continues as if it had done the thing itself.

The model never touches the calendar. It produces a description of the action so precise that a program can execute it. That is the entire bridge between language and the world, and it is worth being clear about how thin it is. A model that can call a search function and a database query is not more intelligent than one that cannot. It is more connected. The intelligence is the same. The reach is different.

MCP, which has become the common protocol for describing these tools, exists so that the reach can be standardized. Anyone can expose a tool, any model can be told about it, and the two do not need to have met before.

Agents and loops: when one step is not enough

Once a model can act, the fourth failure arrives on schedule. Real tasks are not one step. "Find out why the build is failing and fix it" is a sequence of looks and decisions, each depending on the last. A single prompt, however well constructed, cannot do that.

An agent is a model placed inside a loop. Observe the situation. Decide on an action. Take it. Look at what happened. Decide again. The simplest published form of this is called ReAct, for reason and act, and its whole content is that the model alternates between writing a thought and choosing a tool until it judges the task done.

Most of the current excitement about AI is excitement about this loop, and it deserves some of it. Watching one work through a problem for the first time is a real shock.

Most of what you read about agents leaves out the arithmetic. Each step in the loop is a prediction, and predictions are not perfect. Suppose each step is right 95 percent of the time, which is generous. After ten steps, the chance that nothing has gone wrong is about 60 percent. After twenty, about 36. The errors do not cancel out. They compound, because a wrong observation at step four produces a confident wrong decision at step five, and the model has no way to know that the ground shifted under it.

This fact has organized the field for the last two years. The question stopped being how to make agents smarter and became how to stop a chain of guesses from drifting into nonsense. That is a different problem, and it has a different kind of answer.

The harness: what surrounds the model matters more than the model

The fifth failure is the one nobody wanted to admit. A single agent in a loop, left alone, is not reliable enough to be trusted with anything that matters. And the response, when it finally came, was not a better model. It was everything built around the model.

That surrounding structure is the harness. The word is well chosen. A harness does not make the horse stronger. It makes the horse's strength usable, and it keeps the cart on the road.

A harness includes the guardrails that check an agent's proposed action before it executes and refuse the ones outside a defined boundary. It includes verification, where a second pass (often a second model, sometimes a plain script) checks the first one's output against the original request. It includes retry logic that catches a failed tool call and tries again with different arguments rather than proceeding on garbage. It includes the checkpoints where the agent stops and waits for a person before doing anything irreversible. And it includes routing: deciding which model, with which tools, gets which part of the job.

That last piece grew into multi-agent systems. Instead of one agent doing everything, you have one that plans, several that execute narrow subtasks, and one that reviews. Each has a small job and a small context, which keeps the error rate on each step low, which is the only thing that matters when steps multiply.

The value has moved. Two years ago the question was which model you had access to. Today the models are close enough to each other that the harness decides whether the thing works. Engineering discipline, the old unfashionable kind with tests and checkpoints and a written list of the ways it can fail, is what separates a demo from a product. The people who skipped that step because "AI handles it" are the ones now explaining to their customers why it did not.

Context engineering: the prompt grew up

Prompting was about a single message. You crafted one block of text and sent it. Context engineering is about the entire window the model sees at the moment of every prediction, across a task that may involve hundreds of predictions. What documents are in there? What tool results? What summary of the last forty steps? What has been thrown out to make room?

The window is finite. A model reads a fixed number of tokens, and a long agentic task fills that budget fast. Every tool result, every intermediate thought, every retrieved document sits in the context and competes for the model's attention. Attention, remember, is the mechanism from the transformer: every token weighing every other. Fill the window with noise and the signal gets weighted down with it. The model does not get dumber. It gets distracted, in the most literal sense that word has.

So context engineering is the discipline of deciding, at each step, what the model should be looking at. Which of the last twenty tool results still matter. When to compress a long history into a short summary and accept the loss. Which retrieved passages to include and which to cut. What to hold in a separate memory outside the window and pull back only when needed. It is curation, unglamorous, and it is the difference between an agent that finishes a two-hour task and one that loses the plot at minute forty.

We began with the problem of arranging text so that the next prediction would be useful. That was prompting. We end with the problem of arranging text so that the next prediction will be useful, except now the arranging happens continuously, programmatically, across a long task, with retrieval and memory feeding into it. It is the same problem. It grew up and got a bigger house.

The burden moved to you

I said at the start that each technique is a scar, and there is a pattern in the scars, and it points somewhere.

At every stage, the failure was the same failure. The model predicts; it does not know. And at every stage, the remedy was to take responsibility away from the model and put it on the person operating it. Prompting made the question your job. Retrieval made the facts your job. Tools made the connection to the world your job. The harness made reliability your job. Context engineering made the model's entire field of view your job.

The instrument got more capable, and as a direct result more was asked of the one holding it. That is what capable instruments have always done. A scalpel demands more of a surgeon than a butter knife does, and the surgeon who thinks the scalpel is doing the work is a surgeon you want to avoid.

Models will keep getting more capable. The next one will make some of today's harness unnecessary and some other part of it more important, and you will not know which in advance. What will not change is where the responsibility sits. The output is yours. The judgment about whether it is good enough is yours. The decision about what the machine sees, and what it is allowed to touch, is yours. The people who will do well in the next five years are the ones who understand that and get serious about it. The ones who will do badly are still waiting for the model to take the weight off them.

It will not. That is the whole lesson, and it took a lot of failures to learn it.