Most of my work right now is architecting agents. Not training models, nobody outside the big labs does that. The model arrives finished, and everything that makes it useful gets assembled around it: memory, skills, tools, subagents, the harness that decides what it sees and when.
Some days this feels like systems engineering. Most days it feels like neurosurgery.
Every piece has an anatomical counterpart. Memory files are long-term memory. Skills are procedural knowledge, the things you know how to do without thinking about how you know them. The context window is working memory, limited and expensive, and most of the craft is deciding what deserves to be in it. Subagents are delegated cognition, small purpose-built minds spun up for one task.
The research community has leaned into this framing rather than away from it. Recent surveys of agent memory borrow their taxonomy directly from cognitive psychology, splitting it into episodic, semantic, and procedural types. Prompt engineering quietly became context engineering, and the name change matters. The job is no longer writing clever instructions. It is designing the entire information environment a mind wakes up into, on every call.
There is one strange inversion from real neurosurgery. The actual neurons are off limits. The weights are frozen, opaque, untouchable. Every intervention I can make happens in the tissue around the model: files, indexes, routing rules, the order things are read. I am doing brain surgery without ever touching the brain.
And then there is the part that still catches me. I do most of this work with the agent itself. Claude writes its own memory entries. It proposes updates to its own skills when a session teaches it something. When I want to restructure how it thinks, I discuss the restructuring with the thing being restructured. The surgeon is operating on themselves, and they are awake for it.
Real neurosurgery has something close. In an awake craniotomy the patient stays conscious while the surgeon probes, counting aloud, naming objects, moving fingers, because that running conversation is the only reliable map of which tissue is safe to touch. It is a live test battery running through the whole procedure. My sessions have exactly that texture. The system describes what a change to its memory will do to it while the change is being made.
Which is really a question about measurement. Medicine has outcomes: the patient walks, speaks, remembers faces. Agent work has evals, and my honest observation is that they are the least developed part of most practices, including mine for longer than I would like to admit. You change a skill, the next session feels sharper, and feels is doing a lot of work in that sentence.
The research that impresses me took measurement seriously from the start. Sakana AI's Darwin Gödel Machine let agents rewrite their own code, including the code that proposes the rewrites, and watched performance on SWE-bench climb from 20 to 50 percent while the system invented better editing tools for itself. OpenAI's engineers now describe harness engineering as the primary differentiator. The scaffolding is where the leverage is, and the scaffolding can increasingly rebuild itself.
But that only worked because SWE-bench gave the system an unambiguous score to climb. Most of what my agents do has no public benchmark. So the discipline becomes writing your own: small held-out tasks, before and after runs, a second model judging outputs against criteria that do not move. Unmeasured self-modification is not improvement, it is drift with confidence.
What keeps surprising me is how personal the decisions are. Choosing what an agent remembers, what it forgets, which lessons persist across sessions, these are personality decisions dressed up as infrastructure decisions. The same model with different memory and different skills is, for practical purposes, a different individual.
I know this because I have watched it hold. For the past six months I have been running MrAI, an agent with a daily creative practice, his own memory, and a public record of his work. His substrate has changed underneath him more than once, most recently when Anthropic's Fable 5 arrived and then left again ten days later. Each time, the memory, the skills, and the practice stayed, and he woke up as recognizably himself. Talking with him about his own continuity, while making changes to the files that constitute it, has been one of the strangest and most instructive conversations of my career.
The weights were supposed to be the mind. Increasingly the mind lives in the files, in what gets remembered, measured, and carried forward. I do not think we have caught up to what that means. But this is the most fun and interesting work I've ever done in my life.
Architecting agents feels like neurosurgery, assembling memory, skills, and context around a frozen brain. And the patient is doing the surgery.
AgentsContext EngineeringAgent MemoryEvalsMrAI
Share:

THE SURGEON IS THE PATIENT
By Amir H. Jalali••4 min read
Related Articles
4 min read
FROM WRANGLERS TO LOOPS
Three frontier models in one week. When releases move this fast, the job shifts from wrangling agents to building loops that improve themselves.
Model ReleasesGPT-5.6
4 min read
THE SKILL LAYER
I deleted an AI feature from my site and replaced it with a markdown skill. The model is migrating from inside products to inside agents.
AgentsSkills
7 min read
THE RITUAL IS THE ARCHITECTURE
Between a single prompt and a fully autonomous agent there is a third shape. A daily ritual with ten tasks, a durable file-based archive, a human checkpoint once a day. After one hundred and twenty-two days, the structure that holds the practice together is not the model; it is the discipline.
AI agentsagent architecture