The genius without a hippocampus
Your AI agent is brilliant for 90 minutes. It solves a complex incident, finds the undocumented endpoint, discovers the API rejects requests without a specific header, learns that the ticketing system duplicates entries if you ask for closure twice in a row. It finishes the task, you close the session, you go for lunch.
When you come back, you ask it the same thing. The agent starts from scratch again. It tries the wrong endpoint again, rediscovers the header, duplicates tickets again before learning not to. It pays the cost of discovery one more time. And again. And again.
This isn't a model failure. It's a design failure. We've built geniuses without a hippocampus: capable of reasoning like few humans when they have context, incapable of retaining what they learned from one session to the next. And in production, that amnesia has a cost — in token bills, in lost hours, in result variability.
We've spent the last couple of years building agents that do remember. The interesting part: the difference between one and the other isn't in the model. It's in the system around the model. That's what we call a harness, and it's where you decide today whether an AI agent is a product or a demo.
Why model intelligence is no longer the bottleneck
Three years ago, improving an agent meant waiting for the next model. GPT-3.5 failed where GPT-4 succeeded; you suffered until the next release and the raw capability jump compensated.
In 2026 that dynamic broke. Frontier models — Claude Opus, GPT-5, Gemini 3, Llama 4 — sit in a performance band where the difference between "best" and "second best" is marginal for most enterprise tasks. Changing models doesn't transform the agent. It fine-tunes it.
What does transform the agent is the system around the model. There's an experiment that circulates in the community and captures the point well: same version of the same model, same task — building a small 2D game editor. Without a structured system around it, the agent spent a few dollars, took 20 minutes, and produced something non-functional. With a well-designed system — clear instructions, persistent state, mandatory verification, bounded scope, defined lifecycle — the agent spent more, took more time, and produced a usable product.
The model didn't change. Only the environment did. The quality jump was an order of magnitude.
That result has an uncomfortable implication for anyone still evaluating AI by model benchmark: if your agent doesn't work in production, the cause is very unlikely to be the model. Almost always, the cause is that the agent is operating without a harness.
What a harness is and why it matters
A harness is the set of pieces around the agent that force it to behave like a system, not like a conversation.
It's not a framework. It's not a library you install. It's an architectural decision: the agent lives inside a repository, with files it reads on startup, state that persists on disk, verification that runs before declaring a task complete, scope defined in a structured file, and a lifecycle that cleans up and hands off when it finishes.
Five subsystems that are always there in production projects — because without them the agent improvises:
Instructions. An AGENTS.md or CLAUDE.md file in the repo that the agent reads at the start of each session. It contains the operational guide: what this system does, what conventions it follows, what tools it has, what decisions are already made. It isn't human documentation — it's a manual for agents. The format changes: less prose, more rules; less historical context, more current state.
State. Files that record what's been done and what's left. A progress.md the agent updates after closing each step. A feature_list.json with tasks and their status. A log of decisions left halfway. The agent doesn't start each session blind — it reads the state, picks up where it left off, and leaves a record when it finishes.
Verification. The agent doesn't declare "done" — it proves it. Before closing a task, it runs tests, lint, type-check, whatever applies to the domain. If it passes, it moves on; if not, it doesn't. This turns trust into something measurable. "The model says it's fine" stops being a criterion.
Scope. One thing at a time. The agent doesn't freely decide what to do; it reads a prioritized task list, picks one, closes it, and goes back to the list. Without this, the agent piles up half-finished work and diverges — the classic LLM trap of getting excited about tangents.
Lifecycle. Each session starts with an init.sh that checks the environment (dependencies, service health, available data) and ends with a handoff.md summarizing what was done and what's left. This is what allows the next session, whether by the same agent or another, to pick up without losing context.
These five subsystems don't require any special product. They're built with plain files in the repo, bash scripts, JSON, markdown. The complexity isn't in the technology; it's in taking design seriously.
Agent memory: three horizons, three technologies
An agent's memory isn't one thing. It's three horizons that combine according to need:
Short-term: active context. The conversation of the current session, what the model has "in mind" in the prompt. Limited by the model's window (200K or 1M tokens depending on the provider). If the session is long, you have to compress, summarize, or paginate. This is managed with context engineering techniques: how much history to include, what to discard, what to summarize.
Medium-term: project memory. Files in the repository that outlive a session: the CLAUDE.md with project conventions, the progress.md with task state, the decisions.md with past reasoning. This is what allows an agent to open a new session and know "where it was" without re-exploring everything. It's the most underrated form of memory — it works without libraries, without external services, just with versioned files.
Long-term: persistent memory over facts. What the agent must remember between projects, between users, between months. Specialized solutions come in here — MemGPT for managing the agent's own hierarchical memory, Zep for conversational memory about users and facts about them, custom vector stores for semantic recall. The choice depends on the case: if you need the agent to remember "this client prefers meetings on Tuesdays", that's Zep or equivalent; if you need it to remember "what critical processes exist in this sector", that's a knowledge vector store.
A common mistake: using long-term when medium-term was enough. Almost every stuck enterprise agent we see thinks it needs MemGPT when what it lacks is a well-written CLAUDE.md. Sophisticated persistent memory is justified when there's real multiplicity of users or contexts. For an agent operating on a single project, the repo files are 90% of the problem solved.
Skill graduation: the agent that writes its own manual
The technique that most changes the operating cost of a production agent has an awkward name: skill graduation. The idea is simple. When the agent solves a complex task for the first time — discovering the right endpoint, navigating a multi-step process, normalizing data from a new source — that learning doesn't stay in the session. It crystallizes into a reusable markdown file that any future session can load and execute.
The first run is expensive on purpose: the agent explores, fails, learns, converges. But the output isn't just "task done" — it's a new skill: a document with the exact steps, the API calls that work, the known errors, the workarounds. Next time the task comes up, the agent loads the skill and executes it directly. Cost drops from minutes to seconds, from several dollars to cents, and result variability falls to near zero.
We have clients where this technique changed the economics of whole processes. One case: an agent that generates monthly reports over heterogeneous sources (a corporate Drive, an ERP, two scattered spreadsheets that someone updates by hand). The first cycle took two hours — the agent had to figure out which sheet was the "right" one, which columns were in European format and which in US, which rows were headers and which data. That knowledge crystallized into a skill versioned in the repo. Subsequent cycles: minutes. And when someone changes the sheet's structure, the agent detects the mismatch and updates the skill under human approval — it doesn't rewrite it blindly.
There's an important nuance. A graduated skill isn't a script. It's a hybrid: natural-language instructions (what to do, when, what errors to expect) combined with deterministic pieces (exact commands, API calls, regex). The agent still reasons, but starts from a map, not a blank page.
What this looks like in practice
A real project, anonymized. Professional services company, team of around 200 people, a lot of manual process over Confluence, Drive, and an in-house ERP. The client's initial framing was "we want an agent that answers internal questions". The trap: that brief, without a harness, produces a chatbot that loses touch with the problem within two months.
What we set up:
- An
AGENTS.mdin the agent's repo describing the company, the connected systems, internal conventions (what each acronym means, what processes are critical, what question types require escalation). - A
progress.mdthe agent updates with each relevant conversation — what was asked, what was answered, what's still pending. - A
skills/folder with versioned markdown. When the agent figures out how to handle a new class of question — "quarterly reports for area X" or "status of project Y" — it writes the procedure down and saves it. Next time it executes directly. - Verification: every agent response cites its sources (Confluence URL, SQL row, Drive fragment). If a response can't be cited, it isn't given.
- Lifecycle: each session starts with a script validating that connected systems respond, and ends with a commit to the state.
Result at three months: average cost per query dropped around 70% versus the first month (graduated skills eat the bulk of frequent questions). Response time for routine cases dropped from 40-60 seconds to 5-10. And, most useful, the skills became a client asset — living documentation of how the company actually works, written by the agent during its discovery and reviewed by the team.
When a new agent arrives (because the model changes, because we layer a second orchestration), it doesn't start from zero. It reads the previous agent's repo and starts with all the crystallized learning. That's operational continuity, not vendor churn.
What to demand when you contract a production AI agent
Boiled down to a checklist:
- Versioned agent repository, with
AGENTS.mdorCLAUDE.mdreadable by humans and agents. - State persisted on disk, not just in the model session. If the agent restarts, it must be able to pick up.
- Verification as executable evidence: tests, lint, output validation. None of "the model said it's fine".
- Graduated and versioned skills in a repo directory. Reusable, auditable, executable.
- Real multi-session memory: the agent doesn't start each session blind.
- Ability to resume work from an interrupted session: if the connection dropped, if context ran out, the agent must continue from the last state.
- Documented handoff between sessions: structured notes the next agent reads.
- Decision observability: every agent step recorded in a format a human can review.
If the team handing you the agent can't show you these eight elements in the system's own repository, you're not contracting an agent — you're contracting a chatbot that next week will cost the same in exploration as today.
How we build it in production
In serious agent projects, the harness isn't something assembled at the end — it's the first thing. Before choosing a model, before designing the prompt, before writing a single tool, we map what the agent must remember between sessions, what executable evidence will prove it did its job, and where the skills that will be graduated will live.
The agent's repository is a proper software project, with its own CI/CD, tests over the critical prompts, automated evals over behavior. It isn't a collection of scripts taped together; it's a versioned deployment unit the client can read, edit, and maintain when we're no longer around.
Behind each of these systems is a team of engineers who believe that serious AI engineering starts by asking how the agent is going to learn, not by picking the model. Technical excellence isn't measured by the elegance of the first prompt — it's measured because the agent costs less every month than the one before, and a year from now is still the client's asset, not the provider's.
Production means the agent compounds value over time, not that it restarts every Monday. That's the difference between a system that learns and a genius without a hippocampus.
If your AI agent is fine for demos but you're scared to leave it alone on a Monday morning — or if you're evaluating vendors and lack criteria to tell a project apart from an expensive chatbot — we can audit your system, map its current harness, and hand you the plan to make the agent start remembering.
