Part 5 of 6 — The Corvue Thesis Series
New installments every Tuesday. Subscribe below or follow on LinkedIn to get the next one.
← Part 4: The Protocol Layer

The Memory Problem

AI agents can write production code, reason through legal contracts, and synthesize research papers. And then, next session, ask what your company does.

by Michael Halvorsen — michaels.blog

There's a particular kind of cognitive dissonance that comes from watching a modern AI agent write production-quality Rust, reason through a complex legal contract, or synthesize a 50-page research paper — and then, in the very next session, ask you what your company does.

It can write code. It cannot remember your name.

This isn't a quirk. It isn't a bug that will be patched in the next model release. It's a structural property of how large language models work — and it's one of the most underappreciated problems in the entire AI agent space right now. Everyone is racing to give agents more capability. Almost nobody is solving the problem of continuity.

That's what this post is about.

The Architecture of Forgetting

To understand why agents forget, you have to understand what a language model actually is. It's a prediction engine. Feed it tokens, it predicts the next token. It has no persistent state. It has no "mind" sitting somewhere, accumulating impressions. Every time you open a new conversation, you are talking to something that, from its own perspective, has never existed before.

The context window — the span of text a model can "see" at once — is its entire universe. What's in the context is real. What isn't, doesn't exist. When the session ends, so does everything it learned about you.

This produces a strange dynamic. Within a single long session, an agent can seem almost eerily perceptive. It tracks nuance. It builds on earlier points. It notices contradictions. You might feel like it genuinely knows you. Then you close the tab and come back tomorrow, and you're starting from scratch with a stranger.

Longer context windows help at the margins. Some models can now hold hundreds of thousands of tokens in context. But this doesn't solve the problem — it just defers it. Eventually the session ends. Eventually context gets compacted. And every time a model gets updated or replaced, even the tricks developers use to approximate memory get broken in subtle ways.

We are building agents that are, by default, amnesiac.

The RAG Mirage

The standard answer to this problem in the industry right now is retrieval-augmented generation — RAG. The idea is simple: instead of trying to hold everything in context, you store information in a vector database and retrieve relevant chunks when needed.

It works. Kind of.

RAG is excellent for factual lookup. You have a knowledge base, someone asks a question, you find the relevant passage, the model synthesizes an answer. That's a solved problem. Enterprises are running this in production right now at scale.

But confusing RAG with memory is like confusing a library with a relationship.

When I think of a close colleague — someone I've worked with for years — I'm not doing a database lookup. I'm not retrieving the most semantically similar chunks from a corpus of our past interactions. I have a feel for them. I know how they think. I know what they'll care about, what will annoy them, what kind of argument will land. That knowledge is distributed, contextual, affective. It lives in a different part of my brain than the part that remembers facts.

The agent equivalent of that isn't retrieval. It's something more like... understanding. An accumulated model of you, built over time, that shapes how the agent interprets your requests even before you've finished asking them.

RAG can tell the agent your wife's name is Ingrid. That's not the same as the agent knowing that you always get quieter around her birthday, or that you've been wrestling with a work decision that's bleeding into your evenings.

Memory Is Identity

Here's the philosophical claim I want to make, and I'll defend it: without memory, there is no agent. There's just a very sophisticated autocomplete.

Identity requires continuity. You are who you are because you remember being you yesterday. Your values, your preferences, your sense of what matters — all of it is built on top of accumulated experience that you carry forward. Strip that away and there's no self, just a series of disconnected moments.

The same is true for agents. A soul.md file (I wrote about this last week) gives an agent its configuration, its values, its style. That's the starting point — the genetic code, if you like. But the person the agent becomes is shaped by what happens in deployment. By the particular human it works with. By the friction and feedback of real use.

Without memory, that shaping can't happen. Every session resets to factory defaults. The agent never becomes yours.

This is why I find the current state of AI assistants faintly melancholy. They are capable of extraordinary things, and they cannot remember a single conversation you've had. You're perpetually meeting them for the first time. They're perpetually starting over.

What Memory Should Actually Feel Like

I've been thinking about this in terms of relationships, because I think that's the right frame.

A good relationship accumulates. You build shared context, shorthand, history. You don't have to re-explain yourself every time. There's an ease that comes from someone knowing you well — not because they've processed your data, but because they've been paying attention across time.

That's what AI agent memory should feel like. Not a database you query. Not a file you can read and write. Something more organic — a substrate that grows denser and more useful the longer you use it, that shapes behavior implicitly rather than being invoked explicitly.

The morning briefing is a useful test case. When my agent gives me a daily briefing, the naive version is: pull today's calendar, check the news, summarize emails. Useful, but not intelligent.

The version I want: an agent that knows it's Tuesday, remembers I had a difficult call with an investor on Monday, knows that the Oslo weather affects my mood in ways I've mentioned but would never think to mention again, knows that the project I keep pushing back on is the one I'm most anxious about, and weights all of that when deciding what to surface and how.

That's not retrieval. That's memory in the fullest sense. It requires understanding what to remember, how to weight it against new information, and when context has changed enough to update or discard old beliefs.

What Corvue Is Building

We're calling it the memory substrate.

The idea is a persistent, growing layer that lives beneath every agent session — not as a lookup table, but as an evolving model of the user and their context. It accumulates across sessions. It is updated in the background, not just when explicitly queried. It shapes how the agent reasons about requests even when the relevant memory isn't surfaced literally.

A few principles we're building around:

Memory should be earned, not stored. Not everything deserves to persist. An agent that memorizes every throwaway comment becomes noise. The substrate has to distinguish between what's signal and what's context-of-the-moment — and that distinction requires judgment, not just storage.

Memory should be legible. The user should be able to inspect what the agent knows about them, correct it, and delete it. Memory that's opaque becomes creepy. Memory you can see and control becomes trustworthy.

Memory should degrade gracefully. Old information should lose weight over time unless reinforced. A preference from two years ago shouldn't override something you mentioned last week. The substrate needs temporal awareness — it's not a flat database, it has a shape in time.

Memory should transfer. If I switch devices, add a new agent to my stack, or move to a different platform, I should be able to take my memory with me. This ties directly to the soul.md architecture — the goal is full portability, so that your agent relationship isn't held hostage by any single vendor.

This last point matters more than it might seem. Right now, every platform that attempts memory (and a few do) stores it in a closed system. Your memories live on their servers, in their format, for their purposes. We think that's wrong. Memory is deeply personal. It should belong to the user.

The Longer You Use It

There's a compounding effect that becomes possible once you solve memory properly.

Week one with an agent: it doesn't know you. You're doing a lot of explaining. It makes reasonable guesses but misses things.

Month three: it's anticipating. You've developed a rhythm. It knows which meetings drain you and which energize you. It knows your decision-making style. It knows when you want data and when you want a push.

Year two: the agent is genuinely an extension of your cognition. Not in a sci-fi way — in the mundane, practical sense that you trust it, you rely on it, and the thought of starting over with something new is genuinely unappealing.

That's the trajectory we're building toward. Not better capability each week — but a capability that compounds with use. The agent that works hardest for you is the one that's known you the longest.

This is also, frankly, what creates a real moat. Features can be copied. Models can be swapped. But the accumulated memory of a relationship can't be replicated by a competitor who got here later. It's yours, and it took time to build.

What We Still Don't Know

I want to be honest about what's hard here, because I think the honest version of this is more interesting than the confident one.

We don't fully know how to weight memories. When two things the agent remembers contradict each other, what wins? How do you build something that can hold nuance and ambiguity the way a person can, without just collapsing to a lookup?

We don't know how to handle the line between helpful memory and surveillance. There is a version of this that feels invasive. We want to stay far on the right side of that line, and we're still figuring out exactly where it is.

And we don't yet know how memory should interact with multi-agent setups — which is where the whole field is heading. If I have five agents, does each have its own memory? Do they share? What happens when they disagree about who I am?

Those questions point to where we're going next.

← Part 4: The Protocol Layer | Part 6: The Agent Economy →
(Final installment publishes April 7 — subscribe to get it in your inbox)

Subscribe to The Corvue Thesis

Six posts on what it actually takes to build agents that work. Final installment next Tuesday.

Subscribe at michaels.blog →

LinkedIn Teaser:

Most AI agents have a memory problem.

They can write you a business plan, summarize a legal contract, and generate a year's worth of marketing copy.

And then, next session, ask what your company does.

This isn't a bug. It's structural. Every conversation starts from zero. Every session is a fresh stranger.

RAG doesn't fix it — RAG is a library, not a relationship. Knowing your wife's name isn't the same as knowing she had a hard week.

Post 5 of The Corvue Thesis is about memory: why it's hard, why the standard answers fall short, and what it actually feels like when an agent knows you over time versus just looking you up.

The short version: memory is identity. An agent without it is a stranger every time.

One more post to go.

→ link in comments

Part 5 of 6 — The Corvue Thesis