Core Memory vs Archival Memory: Letta's Two-Tier Model

Every context window is a finite resource, and every fact you keep permanently loaded into it is a tax on everything else the model does in that turn. This is the problem that most memory architectures dance around, and it's the one Letta (formerly MemGPT) confronts head-on with a deceptively simple idea: not all memory deserves the same real estate.

Letta's runtime splits an agent's memory into two tiers. Core memory consists of editable blocks that live permanently inside the prompt — the user's name, their stated goals, standing constraints, whatever the agent should never have to look up because it should never have forgotten it. Archival memory lives outside the prompt, in a queryable Postgres or SQLite database, and gets pulled in only when the agent decides it's relevant. It's the difference between what you keep in your head walking into a meeting and what you keep in a filing cabinet you can walk over to.

This split isn't a memory feature so much as it's an admission that context windows are a budget, and budgets require triage.

Why "remember everything, always" doesn't scale

The naive version of agent memory is to accumulate facts and inject all of them into every prompt. This works exactly until it doesn't — usually around the point where the accumulated facts start crowding out the actual conversation, or where retrieval latency and token cost start climbing linearly with how long a user has been using the product. A support agent that's been talking to the same enterprise customer for eight months has, at that point, more historical context than any context window can hold at once, let alone hold usefully.

Core memory forces a decision at write time: is this fact important enough to pay rent in every single prompt, forever? For a user's name, their communication style preference, or a hard constraint like "never suggest solutions that require downtime" — yes, obviously. For the specific ticket number they mentioned in March — no, that belongs in archival storage, retrieved only when the conversation actually needs it.

Archival memory is a database, and that's the point

Treating archival memory as a real, queryable database rather than a growing blob of text is what makes the tier system work. Because Letta backs archival memory with Postgres or SQLite, an agent can search it, filter it, and retrieve exactly the slice relevant to the current turn — the same way an application queries a user table instead of loading every row into memory to find one record. This is a deliberate departure from treating "memory" as a vague conceptual layer and instead treating it as ordinary structured storage that happens to be readable by an LLM at inference time.

The practical effect: archival memory can grow without bound — years of interactions, thousands of facts — without ever directly costing prompt tokens. Only what's retrieved for a specific turn enters the model's context. Core memory, by contrast, must stay small by design, because every byte in it is paid for on every single call.

The editable part matters as much as the tiered part

What separates Letta's core memory from a static system prompt is that it's editable at runtime, by the agent itself. The agent can update its own core memory blocks as it learns new standing facts about the user — correcting an outdated preference, adding a new constraint, removing something that's no longer true. This is subtly different from the append-only pattern most memory systems default to, where new facts pile on top of old ones and contradiction resolution becomes someone else's problem.

An agent that can edit its own core memory can also demote something from core to archival, or promote a fact that turns out to matter more than expected. That flexibility is what keeps the always-in-prompt tier from becoming its own kind of bloat over time — a problem worth examining on its own, since what an agent should forget turns out to be one of the harder open questions in this space.

Where this tiering logic applies beyond Letta specifically

You don't have to adopt Letta's exact runtime to benefit from thinking in these terms. The underlying principle — separate what must always be present from what should be looked up on demand — shows up, in different clothing, across nearly every serious memory framework. Mem0 distinguishes short-term working memory from longer-term persisted memory. Zep, via its Graphiti engine, treats recent conversational context differently from the deeper temporal graph it builds over time. Even a hand-rolled system using nothing but Postgres and pgvector benefits from asking the same question Letta's architecture forces explicitly: does this fact earn a permanent seat, or does it get looked up?

The mistake to avoid is treating this as a binary you set once. A fact's tier should be able to change. A user's dietary restriction might start in archival memory (mentioned once, months ago) and get promoted to core memory the moment it becomes clear it governs every food-related recommendation the agent makes. Static tiering, decided at ingestion time and never revisited, recreates the same staleness problem tiering was meant to solve.

The budget framing is the useful part

It's tempting to treat memory tiering as a purely technical decision — "how much can fit in the context window" — but the more useful frame is economic. Every core memory block is a recurring cost paid on every single inference call, for as long as that block exists. Every archival memory entry is a one-time storage cost plus a marginal retrieval cost paid only when it's actually needed. Once you think about it that way, the design question becomes obvious: what facts about this user or task have a high enough hit rate — the odds they'll matter on any given turn — to justify permanent placement, versus facts with a low hit rate that are still worth keeping, just not worth paying for constantly?

A user's name has a near-100% hit rate; it's relevant to nearly every interaction. A note about a product they asked about once, six weeks ago, has a low hit rate — most turns don't need it, but the rare turn that does need it fails badly without it. Core memory is for the former. Archival memory, properly indexed and retrievable, is for the latter.

Building this without Letta

If you're rolling your own agent runtime rather than adopting Letta directly, the two-tier pattern is worth stealing even without the framework. A small, hand-curated set of "always inject" facts, kept deliberately short, paired with a proper database — vector-indexed, graph-indexed, or both — that the agent can query on demand, gets you most of the benefit without committing to a specific runtime. The discipline is in resisting the urge to let the "always inject" set grow unchecked, because that's the tier where bloat is most expensive and hardest to walk back once users start depending on what's in it.

Letta's real contribution here isn't the specific storage backend — it's making explicit a tradeoff that every memory system has to make implicitly anyway. Naming it, and building runtime tooling around it, turns an ad hoc compromise into a deliberate architecture decision.

---

Part of the "Agent Memory" series on aiskill.market.