How to Give an AI Agent Long-Term Memory with Letta

November 6, 2026

Letta gives an AI agent persistent memory across sessions. Instead of stuffing a growing chat log into the context window, the agent keeps labelled memory blocks and a recall archive, and decides for itself what to keep, edit, or forget. It was known as MemGPT until the rename.

Why not just keep the whole conversation?

Because context windows are finite and expensive, and after roughly 50 turns of a support conversation, most of the tokens are repetition. An agent with unmanaged history degrades: it forgets early constraints and over-weights the last few messages.

Letta’s approach is a small working memory that the agent edits. It reads from archival storage on demand rather than carrying everything.

What are the pieces of Letta’s memory?

Three layers, each with a different job.

  • Core memory blocks — labelled sections in the context window, such as user profile or persona. Usually a few hundred to a few thousand tokens. Writable by the agent.
  • Archival memory — an external vector store the agent can search using a tool call.
  • Recall memory — searchable conversation history, also reachable by tool call.

The key idea: the agent controls the boundaries. It writes to core memory when it learns a durable fact and searches archival memory when it needs an old detail.

How do you set up an agent?

Through the ADE (Agent Development Environment) or the SDK.

  1. Create an agent with a model and a system prompt.
  2. Define memory blocks: at minimum a persona block and a human block. Both are text, both editable.
  3. Attach tools the agent may call — archival search, archival insert, conversation search.
  4. Expose a REST endpoint or the Slack/Discord connector depending on where it lives.
  5. Send a message and watch which tool calls it makes. That observation is the whole debugging method.

Start minimal. Adding twelve tools to a new agent makes its behaviour unpredictable before you understand its memory writes.

How do you stop it forgetting something important?

Pin it. Anything the agent must never drop belongs in a block that is always in context, not in archival memory where it must be retrieved.

Then set the block labels carefully. A block called “preferences” that gets rewritten weekly is different from one called “account facts” that should be append-only. Giving a block an explicit instruction in its description — “append, never overwrite” — measurably reduces loss.

Audit what it stores. Dump the memory blocks weekly for the first month. You will find entries that are wrong, entries that duplicate, and entries that should have been deleted.

What does it cost to run?

Memory itself is free — the open-source core runs where you want. Cost comes from context tokens: every core memory block is tokenised on every call. A 2,000-token core memory on every message is a permanent tax.

The managed cloud option adds hosting and monitoring if you do not want to run the service. Self-hosting means Postgres and whatever storage your model calls need.

The practical lever is block size. Teams that trim core memory from 4,000 tokens to 1,200 often see better answers and a smaller bill, because the model focuses.

Where does self-editing memory go wrong?

Agents can write wrong facts into memory and then treat them as truth. A misheard name persists across every future session unless something corrects it.

Agents can also loop: writing a memory, reading it back, deciding it needs refinement, writing again. Cap the number of memory operations per turn.

And there is no automatic expiry. Without an explicit cleanup routine, archival memory grows indefinitely and retrieval quality decays as irrelevant older entries compete with current ones.

Who should use Letta?

  • Assistants that need to know a returning user’s context without re-explaining.
  • Support agents that should remember a customer’s history and prior resolutions.
  • Research agents that accumulate findings over days.

Not for: one-shot Q&A tools where no recall is needed, or products where storing user memory requires a consent and data-deletion story the team has not built yet. That obligation is real and arrives quickly.

Quick Summary

  • Letta (formerly MemGPT) splits memory into core blocks, archival storage and recall history.
  • The agent writes its own memory; you define blocks, labels and rules.
  • Pin critical facts in core memory rather than hoping retrieval finds them.
  • Every core-memory token is charged on every call — trim blocks and answers often improve.
  • Audit memory weekly; agents will store wrong facts and never expire them on their own.

Supported models, the ADE, and cloud plan details are on the Letta tool page.

Tags: