How to Give an AI Agent Long-Term Memory with Zep

November 9, 2026

Zep is a memory layer for AI agents. You push conversation turns into it, it extracts facts into a temporal knowledge graph, and you pull back a small memory block that says what the user wants, did, and when.

The pitch is simple once you have hit the wall it addresses. Chat history in a prompt does not scale: forty turns of context costs money, confuses the model, and still misses the thing the user said in March. Zep keeps the March fact and hands you three hundred tokens of it.

This walkthrough goes from API key to a working memory retrieval, using the zep-cloud Python SDK, then covers the settings that decide whether it works for your product.

What does Zep store that a vector database does not?

A vector database stores chunks and returns the most similar ones. Zep stores entities and facts with timestamps, extracted by an LLM, and answers questions about them in a graph. If a user said “I use Postgres” in January and “we migrated to DuckDB” in June, a similarity search may return both. Zep’s graph knows the second supersedes the first.

That distinction is why the returned memory is short and accurate instead of long and partly wrong.

Who this is for: chat products, support agents, and assistants where the same user returns over weeks. Who it is not for: single-session tools, or systems that need an exact audit log of every message, because Zep is summarising and extracting, not archiving.

How do you set up Zep in five minutes?

Create a project in the Zep dashboard and copy the API key. Install the client:

pip install zep-cloud

Initialize with your key. Then create a user and a session. Zep’s data model has exactly three levels, and mixing them up is the most common setup error:

  • User — a person. Persistent identity.
  • Session (formerly called a thread) — one conversation. Belongs to a user.
  • Message — one turn inside a session.
from zep_cloud.client import Zep
z = Zep(api_key=KEY)
z.user.add(user_id="u_1042", email="a@b.com")
z.memory.add_session(session_id="s_1", user_id="u_1042")

Older docs use z.memory.add_thread. Sessions and threads are the same object; the SDK renamed the surface in 2025 and both entry points still resolve for now.

How do you feed messages in?

Add both sides of the conversation, with role labels. Zep extracts facts from user turns and uses assistant turns for context.

z.memory.add(session_id="s_1", messages=[
  {"role": "user", "content": "We moved billing to Stripe last month."},
  {"role": "assistant", "content": "Got it, Stripe for billing."},
])

Processing is asynchronous. Extraction runs in the background and typically lands within a second or two for short messages. If the very next call in the same script reads memory, you may get an empty block. Deal with it by awaiting the task or by making retrieval tolerant of a miss.

How do you get a memory block back into your prompt?

Two endpoints matter, and they answer different questions.

Memory gives you the distilled facts for a session or a user. This is the one for a prompt, because you control the budget:

mem = z.memory.get(session_id="s_1")
prompt_context = mem.context

Search takes a query string and returns the most relevant facts and summaries:

res = z.memory.search(session_id="s_1", text="what database do they use", limit=5)

Use memory for the “who is this person” preamble in every request. Use search when the user asks a question that needs a targeted lookup out of months of history.

What is Graphiti, and do you need it?

Graphiti is Zep’s open-source engine, and it is what lets you build the graph yourself instead of through the hosted API. You need it if you want to run memory on your own infrastructure, or if you have custom entity types (say, medical codes or part numbers) that the default extractor will not recognise.

The hosted product and Graphiti produce the same shape of graph. The difference is operational: self-hosting means running Neo4j or FalkorDB, an LLM for extraction, and background consolidation jobs. That is three moving parts you do not manage on the cloud tier.

What does it cost and where are the limits?

Zep’s cloud pricing is per-credit, and credits burn on extraction and retrieval calls, not on message count. Free tier covers prototyping; production plans are usage-based and billed monthly at higher credits.

Three practical limits:

  1. Extraction needs an LLM. On the cloud, Zep supplies it. Self-hosted, you pay your own model provider per message ingested, which on GPT-4-class models is roughly a cent per long conversation.
  2. Sessions need to be ended. A session that never closes keeps accumulating and eventually exceeds the summary window. Call the end-session path when a conversation concludes.
  3. Entity types are opinionated. The default extractor finds people, companies, projects and preferences. Domain-specific nouns need custom instructions, or they arrive as untyped text and search for them is weaker.

How does this compare with just storing history?

A rough comparison for a support assistant that handles 30 turns per user and 200 users a day:

Approach Tokens per request Remembers March? Effort
Full history in prompt 4,000–12,000 Yes, if recent None
Last 10 turns 1,500 No Low
Vector search over turns 800 Maybe Medium
Zep memory block 300–500 Yes, with dates Medium

At scale the token column is the argument. Cutting 8,000 tokens down to 400 per call at 6,000 calls a day is a real invoice difference, not a theoretical one.

Quick Summary

  • Zep extracts timestamped facts into a graph instead of storing raw chat history, so facts that change get superseded rather than duplicated.
  • The data model is user, then session (formerly thread), then message; creating messages without a session is the usual first error.
  • Use memory.get() for a short prompt-ready block and memory.search() for targeted lookups over old history.
  • Ingestion is asynchronous, so tolerate an empty read immediately after writing.
  • Self-hosting Graphiti means running Neo4j or FalkorDB plus your own extraction LLM; the cloud tier removes all three.
Tags: