Direct answer: Create one Zep user per person, one session per conversation thread, then search across all sessions with memory.search to pull cross-thread context at answer time.

Why Does One Big Session Break Down?

New builders dump every message into a single session. It works for a demo, then falls apart. Summaries get noisy, latency grows, and Zep cannot tell a quick throwaway question from a project you have been running for months.

The fix is to treat sessions as threads, not as users. A user is a person. A session is one line of work — a support ticket, a coding task, a weekly check-in.

How Should You Model Users and Sessions?

Use stable IDs you already own. Do not let Zep invent them.

If several sessions should share facts — say an org-wide policy — attach them to the same graph ID. Facts written there become visible to every session in that graph.

How Do You Write and Read Across Sessions?

Write with memory.add as messages happen. Read with memory.search when you want semantic lookup rather than just the last N messages.

hits = client.memory.search(
    user_id="u_8421",
    query="preferred deployment target",
    limit=5)
for h in hits:
    print(h.fact, h.created_at)

Note the parameter: user_id, not session_id. That is what makes search span threads. Searching by session only returns facts from that single thread — a common mistake that makes memory seem broken.

Zep Memory Search vs Plain Vector Search

Both retrieve facts. They differ in what they do with time.

If your users never change their mind, a vector store is cheaper. Once facts evolve, the temporal layer earns its keep.

How Do You Handle Latency and Token Cost?

Three practical levers worked for us on a real assistant:

• Request mode="summary" and cap lastn at 4-6 messages.
• Trigger writes right after the user turn, not after the assistant reply, so ingestion overlaps with generation.
• Cache the returned context string for the session and reuse it within one turn instead of calling memory.get twice.

Skipping these pushed p95 latency from roughly 400 ms to over 1.5 s in our tests, mostly from unnecessary ingestion calls.

Is the Graph Layer Worth the Complexity?

Only sometimes. If your assistant answers questions about a single person’s history, a user-scoped memory is enough. The graph starts paying off when you need relationships — “which customers reported this bug” — where entities connect to each other, not just to a user.

One honest limitation: you cannot hand-edit the graph. Facts are extracted by a model, so bad input produces bad facts, and the only fix is re-ingesting corrected messages.

Quick Summary

• Model people as users and threads as sessions; never merge them.
• Use memory.search with user_id to pull context across sessions.
• Share facts between sessions through a common graph ID.
• Keep token cost down with summary mode and a small lastn.
• Skip the graph until you actually need entity relationships.

Direct answer: Install Zep via Docker or its cloud API, add memories with the SDK, then query them with a user or session ID to get ranked context.

We will use Zep in Python, though it also ships Node and Go clients. All you need is an API key and one small script.

What Is Zep and What Problem Does It Solve?

Zep is a memory service that sits between your LLM and your application. Instead of feeding raw transcripts, you send messages to Zep as they happen. Zep extracts facts, tracks how they change over time, and returns a compact summary plus relevant facts on each turn.

The temporal part matters. If a user says “I live in Berlin” in January and “I moved to Lisbon” in June, Zep does not lose the history — it invalidates the old fact and keeps the new one current. Plain vector search usually cannot do this.

How Do You Install Zep?

You have two options. The hosted cloud (app.getzep.com) needs only an API key and is the fastest route. Self-hosting uses Docker Compose and bundles Postgres, Neo4j, and the Zep service.

1. Create a free account at app.getzep.com and copy your API key.
2. Install the SDK: pip install zep-cloud.
3. Export your key: export ZEP_API_KEY=your_key.

For self-hosting, clone the repo and run docker compose up. It spins up four containers and takes roughly 3-4 GB of RAM, so a cheap 2 GB VPS will struggle.

How Do You Add Memories to Zep?

Every memory is attached to a session, and sessions belong to a user. That two-level model is the core of Zep, so start by creating a user.

from zep_cloud.client import Zep
client = Zep(api_key="your_key")
client.user.add(user_id="u_1", email="a@b.com")
client.memory.add_session(session_id="s_1", user_id="u_1")
client.memory.add(session_id="s_1", messages=[
  {"role": "user", "content": "I am allergic to peanuts"},
  {"role": "assistant", "content": "Noted."}
])

Zep ingests messages asynchronously. Facts usually appear within a second or two, but do not expect them instantly in a tight loop.

How Do You Query Memory and Inject It Into a Prompt?

Call memory.get with the session ID. You get back a summary, a list of relevant facts, and recent message context. Drop those strings into your system prompt, not the whole transcript.

mem = client.memory.get(session_id="s_1",
    lastn=6, mode="summary")
context = mem.context
prompt = f"Known facts about the user:\n{context}\n\nAnswer:\n{question}"

Using mode="summary" keeps token counts low. Switching to mode="basic" returns more raw detail when you need exact wording.

Is Zep Free?

Zep Cloud has a free tier aimed at development, with paid plans billed on stored memories and API calls. The open-source version is free to self-host under the Apache 2.0 license. For a side project, self-hosting with Docker costs nothing beyond server time.

Who Should Use Zep — and Who Should Not

It fits chatbots, assistants, and support agents that talk to the same user across weeks. If your agent is stateless and answers single questions, Zep is overhead you do not need — a plain prompt cache will do.

Two real drawbacks. First, self-hosting pulls in Neo4j, which is heavy for small deployments. Second, fact extraction is probabilistic: occasionally an odd statement gets stored as a “fact” and you will need to prune it.

Quick Summary

• Zep is an open-source memory layer that stores agent conversations in a temporal knowledge graph.
• Install the SDK with pip install zep-cloud or self-host through Docker Compose.
• Everything hangs off two IDs: a user and a session.
• Query with memory.get and inject the returned summary into your prompt.
• Best for long-running assistants; overkill for one-shot Q&A.