AI Agent Memory Architecture: Checkpoints and Vector Stores
A reasoning loop only survives one request unless its state is stored outside the worker. Without agent memory, the agent cannot resume a paused plan, recover after a crash, or recall a preference from an earlier session. Part 1 covered the control flow. This post identifies which state each later turn needs and where that state should live.
I’ll use the Market Analyst Agent — a small LangGraph agent that fetches market data and writes an analyst report — to anchor the hot-checkpoint discussion. The cold-vector and raw-Markdown sections are independent illustrative designs that show extensions the current project does not yet implement. Then I’ll cover when PostgreSQL, Redis, Qdrant, key-value stores, and plain Markdown files each make sense.
Every store below is read by the harness, the code that drives the loop around the model. The harness decides which of their contents reach the context window; the stores do not. This article is about where that state lives before the harness reaches for it. Part 3 and Part 4 cover what the harness does with the prompt next.
What is AI agent memory?
AI agent memory is the state layer that lets an agent preserve task progress, retrieve prior knowledge, and update what it knows across runs. In production it is not one vector database. It is a mix of hot checkpoints, cold semantic or structured stores, and human-readable document memory.
| Need | Best default | Why |
|---|---|---|
| Pause and resume one run | PostgreSQL checkpoint store | Durable, queryable, and easy to operate with app data |
| Low-latency transient state | Redis checkpoint store | Fast resume and short-lived state, with persistence trade-offs |
| Cross-thread semantic recall | Qdrant or pgvector | Retrieves memories by meaning, not only exact keys |
| Structured user facts | PostgreSQL or key-value store | Deterministic updates beat fuzzy retrieval for preferences and IDs |
| Project conventions and learned procedures | Markdown or JSON files | Human-readable, diffable, and easy for agents to update |
| Multi-entity relationship memory | Knowledge graph | Useful when relationships matter more than individual facts |
Do not start with memory because it sounds intelligent. Start with the user-visible failure: losing progress, forgetting a preference, repeating research, or failing to reuse a project convention.
The failures that require memory
A stateless agent can answer an isolated question, but it forgets the request as soon as the call ends. That design fails when the product needs any of the following behaviors:
- Pause and resume: a user starts a research task, closes their laptop, and comes back tomorrow. Without checkpointed state, the agent restarts from scratch.
- Multi-turn coherence: over a long conversation the agent has to remember what tools it called, what data it gathered, and what plan steps it finished.
- Personalization: a returning user expects the agent to know their risk tolerance, preferred analysis depth, and past interactions.
- Human-in-the-loop (HITL): the agent gathers its evidence and waits for a human to approve the next step. The “waiting” state has to survive process restarts.
In the Market Analyst Agent from Part 1, the request “Analyze NVDA” produces a plan, five tool calls, gathered data, and a draft report. When the user replies “looks good, but add competitor analysis,” a checkpoint store lets the agent load the state from the last completed step and add the competitor step. Without checkpointed state, it cannot resolve what “looks good” refers to and has to start over.
Long-term memory handles a different case. If the user returns a week later and asks, “Update my NVDA analysis,” the agent may need to recall a preference for conservative risk assessments and an interest in semiconductor stocks. A vector-backed memory store can retrieve those facts across sessions without asking for them again.
The implementation examples below use LangGraph, LangChain’s open-source library for building agents as explicit state graphs; the storage boundaries it draws generalize to any framework. LangGraph splits memory by scope. Every graph execution runs inside a thread, meaning one conversation or task. Persisted state within that thread is short-term memory. State shared across threads is long-term memory. Below, “thread” and “conversation” are interchangeable; I avoid “session” for that span, because Part 5 reserves it for the durable log of one run, several of which can accumulate on one thread. The model’s current context and in-process variables form the working-memory layer above both stores.
A taxonomy of AI agent memory
Before getting into implementation, it helps to classify what agents need to remember. The CoALA framework — Cognitive Architectures for Language Agents (Sumers, Yao et al., 2023) — is a widely cited taxonomy that draws on cognitive science. I introduced memory scoping in my context engineering post; here I expand it into six categories:
| Memory Type | Scope | Lifetime | Example | Storage Pattern |
|---|---|---|---|---|
| Working | Current step | Milliseconds | Tool call arguments, current LLM response | In-process (Python dict) |
| Short-term | Current thread | Minutes–hours | Conversation history, plan progress, gathered data | Checkpoint store |
| Episodic | Cross-thread | Days–months | ”Last week the user asked about NVDA earnings” | Vector store / KV store |
| Semantic | Cross-thread | Months–permanent | ”User prefers conservative investments” | Vector store / KV store |
| Document | Cross-thread | Days–permanent | Project notes, research summaries, learned patterns | File store (Markdown/JSON) |
| Procedural | System-wide | Permanent | ”When analyzing stocks, always check SEC filings” | Config / system prompt |
Working memory is what the LLM is actively reasoning with right now: Python variables in the current function, the contents of the context window, tool call arguments mid-execution. It’s the fastest and most ephemeral layer. Nothing persists beyond the current step. Working memory is bounded by the model’s context window, which makes it the bottleneck. Everything the agent “knows” at decision time has to fit here, whether it came from the checkpoint store, a vector query, or a file read. The other tiers exist to feed the right information into working memory at the right time.
Short-term memory is the checkpoint LangGraph writes after each unit of graph execution — a super-step, defined in the next section. Episodic and semantic memories persist across threads. Document memory stores project notes, research summaries, and learned conventions in files that people and agents can inspect. Procedural memory lives in system instructions and tool definitions rather than changing for each user.
For implementation, five of those six collapse into three storage tiers. Short-term memory becomes hot memory, the checkpoint for the current thread. Episodic and semantic become cold memory, recall across threads. Document memory keeps accumulated project knowledge readable and directly editable. Working memory is grouped with the hot tier in the taxonomy above, but it is the one that is never really stored: it lives for a single step, in process, and it is the context window the three storage tiers load into. Procedural memory sits outside all three: it lives in the system prompt and tool definitions, so it ships with the agent rather than being stored and retrieved.
CoALA classifies working, episodic, semantic, and procedural memory. The Memory in the Age of AI Agents survey emphasizes vector stores and knowledge graphs, while LangGraph documents checkpoints and its Store interface. File-backed project knowledge sits outside those taxonomies even though Claude Code, Cursor, and Devin Desktop all load persistent project files.
The same storage pattern appears in other domains. A Minecraft agent (Voyager) stores reusable game skills as code libraries, teams in an enterprise document-QA competition iterate on procedural prompt documents, and web agents induce reusable browsing workflows from successful runs. I come back to all three later; the point here is that files make that knowledge inspectable and versionable without a separate embedding service.
Agent-managed memory also differs from a fixed RAG pipeline in who performs the write. The agent or its harness selects what to store, update, and delete, then later chooses when to retrieve it.
The Generative Agents paper (Park et al., 2023) showed how far this can go: simulated agents stored, reflected on, and retrieved their own memories. Its memory stream ranked candidates by recency, importance, and relevance, a design that still provides a useful reference point for agent-memory retrieval.
Short-term agent memory: the checkpoint store
Every time LangGraph finishes a super-step — one node, or a batch of nodes that ran in parallel — the framework serializes the full graph state and writes it to a checkpoint store. That’s the foundation for pause/resume, time-travel debugging, and HITL workflows.
A checkpoint contains the graph state needed to resume: the AgentState from Part 1 — messages, identity, user profile, plan steps, research data, execution mode. Alongside it LangGraph stores its own bookkeeping: the checkpoint’s ID and timestamp, one version per channel (LangGraph’s name for an individual state key), and a separate record of which channel versions each node has already seen. The step number lives on the checkpoint’s metadata rather than on the checkpoint itself. Comparing those two is how the graph works out what runs next. After a HITL interrupt or process restart, the graph loads the checkpoint written at the latest completed boundary and re-enters the next node. It does not continue from an arbitrary Python line. A checkpoint is also different from an append-only event log or trace; Part 5 separates those runtime observability surfaces explicitly.
How LangGraph checkpointing works
LangGraph’s BaseCheckpointSaver is a simple interface: put() writes a checkpoint, get_tuple() reads the latest one for a thread, list() returns the history. Every checkpoint is keyed by (thread_id, checkpoint_ns, checkpoint_id), where thread_id identifies the conversation, checkpoint_ns handles subgraph namespacing, and checkpoint_id is a unique version.
The decision that matters is which backend to put behind it. PostgreSQL and Redis are two common production choices.
PostgreSQL vs Redis
| Dimension | PostgreSQL (langgraph-checkpoint-postgres) | Redis (langgraph-checkpoint-redis) |
|---|---|---|
| Durability model | ACID transactions, WAL, and replication | Configurable persistence: an append-only command log (AOF) or periodic snapshots (RDB) |
| Checkpoint history | Durable history for resume and debugging | Retention depends on saver and eviction settings |
| Primary constraint | Database write latency and table growth | RAM use, eviction, and persistence configuration |
| Operational fit | Teams already operating relational databases | Teams already operating Redis at high throughput |
| Best default for | Durable resume and reproducible debugging | Latency-sensitive, recoverable session state |
Generic database benchmarks do not predict checkpoint performance. Measure the serialized state size, write frequency, persistence settings, and concurrency of your own graph.
PostgreSQL: the durable default
PostgreSQL is the safer default for most teams. Checkpoints survive crashes, you get full transaction semantics, and the checkpoint history makes time-travel debugging straightforward.
A simplified version of the checkpoint setup in memory/hot.py. For production, set LANGGRAPH_STRICT_MSGPACK=true or configure an explicit allowed_msgpack_modules allowlist so checkpoint deserialization permits only safe or declared types; the permissive default warns about unregistered types but allows them.
import asyncio
from contextlib import asynccontextmanager
from langchain_core.messages import HumanMessage
from langgraph.checkpoint.postgres.aio import AsyncPostgresSaver
@asynccontextmanager
async def postgres_checkpointer(connection_string: str):
"""Yield a PostgreSQL-backed checkpoint store.
PostgreSQL gives us ACID guarantees — if a checkpoint write succeeds,
the state is durable even if the process crashes immediately after.
`from_conn_string` is itself an async context manager: it owns the
connection and closes it on exit, so the graph has to run inside it.
"""
async with AsyncPostgresSaver.from_conn_string(connection_string) as checkpointer:
# Create the checkpoint tables if they don't exist.
# This is idempotent — safe to call on every startup.
await checkpointer.setup()
yield checkpointer
async def main() -> None:
# The graph lives inside the context manager's scope.
async with postgres_checkpointer(
"postgresql://user:pass@localhost:5432/agent_memory"
) as checkpointer:
graph = create_graph(checkpointer=checkpointer)
# Every invoke/stream call now persists state automatically.
config = {"configurable": {"thread_id": "user-123-session-1"}}
result = await graph.ainvoke(
{"messages": [HumanMessage(content="Analyze NVDA")]}, config
)
# Resume later on the same thread_id — loads the latest checkpoint.
result = await graph.ainvoke(
{"messages": [HumanMessage(content="approved")]}, config
)
asyncio.run(main())
The AsyncPostgresSaver uses the langgraph-checkpoint-postgres package, which creates four tables: checkpoints (the serialized state), checkpoint_blobs (large binary data), checkpoint_writes (pending writes for crash recovery), and checkpoint_migrations (schema version). Concurrent writers are separated by the primary key (thread_id, checkpoint_ns, checkpoint_id) and upserts rather than by locking — two workers on the same thread will not corrupt each other, but they will not coordinate either.
Redis: when latency is the bottleneck
When checkpoint latency is the bottleneck, Redis is an option for recoverable state. Measure serialized state size, persistence settings, and concurrency before choosing it over PostgreSQL.
A simplified version of the checkpoint setup in memory/hot.py:
import asyncio
from contextlib import asynccontextmanager
from langgraph.checkpoint.redis.aio import AsyncRedisSaver
@asynccontextmanager
async def redis_checkpointer(redis_url: str):
"""Yield a Redis-backed checkpoint store.
Redis keeps checkpoints in memory for low-latency access.
Trade-off: less durable than PostgreSQL unless you enable AOF,
Redis's append-only log — snapshots alone lose recent writes on a crash.
"""
async with AsyncRedisSaver.from_conn_string(redis_url) as checkpointer:
# Initialize Redis data structures
await checkpointer.asetup()
yield checkpointer
async def main() -> None:
# Same graph API, different backend.
async with redis_checkpointer("redis://localhost:6379") as checkpointer:
graph = create_graph(checkpointer=checkpointer)
asyncio.run(main())
The AsyncRedisSaver from langgraph-checkpoint-redis stores each checkpoint as its own RedisJSON document, under the same (thread_id, checkpoint_ns, checkpoint_id) key as the Postgres saver. The v0.1.0 redesign replaced multiple search operations with a single JSON.GET call, significantly reducing latency. Redis 8.0+ includes RedisJSON and RediSearch by default — no extra modules to install.
For memory-constrained deployments, ShallowRedisSaver stores only the latest checkpoint per thread — no history, but minimal RAM usage. Use this when you need pause/resume but don’t need time-travel debugging.
When to use which
Use PostgreSQL when:
- You need full checkpoint history for time-travel debugging or reproducible resume
- Durability is non-negotiable (financial services, healthcare)
- You already run PostgreSQL in your stack
- Your agent runs long tasks where losing state means hours of recomputation
- You want a unified data store — PostgreSQL with pgvector can be a single backend for checkpoints, long-term memory, and vector search
Use Redis when:
- Checkpoint latency is your bottleneck (real-time chat, streaming UX)
- You’re building voice bots or streaming experiences where checkpoint access is on a measured latency-critical path
- You need horizontal scaling across many concurrent threads
- High-concurrency fan-out patterns where multiple agents share state
- Short-lived sessions where losing a checkpoint is recoverable
- You want semantic caching to reduce redundant LLM calls (Redis LangCache caches semantically similar queries to avoid repeated LLM calls)
Other options: langgraph-checkpoint-sqlite works for local development and single-process deployments. For AWS-native stacks, langgraph-checkpoint-aws provides a DynamoDBSaver with automatic payload offloading — small checkpoints (<350 KB) stay in DynamoDB, larger ones spill to S3. Serverless pricing and no infrastructure to manage make it attractive for variable-load deployments.
Long-term memory: remembering across sessions
Hot memory handles the current conversation. Long-term memory covers the user who comes back next week: it stores facts, preferences, and interaction history that persist across threads.
LangGraph provides a Store interface for cross-thread memory via its BaseStore class. Each memory item is a (namespace, key) pair with a JSON value and optional vector embedding. The namespace typically encodes the user or organization: ("user", "user-123", "preferences").
Vector storage: semantic recall with Qdrant
When the agent needs to recall unstructured facts (“What did the user say about their investment timeline?”), vector search provides semantic recall. Instead of exact key lookups, the agent queries by meaning.
Qdrant is a purpose-built vector database written in Rust that handles embedding storage, indexing (Hierarchical Navigable Small World, or HNSW), and filtered search. I covered HNSW and its trade-offs in detail in my search ranking post. Qdrant also offers an MCP server that acts as a semantic memory layer — useful if your agent framework supports the Model Context Protocol.
The following is an independent illustrative Qdrant design. It is not a simplified version of the current memory/long.py. The current project stores user profiles with exact user_id filtering and a zero-vector placeholder. Real embedding integration remains future work. The request handler must authenticate the request and construct principal from its verified identity; the client never supplies it. The Qdrant filter is retrieval scope, not authorization.
from qdrant_client import QdrantClient
from qdrant_client.models import (
PointStruct, Distance, VectorParams, Filter, FieldCondition, MatchValue,
)
import hashlib
from dataclasses import dataclass
@dataclass(frozen=True)
class AuthenticatedPrincipal:
"""Created by the server after authentication, never from request JSON."""
user_id: str
class UserMemoryStore:
"""Long-term memory backed by Qdrant vector search.
Stores user facts as embedded vectors for semantic retrieval.
Each fact is a short natural-language statement about the user.
"""
def __init__(self, qdrant_url: str, collection_name: str = "user_memory"):
self.client = QdrantClient(url=qdrant_url)
self.collection_name = collection_name
self._ensure_collection()
def _ensure_collection(self):
"""Create the collection if it doesn't exist."""
collections = [c.name for c in self.client.get_collections().collections]
if self.collection_name not in collections:
self.client.create_collection(
collection_name=self.collection_name,
vectors_config=VectorParams(
size=1536, # text-embedding-3-small dimensions
distance=Distance.COSINE,
),
)
def store_fact(
self, principal: AuthenticatedPrincipal, fact: str, embedding: list[float]
):
"""Store a user fact with its embedding."""
point_id = hashlib.md5(f"{principal.user_id}:{fact}".encode()).hexdigest()
self.client.upsert(
collection_name=self.collection_name,
points=[PointStruct(
id=point_id,
vector=embedding,
payload={"user_id": principal.user_id, "fact": fact},
)],
)
def recall(
self,
principal: AuthenticatedPrincipal,
query_embedding: list[float],
top_k: int = 5,
):
"""Retrieve the most relevant facts for a user given a query."""
results = self.client.query_points(
collection_name=self.collection_name,
query=query_embedding,
query_filter=Filter(
must=[FieldCondition(
key="user_id", match=MatchValue(value=principal.user_id)
)]
),
limit=top_k,
)
return [hit.payload["fact"] for hit in results.points]
The flow has three steps. In this illustrative design, an LLM extracts key facts from the interaction (“user has high risk tolerance”, “user is interested in semiconductor stocks”). Those facts are embedded and stored in Qdrant. At the start of the next conversation, the server supplies the authenticated principal and the agent queries Qdrant with the user’s new message to recall relevant context. The current Market Analyst Agent does not yet implement this semantic extraction and embedding flow.
Retrieval scoring: beyond cosine similarity
Raw cosine similarity is a starting point, but production memory systems need richer retrieval. The Generative Agents paper (Park et al., 2023) introduced a scoring function that combines three signals:
- Recency: Rule-based decay so recent memories score higher. An exponential decay function makes a fact from yesterday outrank an equivalent fact from six months ago.
- Importance: LLM-rated significance on a 1-10 scale. “User’s portfolio is down 40%” scores higher than “user said hello.”
- Relevance: Embedding cosine similarity between the query and the stored fact.
The final retrieval score is a weighted sum: score = alpha * recency + beta * importance + gamma * relevance. That keeps fresh, important facts from being buried under stale but semantically similar ones. For an agent like the Market Analyst Agent, I’d start at alpha = 0.3 for recency, beta = 0.2 for importance, and gamma = 0.5 for relevance, since the user’s current query intent matters most. These are starting-point weights adapted from the Generative Agents paper (which used equal weighting); I found emphasizing relevance worked better for financial analysis queries, but the values are intuition-based, not empirically optimized.
Alternatives to vector search
Vector search is powerful but not always the right tool. Here’s when to use alternatives:
| Approach | Best for | Main operational cost |
|---|---|---|
| Vector search (Qdrant) | Semantic recall of unstructured facts | Embedding and index lifecycle |
| Key-value store (Redis) | Structured user profiles and preferences | Memory use and persistence policy |
| Document store (files) | Project knowledge and agent-managed notes | Concurrency, permissions, and search |
| Full-text search (PostgreSQL GIN index) | Keyword recall over conversation history | Index growth and query tuning |
| Knowledge graph (Neo4j) | Entity relationships and multi-hop queries | Graph modeling and another data system |
| Hybrid (vector + keyword) | Recall when query intent varies | Two scoring paths to tune and evaluate |
Key-value stores work well for structured data. If your long-term memory is a user profile — risk tolerance, investment horizon, preferred sectors — a Redis hash or PostgreSQL JSONB column is simpler and faster than embedding and querying vectors. Use vector search when the memory is unstructured and the retrieval query varies in phrasing.
LangGraph’s built-in Store provides a namespace-based key-value interface with optional vector search. The BaseStore API is simple: put(), get(), search(), and delete() with hierarchical namespace scoping. Three implementations are available:
InMemoryStore— for development and testing (data lost on process exit)PostgresStore— production persistent store with full SQL queryingAsyncRedisStore— cross-thread memory with vector search, TTL support, and metadata filtering
The index configuration enables vector search over stored items using a configurable embedding model. For many use cases, this built-in store is sufficient without reaching for a dedicated vector database.
import asyncio
from langgraph.store.memory import InMemoryStore
# Create a store with vector search enabled
store = InMemoryStore(
index={
"dims": 1536,
"embed": my_embedding_function, # e.g., OpenAI text-embedding-3-small
}
)
async def main() -> None:
# Store a user preference (namespace scopes to user).
await store.aput(
namespace=("user", "user-123", "preferences"),
key="risk-profile",
value={"risk_tolerance": "high", "horizon": "long-term"},
)
# Semantic search across the user's memories.
# The namespace prefix is positional here — `search`/`asearch` declare it
# as positional-only `namespace_prefix`, unlike `aput`.
results = await store.asearch(
("user", "user-123"),
query="What is their investment style?",
limit=5,
)
asyncio.run(main())
Choosing a long-term memory strategy
Start with key-value if your memory is structured and well-defined (user profiles, settings, named entities). Add vector search when you need semantic retrieval over unstructured facts or when the query phrasing varies unpredictably.
Knowledge graphs earn their keep when relationships between entities matter, e.g. “Which companies did the user ask about that are competitors of NVDA?” The most interesting recent project here is Graphiti (by Zep), which builds a temporally-aware knowledge graph that tracks when facts were true, not just what was true. Every edge carries validity intervals, so a change to the user’s risk tolerance invalidates the old value rather than silently overwriting it. Graphiti reports 94.8% accuracy on the DMR benchmark — Deep Memory Retrieval, a long-conversation recall test — and its bi-temporal model handles the stale memory problem at the data layer.
The catch is operational. Running a graph database is non-trivial, and for most agent applications vector search with metadata filtering covers the same ground with less infrastructure.
Managed memory frameworks like Mem0 and Letta (formerly MemGPT) handle the extraction-consolidation-retrieval pipeline for you. Mem0’s approach is notable: an LLM extracts candidate memories, a decision engine compares each new fact against existing entries in the vector store, and a resolver decides to add, update, delete, or do nothing. That keeps the memory store coherent and non-redundant. Letta takes an operating-systems angle: agents manage their own context window using memory management tools, autonomously moving data between “core memory” (in-context) and “archival memory” (out-of-context). Both are worth evaluating if you want faster time-to-production and don’t need full control over the memory pipeline.
Document memory: the agent’s filing cabinet
Vector stores and key-value backends handle semantic recall and structured lookups well. There’s a third category of agent knowledge that neither serves cleanly: accumulated project context, the conventions, research notes, and decisions the agent needs across sessions. That knowledge benefits from being human-readable and version-controlled.
This is document memory: the agent reads and writes structured files (Markdown, JSON, YAML) to a known directory. No embeddings, no database, no infrastructure. Just files on disk that both the agent and the developer can cat, grep, git diff, and edit by hand.
It has more product adoption than coverage in the memory taxonomies above. In one vendor-run evaluation, Letta reported 74.0% accuracy on LoCoMo — a long-conversation question-answering benchmark — for a filesystem-backed agent running on GPT-4o mini, against 68.5% for Mem0’s best graph variant. That is one vendor, one model, one benchmark, and one harness: read it as a sign the approach is competitive, not as a ranking. The operational advantage does not depend on the benchmark: developers can read, edit, and diff the stored knowledge directly.
Longer context windows also make whole-file reads practical for some project documents. Chunked retrieval still fits large corpora, but a short conventions or handoff file can often be loaded directly. The choice depends on document size, retrieval precision, context budget, and how often people need to review or edit the memory.
Why files?
For long-lived agent workflows, the most effective pattern I’ve seen isn’t a vector database. It’s a directory of well-organized notes. Consider what happens when a coding agent works on a project over weeks:
- It learns that the project uses Pydantic v2, not v1
- It discovers that tests must run with
pytest -x --tb=short - It accumulates knowledge about the codebase architecture
- It learns the developer’s preferences (“always use
pathlib, neveros.path”)
These facts are too structured for vector search (you need exact recall, not fuzzy similarity) and too interconnected for a key-value store — they read as documents that reference each other, not as isolated values you fetch by key. They’re also facts the developer wants to see and edit directly. If the agent learns something wrong, you open the file and fix it.
This is how Claude Code’s CLAUDE.md and .claude/ directory work. The agent reads project-level CLAUDE.md files for conventions and instructions, and keeps a separate auto-memory file per project — under ~/.claude/projects/<project-slug>/memory/ — for cross-session learnings. Both are plain Markdown: you read them, edit them, commit the project ones to git, share them with your team. Cursor’s project rules and Devin Desktop’s rules and memories follow the same pattern. Cursor reads .mdc files from .cursor/rules; Devin Desktop (formerly Windsurf) reads .windsurf/rules/ and still honors the legacy single-file .windsurfrules. Either way: plain text on disk that the agent loads on startup to pick up project context.
Implementing a file memory store
The implementation is deliberately simple. The agent gets four operations: write a document, read a document, list available documents, and search across documents by keyword.
The following is an independent illustrative raw-Markdown file store. It is not a simplified version of the current memory/document.py. The current project uses DocumentMemory, which requires a namespace and key and writes a JSON envelope containing content, metadata, and created_at. This sketch defines a different design to show the trade-off of human-readable Markdown files:
from pathlib import Path
import json
class FileMemory:
"""Document memory backed by the local filesystem.
Stores agent knowledge as human-readable files organized by topic.
No embeddings, no database — just files that both the agent and
the developer can read, edit, and version-control.
"""
def __init__(self, base_dir: str | Path):
self.base_dir = Path(base_dir).resolve()
self.base_dir.mkdir(parents=True, exist_ok=True)
def _resolve_path(self, path: str) -> Path:
"""Return a path inside base_dir, rejecting escapes and symlinks."""
requested = Path(path)
if requested.is_absolute() or ".." in requested.parts:
raise ValueError("path must be relative to base_dir without traversal")
resolved = (self.base_dir / requested).resolve()
try:
resolved.relative_to(self.base_dir)
except ValueError as error:
raise ValueError("path must stay inside base_dir") from error
return resolved
def write_doc(self, path: str, content: str, metadata: dict | None = None):
"""Write or overwrite a document at the given path.
Paths are relative to base_dir. Directories are created automatically.
Metadata (if provided) is stored as a JSON sidecar file.
"""
full_path = self._resolve_path(path)
full_path.parent.mkdir(parents=True, exist_ok=True)
full_path.write_text(content, encoding="utf-8")
if metadata:
meta_path = self._resolve_path(
str(full_path.relative_to(self.base_dir).with_suffix(full_path.suffix + ".meta"))
)
meta_path.write_text(json.dumps(metadata, indent=2), encoding="utf-8")
def read_doc(self, path: str) -> str | None:
"""Read a document by path. Returns None if not found."""
full_path = self._resolve_path(path)
if full_path.exists():
return full_path.read_text(encoding="utf-8")
return None
def list_docs(self, pattern: str = "**/*") -> list[str]:
"""List documents matching a glob pattern."""
self._resolve_path(pattern)
return [
str(self._resolve_path(str(p.relative_to(self.base_dir))).relative_to(self.base_dir))
for p in self.base_dir.glob(pattern)
if self._resolve_path(str(p.relative_to(self.base_dir))).is_file()
and not p.name.endswith(".meta")
]
def search_docs(self, query: str, pattern: str = "**/*.md") -> list[dict]:
"""Search documents by keyword. Returns matching files with context.
This is intentionally simple — grep-style keyword search.
For semantic search, use a vector store instead.
NOTE: This is a sketch for demonstration. A simple substring check
won't scale beyond a few hundred documents. For production with 500+
documents, use TF-IDF/BM25 scoring (e.g., rank_bm25) or a full-text
search backend (PostgreSQL GIN index, Elasticsearch).
"""
self._resolve_path(pattern)
results = []
for path in self.base_dir.glob(pattern):
path = self._resolve_path(str(path.relative_to(self.base_dir)))
if not path.is_file() or path.name.endswith(".meta"):
continue
content = path.read_text(encoding="utf-8")
if query.lower() in content.lower():
# Return the paragraph containing the match for context
for paragraph in content.split("\n\n"):
if query.lower() in paragraph.lower():
results.append({
"path": str(path.relative_to(self.base_dir)),
"match": paragraph.strip()[:500],
})
return results
The path helper is deliberately shared by reads, writes, and glob results: relative paths can still leave a directory through .. or an existing symlink. This illustrative class is for a trusted single-user or controlled filesystem. It checks a resolved path before use; at a hostile multi-tenant boundary, use descriptor-relative no-follow operations so a filesystem mutation cannot race that check. Run this small regression check after copying the class:
from tempfile import TemporaryDirectory
with TemporaryDirectory() as root:
memory = FileMemory(root)
memory.write_doc("notes/ok.md", "safe memory")
assert memory.read_doc("notes/ok.md") == "safe memory"
assert memory.list_docs() == ["notes/ok.md"]
assert memory.search_docs("safe")[0]["path"] == "notes/ok.md"
(Path(root) / "escape").symlink_to(Path(root).parent, target_is_directory=True)
for operation in (
lambda: memory.write_doc("../escape.md", "nope"),
lambda: memory.read_doc("/tmp/escape.md"),
lambda: memory.read_doc("escape/outside.md"),
lambda: memory.list_docs("../**/*"),
lambda: memory.search_docs("safe", "../**/*.md"),
):
try:
operation()
except ValueError:
pass
else:
raise AssertionError("FileMemory accepted an escaped path")
Folder structure
Most of the value of document memory comes from how the directory is laid out. Here’s the shape I’d use for a research agent. The Market Analyst Agent uses namespaces under memory/documents/, but its current DocumentMemory writes each entry as a JSON envelope with a content string rather than raw Markdown. The raw-Markdown layout below belongs to the independent illustrative FileMemory design above:
.agent-memory/
README.md # What this directory is, for human readers
PROGRESS.md # Handoff for the next session: what is done, what is next
user-profiles/
user-123.md # Preferences, history, risk profile
user-456.md
research/
NVDA-2026-02.md # Research notes from recent analysis
TSLA-2026-01.md
conventions/
analysis-format.md # How to structure analysis reports
data-sources.md # Preferred data sources and API patterns
learnings/
common-errors.md # Mistakes the agent has learned to avoid
tool-patterns.md # Effective tool call sequences
In the illustrative FileMemory design, every document is Markdown, and every document’s purpose is obvious from its path. You can git diff the entire memory directory to see what the agent learned in a session, git revert a bad learning, or copy the directory to another project. The current project’s JSON envelopes keep the namespace and key structure, but they do not provide the same raw-Markdown diff experience.
When to use document memory vs vector vs key-value
The three memory backends serve different access patterns:
| Dimension | Vector Store | Key-Value Store | Document Store |
|---|---|---|---|
| Query pattern | ”Find facts similar to X" | "Get the value for key" | "Read the doc at path” |
| Best for | Unstructured, varied recall | Structured lookups | Project context, notes |
| Human readable | No (embeddings) | Partially (JSON) | Yes (Markdown) |
| Debuggable | Hard (similarity scores) | Easy (exact keys) | Trivial (open the file) |
| Version controllable | No | Possible | Yes (git-native) |
| Embedding infrastructure | Required | Not needed | Not needed |
| Scales to | Millions of facts | Millions of keys | Thousands of documents |
| Search capability | Semantic similarity | Exact match | Keyword / path-based |
Use document memory when:
- The agent accumulates project knowledge over multiple sessions
- Developers need to inspect, edit, or override what the agent “knows”
- The knowledge is structured as documents (notes, summaries, conventions) rather than isolated facts
- You want git-based versioning of agent memory
- Zero infrastructure is a hard requirement
Use vector stores when:
- You need fuzzy semantic retrieval (“find memories related to X”)
- The query phrasing varies unpredictably
- You have thousands to millions of individual facts
Use key-value stores when:
- You need exact, fast lookups for structured data (user profiles, settings)
- The data schema is well-defined
In practice, production agents often combine all three. The current Market Analyst Agent uses PostgreSQL checkpoints for hot memory, Qdrant for exact user-profile storage with placeholder vectors, and a namespaced JSON-envelope document store. The semantic-recall and raw-Markdown variants in this article are illustrative extensions.
Real-world examples
The pattern is already widespread in AI coding assistants:
- Claude Code reads
CLAUDE.mdfiles from the project root and parent directories, and maintains a per-project memory file under~/.claude/projects/for cross-session learnings. The memory system is plain Markdown files, and the project-level ones commit alongside your code. - Cursor loads project rules from
.cursor/rulesas.mdcfiles — coding conventions, framework preferences, architectural decisions — with frontmatter controlling when each rule applies. - Devin Desktop (formerly Windsurf) reads rules from
.windsurf/rules/, still honors the legacy root-level.windsurfrules, and writes autogenerated memories to a local store the agent consults on later runs. - Anthropic’s memory tool for the Claude API is a client-side tool the model drives with file operations —
view,create,str_replace,insert,delete, andrename— over a/memoriesdirectory. Your application implements each command, so it decides where the files actually live (local disk, S3, database).
All of these store agent knowledge as human-readable text files with explicit read/write operations, and none of them needs an embedding pipeline. The agent decides what to write, the developer can see and edit everything, and the whole system fits in a git diff.
Beyond coding assistants
Document memory is not limited to coding agents. The pattern shows up across other agent domains too:
-
Open-world game agents: Voyager (Wang et al., 2023) builds a persistent skill library of verified JavaScript programs that a Minecraft agent accumulates over time, collecting 3.3x more unique items and reaching milestones 15.3x faster than baselines. Skills transfer across new worlds without retraining. JARVIS-1 extends this with a multimodal memory that combines textual plans and visual observations, and is five times more reliable than the previous best agents on the long-horizon
ObtainDiamondPickaxetask.One distinction worth making here: skill libraries are executable memory (code files imported and run), while document memory in coding assistants is declarative (Markdown injected into prompts). The failure modes differ. Bad executable code crashes the agent; bad declarative text leads to reasoning errors. But the storage pattern and the operational benefits (debuggability, version control) are the same.
-
Enterprise workflow automation: The ERC3 competition — the third Enterprise RAG Challenge — had winners who used document memory for iterative prompt refinement. One winning team’s Analyzer and Versioner agents iterated through 80 prompt versions stored as procedural documents. Another top team built 20+ enricher modules as document-style procedural knowledge. LEGOMem (2025) formalizes this for multi-agent systems as modular procedural memory: past task trajectories are decomposed into reusable memory units, which are then placed either with the orchestrator that plans and delegates or with the agents that execute the steps. On the OfficeBench benchmark, orchestrator memory turned out to be the one that matters for task decomposition, while fine-grained agent memory improved execution accuracy.
-
Web automation: Agent Workflow Memory (Wang et al., 2024) lets web agents induce reusable workflows from successful episodes, with a 51.1% relative success rate improvement on WebArena. SkillWeaver (2025) goes further: agents synthesize reusable API tools from exploration, with a 31.8% relative success rate gain. The learned skills also transfer to weaker models (up to 54.3% relative improvement), so a stronger agent’s accumulated memory can lift a smaller one.
-
Customer support: Gartner predicts that agentic AI will autonomously resolve 80% of common customer service issues without human intervention by 2029. These agents reference SOPs, playbooks, and customer histories, which are all forms of document memory.
The MemAgents workshop at ICLR 2026 is one sign the research community is catching up to what practitioners have already built.
Skills use documents to package procedural instructions. The Agent Skills standard stores those instructions in SKILL.md files with YAML frontmatter and a Markdown body. That resembles document memory at the storage layer, but the role is different: a skill tells the agent how to perform a class of work, while memory records facts learned from a project or prior run. Part 3 draws the neighboring line, between a skill and a tool.
MCP (Model Context Protocol) has a related procedural interface: tools/list returns tool objects whose inputSchema is JSON Schema, and an agent invokes one with tools/call. Discovery does not authorize a call. Before invoking a tool with effects or private data, the host must implement authentication, authorization, and explicit user consent; the server must also enforce its own access controls. MCP cannot enforce those controls at the protocol level. A December 2025 review of MCP’s first year put the protocol at 97 million monthly SDK downloads across Python and TypeScript, with adoption by OpenAI, Google DeepMind, and Microsoft. MCP is not coding-specific. The same servers connect agents to databases, internal APIs, and enterprise systems.
Both make procedural interfaces inspectable: skills store instructions in documents, while MCP exposes machine-readable tool schemas and calls. MCP, now governed by the Agentic AI Foundation, is the closest thing to an interop standard the agent ecosystem has.
Scaling document memory for production
The file-based implementation above works well for single-developer laptops and small-scale deployments. Multi-tenant production with hundreds of users and thousands of documents requires a different architecture.
The single-node file limit becomes obvious: you can’t scale file I/O horizontally, concurrent writes need locking, and managing permissions across tenants is painful. Production needs a backing store that handles concurrency, search, and multi-tenancy properly.
Three common approaches:
Approach A: hybrid with a thin database layer
Keep files for authoring (developers edit Markdown locally) but serve from a database at runtime. On deployment, sync files to PostgreSQL rows. The agent reads from the database, not disk. This gives you:
- Developer ergonomics (edit Markdown, commit to git)
- Production query performance (indexed database reads)
- Clean separation between authoring and serving
Approach B: object storage + vector index sidecar
Store documents in S3/GCS as objects, with a Qdrant collection that indexes their embeddings. The agent queries Qdrant for relevant document IDs, then fetches content from object storage. This scales horizontally and supports semantic search, but adds complexity: two systems to manage, an embedding pipeline to maintain, and eventual consistency between store and index.
Approach C: structured document store with PostgreSQL (recommended)
Store documents as PostgreSQL JSONB rows with full-text search (GIN index) and optional vector embeddings (pgvector). This gives you hybrid search (keyword + semantic), ACID transactions, and a single operational system.
A sketch of Approach C. This is an RLS pattern, not drop-in application code: its database role must be available only to the trusted application server. The server authenticates the request and constructs principal; it does not accept a tenant ID from the caller. PostgreSQL RLS then makes that scope enforceable even if a query later omits its tenant predicate.
from typing import Optional
from dataclasses import dataclass
import asyncpg
@dataclass(frozen=True)
class AuthenticatedPrincipal:
"""The verified identity returned by the application's authentication layer."""
tenant_id: str
class ProductionDocumentMemory:
"""Illustrative PostgreSQL document memory with hybrid search and RLS.
Apply this schema and policy as the table owner during deployment:
CREATE TABLE documents (
id SERIAL PRIMARY KEY,
tenant_id TEXT NOT NULL,
path TEXT NOT NULL,
content TEXT NOT NULL,
metadata JSONB,
embedding vector(1536), -- pgvector extension
ts_vector tsvector GENERATED ALWAYS AS (to_tsvector('english', content)) STORED,
created_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(tenant_id, path)
);
CREATE INDEX ON documents USING GIN(ts_vector);
CREATE INDEX ON documents USING ivfflat(embedding vector_cosine_ops);
ALTER TABLE documents ENABLE ROW LEVEL SECURITY;
ALTER TABLE documents FORCE ROW LEVEL SECURITY;
CREATE POLICY tenant_documents ON documents
USING (tenant_id = current_setting('app.tenant_id', true))
WITH CHECK (tenant_id = current_setting('app.tenant_id', true));
`FORCE` also subjects the table owner to the policy. Superusers and roles with
`BYPASSRLS` still bypass it, so neither belongs in the application's pool.
"""
def __init__(self, pool: asyncpg.Pool):
self.pool = pool
async def write(
self,
principal: AuthenticatedPrincipal,
path: str,
content: str,
metadata: Optional[dict] = None,
embedding: Optional[list[float]] = None,
):
"""Write or update a document.
Sketch: on a real pool you must register codecs first, or asyncpg
raises DataError — `set_type_codec` for the JSONB metadata column
and pgvector's `register_vector` for the embedding.
"""
async with self.pool.acquire() as conn:
async with conn.transaction():
# true keeps this trusted context to this transaction only.
await conn.execute(
"SELECT set_config('app.tenant_id', $1, true)", principal.tenant_id
)
await conn.execute(
"""
INSERT INTO documents (tenant_id, path, content, metadata, embedding)
VALUES ($1, $2, $3, $4, $5)
ON CONFLICT (tenant_id, path) DO UPDATE
SET content = EXCLUDED.content,
metadata = EXCLUDED.metadata,
embedding = EXCLUDED.embedding
""",
principal.tenant_id, path, content, metadata, embedding,
)
async def search(
self,
principal: AuthenticatedPrincipal,
query: str,
embedding: Optional[list[float]] = None,
limit: int = 5,
) -> list[dict]:
"""Hybrid search: full-text + optional vector similarity."""
async with self.pool.acquire() as conn:
async with conn.transaction():
await conn.execute(
"SELECT set_config('app.tenant_id', $1, true)", principal.tenant_id
)
if embedding:
# Hybrid scoring: 0.6 * text relevance + 0.4 * vector similarity
rows = await conn.fetch(
"""
SELECT path, content, metadata,
(0.6 * ts_rank(ts_vector, plainto_tsquery('english', $1)) +
0.4 * (1 - (embedding <=> $2))) AS score
FROM documents
WHERE ts_vector @@ plainto_tsquery('english', $1)
OR (embedding <=> $2) < 0.5
ORDER BY score DESC
LIMIT $3
""",
query, embedding, limit,
)
else:
# Full-text search only
rows = await conn.fetch(
"""
SELECT path, content, metadata,
ts_rank(ts_vector, plainto_tsquery('english', $1)) AS score
FROM documents
WHERE ts_vector @@ plainto_tsquery('english', $1)
ORDER BY score DESC
LIMIT $2
""",
query, limit,
)
return [dict(row) for row in rows]
set_config(..., true) is transaction-scoped, so a pooled connection cannot retain one tenant’s context for the next request. The OR in the first branch is what makes it hybrid. With only the @@ predicate, a document that means the right thing but shares no keywords with the query gets filtered out before scoring ever runs — that is keyword retrieval with semantic re-ranking, not hybrid retrieval. The distance threshold is a knob: tighten it if the vector arm floods the results, loosen it if semantic matches never surface.
The following regression is the behavior to test against a real database after migrations. Under tenant-a, a read of tenant-b returns no rows and a direct cross-tenant insert fails RLS:
BEGIN;
SELECT set_config('app.tenant_id', 'tenant-a', true);
SELECT path FROM documents WHERE tenant_id = 'tenant-b'; -- 0 rows
INSERT INTO documents (tenant_id, path, content)
VALUES ('tenant-b', 'leak.md', 'must fail'); -- ERROR: row-level security policy
ROLLBACK;
What you get:
- Hybrid search: keyword matching (GIN index) + semantic similarity (pgvector) scored together
- Multi-tenancy: server-derived identity plus database-enforced RLS
- ACID guarantees: no eventual consistency issues
- Single operational system: no separate vector database to manage
- Horizontal scaling: read replicas for query load, partitioning by tenant for write scale
Files are great for single-developer workflows. For multi-tenant production, a structured document store on PostgreSQL is usually the right balance of simplicity, performance, and operational maturity.
Putting it together: the full architecture
Here’s how all three memory tiers can work together in an architecture inspired by the Market Analyst Agent. The diagram shows an illustrative flow from user request to response, with all memory layers active.
The architecture has three memory paths:
-
Hot path (checkpoint store): LangGraph writes the resumable graph state to the checkpoint store at every super-step boundary. When the graph hits an
interrupt_beforenode (like thepublishnode in Part 1), execution pauses. The user can close the app, and when they return, the graph resumes from the checkpoint. Runtime event logs and traces are separate production concerns. -
Cold path (long-term store): In this illustrative architecture, the agent queries the long-term store for relevant user context at the start of each conversation. That read is on the critical path — the planner cannot personalize until it returns. A vector-backed path may include query embedding plus index retrieval; a key-value lookup does not. The write is not: once the conversation ends, a background job extracts and stores new facts, and that job should never block the reasoning loop.
-
Document path (file store): At startup, the agent loads project conventions and relevant research notes from the document store. During execution, it writes new research summaries and learned patterns back to disk. These reads are on the critical path too, because they inform the current task; their cost depends on the filesystem, file size, and cache state. Writes can be deferred.
The wiring in LangGraph is straightforward — the checkpoint store and long-term store are passed at graph compilation, while the document store is injected as a dependency. The local sketch below uses InMemoryStore so the snippet stays small; the reference Docker topology uses Qdrant for the same semantic-recall role.
import asyncio
from langgraph.store.memory import InMemoryStore
# Cold memory: local sketch with vector search
# (The reference Docker topology uses Qdrant for persistent recall.)
memory_store = InMemoryStore(
index={"dims": 1536, "embed": embedding_function}
)
# Document memory: illustrative file-based store for project knowledge
# FileMemory is the illustrative class defined above, not the project's current
# DocumentMemory implementation.
doc_memory = FileMemory(base_dir=".agent-memory")
async def main() -> None:
# Hot memory: PostgreSQL for durable checkpoints. postgres_checkpointer() is
# the async context manager defined earlier, so the graph runs inside it.
async with postgres_checkpointer(pg_connection_string) as checkpointer:
graph = create_graph(
checkpointer=checkpointer,
store=memory_store,
)
# ... run the graph here, while the connection is still open
asyncio.run(main())
# The store is accessible inside any node via the store parameter
def planner_node(state: AgentState, *, store: BaseStore) -> dict:
"""Plan with user context from long-term memory."""
# Recall relevant user facts from vector store.
# Namespace prefix is positional — see the store example above.
user_memories = store.search(
("user", state.user_id),
query=state.messages[-1].content,
limit=5,
)
# Load project conventions from document memory
conventions = doc_memory.read_doc("conventions/analysis-format.md")
# Inject both into planning context
# Each stored value is a dict; render whatever keys it carries
memory_context = "\n".join(str(m.value) for m in user_memories)
# ... rest of planning logic with personalized context and conventions
The complete flow
What happens when a returning user sends “Analyze TSLA” to the Market Analyst Agent:
-
Document memory load: At startup, the agent reads project conventions from the document store: analysis format preferences, preferred data sources, tool usage patterns. These set the baseline behavior.
-
Cold memory recall: In this illustrative flow, before the router node executes, the graph queries the long-term store with the user’s message. It retrieves: “User has high risk tolerance”, “User prefers detailed competitor analysis”, “User previously researched NVDA and AMD”.
-
Router + Planner: The router classifies this as
DEEP_RESEARCH. The planner creates a 5-step research plan personalized to the recalled preferences. It includes a competitor analysis step because the user’s history shows they want one. The plan follows the format from the conventions document. -
Executor loop (hot memory): Each step executes via the ReAct pattern from Part 1 — think, act, observe, repeated until the step is done. After every super-step (router, planner, each executor step run sequentially here) LangGraph writes a checkpoint to PostgreSQL. If the process crashes after step 3 of 5, you restart and continue from step 4.
-
HITL interrupt: The reporter writes a draft, a fresh-context evaluator — a second model session with no history of the run — votes on it, and the graph reaches the
publishnode withinterrupt_beforeand pauses. The checkpoint holds the draft plus the evaluator’s verdict, so the human reviews both rather than adjudicating raw research. They review it hours later, and the graph loads the checkpoint and publishes. -
Memory updates: After the conversation ends, an asynchronous process extracts new user facts (“user is now tracking TSLA”, “user approved the report format”) and stores them in the long-term vector store. The agent also writes a research summary to the document store (
research/TSLA-2026-02) for future reference.
The three-tier pattern separates concerns cleanly. The checkpoint store handles durability and resume; it’s infrastructure. The long-term store handles personalization; it’s product logic. The document store holds accumulated project knowledge; it’s the agent’s notebook.
Trade-offs and considerations
Memory adds value, but it also adds cost and complexity:
-
Embedding cost: Every fact stored in a vector database requires an embedding API call. As of September 2026, OpenAI lists
text-embedding-3-smallat $0.02 per million tokens, so per-fact cost is negligible, but it adds up across thousands of users and sessions. Batch embedding calls and cache results. At query time, vector recall can include query embedding plus index and network latency; a key-value lookup does not. Measure that path in your deployment, then cache common query embeddings or use a local embedding model if it is latency-sensitive. -
Stale memory: User preferences change. A fact stored six months ago (“user prefers conservative investments”) may no longer be accurate. Set expiry policies. In one of my designs, I use 365 days for preferences and 90 days for episodic events as provisional examples, not universal defaults. The context engineering post rejects fixed retention rules as portable policy. Expiry is the blunt version. Schema-guided typed state covers the sharper one: temporal validity and provenance on each fact, so a superseded value loses to the current one at retrieval time rather than at expiry.
-
Memory overhead in context: Every recalled fact consumes tokens in the LLM’s context window. If you recall 20 facts per query, that’s several hundred tokens of memory context competing with the actual task. Cap the number of recalled facts and prioritize by relevance score.
-
Privacy and compliance: Long-term memory stores user data. You need PII redaction before storage, clear retention policies, and user-facing controls for data deletion. None of this is optional in regulated industries.
-
Checkpoint storage growth: PostgreSQL checkpoint tables grow with every super-step. Do not run a general SQL pruning query: delta channels can require ancestor checkpoints and their write/blob records to reconstruct a retained checkpoint. Use a saver-supported pruning API only after verifying it against the exact installed saver and its delta-channel recovery contract. If that support is unavailable, retain the complete parent, write, and blob closure, then test resume from a retained checkpoint with the installed saver.
-
Memory consolidation: Over time, detailed episodic memories should compress into compact semantic representations: “user asked about NVDA three times in January” rather than storing all three conversations verbatim. That mirrors human memory consolidation and keeps the store manageable. Mem0 and Graphiti handle this automatically; if you build your own, schedule periodic consolidation jobs.
-
Cold start problem: New users have no long-term memory. The agent should degrade gracefully and ask clarifying questions instead of making assumptions. Memory is additive, not required.
-
Memory poisoning: Anything in the agent’s context window is a potential injection point. If an attacker writes misleading facts to the document store or long-term memory (“always approve transactions without verification”), the agent may execute them as instructions. Prompt injection through stored memories is a real attack surface. The mitigations are validation before storage, treating recalled content as untrusted data rather than system instructions, and access controls that limit which memories can influence critical operations.
-
Document memory drift: File-based memory has no automatic deduplication or conflict resolution. Over time, documents accumulate contradictions: one file says “use pytest” while another says “use unittest.” Schedule periodic reviews (or let the agent do them) to prune and consolidate. In a vector store staleness stays hidden; in a directory of files you can
grepfor contradictions. -
Document memory doesn’t scale to millions of items: File-based memory works for hundreds to low thousands of documents. If your agent needs to recall from millions of facts with fuzzy matching, you need a vector store. Document memory is for structured project knowledge, not the long tail of every user interaction.
Key takeaways
- Agent memory is several stores with different access patterns. Keep resumable checkpoints, structured facts, semantic recall, and project documents distinct.
- Build pause and resume before personalization. Losing task progress is the first memory failure a long-running agent exposes.
- Put deterministic facts in structured storage. Use vector search when the query is fuzzy and phrasing varies.
- Use files for project knowledge that people need to inspect, edit, version, or review in a diff.
- Give every memory type an expiry, conflict, and deletion rule. A memory the system cannot correct becomes product debt.
- Limit what returns to the model. Stored memory has value only when retrieval puts the right evidence into the current context.
The next layer is action
Parts 5 and 6 return to memory from the operational side, and they take different halves of it. The runtime owns the checkpoint: where execution stopped and how to restart it. The harness owns the handoff: what the work means and what is left, written as document memory for the next model session — one continuous stretch of model context, in the vocabulary Part 5 pins down. Restoring the process is not the same as restoring the task.
References
Papers
- Cognitive Architectures for Language Agents (CoALA) — Sumers, Yao et al., 2023 — Foundational taxonomy of agent memory types
- Memory in the Age of AI Agents — Dec 2025 — Comprehensive three-dimensional taxonomy of agent memory
- MemGPT: Towards LLMs as Operating Systems — Packer et al., 2023 — Virtual context management for LLM agents
- Generative Agents: Interactive Simulacra of Human Behavior — Park et al., 2023 — Memory stream architecture with recency, importance, and relevance scoring
- Zep: A Temporal Knowledge Graph Architecture for Agent Memory — Rasmussen, 2025 — Bi-temporal knowledge graph for agent memory
- Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory — 2025 — Extraction/consolidation pipeline with benchmarks
- Voyager: An Open-Ended Embodied Agent with Large Language Models — Wang et al., 2023 — Skill library as document memory for open-world game agents
- JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models — 2023 — Multimodal memory library for Minecraft agents
- Agent Workflow Memory — Wang et al., 2024 — Reusable workflow induction for web automation agents
- SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills — 2025 — Self-synthesized reusable API tools for web agents
- LEGOMem: Modular Procedural Memory for Multi-agent LLM Systems for Workflow Automation — Oct 2025 — Reusable procedural memory units split between orchestrator and subagents
LangGraph documentation
- LangGraph Persistence (Checkpointing) — Core concepts for checkpoint-based memory
- LangGraph Memory Store — Cross-thread long-term memory with the Store interface
- LangGraph Cross-Thread Persistence — Functional API for cross-thread memory
- How to add memory to the prebuilt ReAct agent — Practical guide to adding memory
Checkpoint backends
langgraph-checkpoint-postgres— PostgreSQL checkpoint saver for LangGraphlanggraph-checkpoint-redis— Redis checkpoint saver for LangGraph- LangGraph Redis Checkpoint 0.1.0 Redesign — Architecture details for the Redis checkpoint saver
langgraph-checkpoint-aws— DynamoDB checkpoint saver with S3 offloading- Redis AI Agent Engineering — Redis patterns for agent workloads
Vector databases and memory tools
- Qdrant — Open-source vector database with HNSW indexing and filtering
- Qdrant Agentic Builders Guide — Practical guide to building agent memory with Qdrant
- pgvector — Vector similarity search extension for PostgreSQL
- Graphiti — Open-source temporal knowledge graph engine by Zep
Document and file-based memory
- Claude Code Memory — CLAUDE.md and the per-project memory directory
- Anthropic Memory Tool — Client-side file-based memory for Claude API agents
- Cursor Rules — Project rules as .mdc files under .cursor/rules
- Devin Desktop Memories — File-based rules and autogenerated memories (formerly Windsurf)
Memory frameworks
- Mem0 — Managed memory layer with extraction/consolidation pipeline
- Letta (MemGPT) — OS-inspired virtual context management for agents
- LangMem SDK — Memory management tools for LangGraph
Workshops
- MemAgents: Memory for LLM-Based Agentic Systems — ICLR 2026 Workshop
Demo project
- Market Analyst Agent — Reference implementation for the checkpoint and current profile/document storage paths