AI agent memory: context, history, sources, and project rules
Why a large context window is not memory, and how to separate session history, long-term facts, sources, procedures, and deletion.
“The agent remembers everything” hides several different mechanisms. A model receives a limited context for the current request. An application may store session history, retrieve document passages, record durable facts about a user, and load project rules. Mixing these levels causes the system to forget important details, repeat stale ones, and disclose data to the wrong task.
A context window is a workbench, not an archive
The context contains system instructions, messages, tool results, files, and the response. Even a large window has a cost and a limit. More irrelevant text makes the required detail harder to retain, and an early caveat can disappear among logs. Trimming history and summarizing creates room, but a summary may itself discard an exception.
Context answers “what can the model see now?” Memory answers “what did the application store and what can it retrieve later?” A larger window does not replace selection, provenance, and retention.
Short-term memory holds one thread's state
The current goal, selected file, tools already called, and intermediate result belong to session or thread state. LangGraph describes threads and checkpoints as a way to preserve state between steps. That enables resuming after a pause, inspecting an earlier checkpoint, or recovering from a failure.
This state should not automatically move into another project or conversation. A temporary token, unfinished hypothesis, and draft file path are useful inside the current thread and hazardous as global memory.
Semantic, episodic, and procedural memory do different jobs
Semantic memory stores facts and preferences: a user's time zone, the project stack, or a chosen report format. Episodic memory stores useful experiences: how a similar failure appeared and which solution worked. Procedural memory defines how work is done: instructions, policies, verification order, and tool rules. The LangMem conceptual guide uses this distinction.
Every fact needs provenance, scope, and time. “The user prefers concise answers” may be global. “Do not change the schema in this release” belongs to a project and a period. An episode without an outcome turns memory into a noisy log; a procedure without a version keeps enforcing an obsolete process.
Sources and RAG provide evidence, not automatic truth
Retrieval finds relevant passages and adds them to the context. It helps with a knowledge base too large to send in full. Quality depends on chunking, metadata, access filters, ranking, and freshness. A passage with similar words may belong to another product or year.
Preserve a document link and cited passage beside a retrieved fact. Show the source for a critical answer. Deleting a document should delete or invalidate its embeddings and derived summaries; otherwise the agent continues to “remember” material that no longer exists.
AGENTS.md and project rules are reviewable team memory
For a coding agent, an explicit rules file is often more useful than hidden automatic memory. It can record build commands, directory boundaries, style, completion criteria, and prohibited actions. Codex best practices recommend providing a goal, context, constraints, and definition of done, while keeping durable instructions in AGENTS.md.
The file is versioned with the code, reviewed, and applied within a legible scope. A long set of contradictory rules still hurts performance. Keep root rules general and narrow instructions close to the relevant project area.
Writing memory should be a separate decision
Do not turn every message into a long-term fact. Classify a candidate first: is it durable, who owns it, how sensitive is it, and can it be verified? Ask for confirmation for user-profile facts: “Save this preference?” A support agent's customer fact belongs in a governed system of record, not an unrestricted vector index.
Updates require conflict resolution. A new job title may replace an old one, while a new address may need verification. Some facts should expire automatically. Procedures should change through a version and owner, not the most recent incidental prompt.
Forgetting, privacy, and tests complete the architecture
Users need a way to inspect, correct, and delete stored material. The system should explain scope: this session, this project, the account, or the organization. Sensitive data is encrypted, access is filtered before retrieval, and logs do not retain secrets indefinitely.
Test memory as a sequence: save a fact, change it, introduce a conflict, move to another project, delete the source, and ask again. Verify both recall and correct forgetting. Relevant products appear in AIDive's AI agent memory collection. Good memory is not the largest store; it is the smallest set of correct data returned in the right scope at the right time.