Agent Memory

What Is Agent Memory?

Agent memory is information an agent can keep available so that later steps can use relevant context instead of starting from zero every time.

Why Do Agents Need Memory?

Without memory, an agent may repeatedly ask for the same information or lose progress between steps.

Working Memory

Working memory is information needed during the current agent run. It can include the goal, recent messages, tool results, and intermediate decisions.

Conversation Memory

Conversation memory is the message history or a useful summary of previous turns that helps the agent understand what the user has already said.

A practical system may keep recent messages and summarize older context when appropriate.

Long-Term Memory

Long-term memory stores selected information that may remain useful across future sessions.

Memory Is Not the Same as RAG

RAG retrieves relevant external knowledge. Memory retains selected information about the agent's interaction or state. A system can use both.

For example, an agent can retrieve a company's return policy with RAG while remembering an approved conversation preference separately.

A Simple Memory Model in Python

This example shows a small in-process memory store. It is a teaching model, not a production persistence layer.

memory = {
    "user_preferences": {},
    "current_task": {},
    "recent_results": []
}

memory["user_preferences"]["format"] = "concise"
memory["current_task"]["order_id"] = "1042"

memory["recent_results"].append({
    "tool": "get_order_status",
    "result": "shipped"
})

print(memory)

A production system would normally persist selected memory in an appropriate datastore and enforce access, retention, validation, and deletion rules.

Memory Retrieval

When an agent has a large memory store, it should not blindly send every stored item to the model. It should retrieve relevant memory.

Memory + Agent Decisions

Memory matters because it can influence the agent's next decision.

That does not mean memory should override the current request. Current, explicit instructions should be handled carefully when they conflict with stored memory.

What Should an Agent Remember?

What Should an Agent Not Remember?

More memory is not automatically better. Avoid retaining information without a clear purpose or appropriate authorization.

• Secrets such as passwords, API keys, or private credentials.
• Information that is unnecessary for the product's function.
• Unverified assumptions presented as facts.
• Temporary information that has no future value.
• Sensitive data when the application has no valid reason and controls to retain it.

Memory Freshness and Expiration

A stored fact can become wrong. Good memory systems consider how long information should remain valid.

Memory vs Conversation Summary

A conversation summary compresses dialogue. Long-term memory is a deliberate store of selected information. They can complement each other.

Memory + Tools

Tools can read or update memory, but memory tools should be treated like any other tool: validate inputs, enforce permissions, and control side effects.

A Safe Memory Update Pattern in Python

Keep the write operation narrow and explicit.

ALLOWED_KEYS = {"format", "language"}

def save_preference(memory, key, value):
    if key not in ALLOWED_KEYS:
        raise ValueError("Preference is not allowed")

    if not isinstance(value, str) or not value.strip():
        raise ValueError("Preference must be a non-empty string")

    memory["user_preferences"][key] = value.strip()

save_preference(memory, "format", "concise")
print(memory["user_preferences"])

In production, authorization, auditing, encryption, retention, and deletion requirements should be handled by the application architecture rather than trusted to the model.

Privacy and User Control

Memory changes how an AI system handles information over time, so users should have understandable controls and the application should have clear policies.

Memory in the Agent Loop

Memory is therefore not a separate “brain.” It is one part of the larger agent architecture alongside the model, tools, instructions, retrieval, and safety controls.

Common Memory Mistakes

1. Storing everything without a clear purpose.
2. Letting stale information override current facts.
3. Mixing users or sessions because memory scope is incorrect.
4. Allowing the model to bypass application authorization.
5. Sending an unnecessarily large memory set to the model.
6. Failing to provide appropriate deletion and retention behavior.

Mini Project: Add Memory to a Support Agent

Design a support agent that remembers a user's approved response-format preference and the current order ID during a support task.

Challenge: Decide which information should be session-only and which information, if any, should be retained across sessions.

Quick Quiz

30-Second Recap

Key Takeaways

When designing agent memory, start with the product requirement, define what should be retained, decide how long it remains valid, and enforce security and user-control requirements in the application.