Skip to content
← The Daily Dram
November 17, 2026

What is an Agent memory ? Why do we need them ?

rajeshsgr · 4 min read · The Daily Dram

THE DAILY DRAM

Small sips. Big ideas.

DRAM 000 · TOPIC · 4 MIN READ

LLMs are stateless, much like HTTP. Every request is independent, and the system doesn’t automatically remember what happened in the previous request.

In web applications, we solve this using cookies, sessions, and server-side storage to maintain state across requests.

AI agents face a similar challenge.

Every LLM inference call is essentially a fresh request. The model processes only the information provided in that call. It doesn’t inherently remember your previous conversations, yesterday’s decisions, or even the mistakes it made five minutes ago.

That’s where agent memory comes in.

Agent memory is an architectural capability that allows AI systems to retain, retrieve, and reuse information across interactions, enabling continuity and learning from past experiences without retraining the underlying model.

Just as sessions make stateless HTTP applications feel stateful, memory makes stateless LLMs capable of powering agents that remember and build on past interactions.

The Three Tiers of Agent Memory

Agent memory is commonly organized into three categories.

1. Working Memory — What’s happening right now?

This is the information available in the LLM’s current context window: conversation history, instructions, recent tool outputs, and intermediate results.

Working memory is immediate but limited by the model’s context window.

2. Episodic Memory — What happened before?

This captures specific past events, interactions, and outcomes.

Instead of repeating the entire investigation, it can retrieve that experience and consider it during troubleshooting.

3. Semantic Memory — What do I know?

This stores persistent facts, preferences, and accumulated knowledge.

Episodic and semantic memory are both forms of long-term memory.

How Does Agent Memory Actually Work?

Here’s the important part.

The LLM doesn’t automatically retrieve its own memories. The application or agent runtime manages that process.

When you send a new request, the application can assemble relevant information from several sources:

  • Conversation history: Previous messages from the current session, supplied directly as context.
  • Long-term memory: Relevant facts, preferences, or previous experiences retrieved from persistent storage.
  • RAG (Retrieval-Augmented Generation): Relevant information retrieved from documents, knowledge bases, or other external sources.
  • Tools and APIs: Current information fetched from enterprise systems.

All this information is assembled into the LLM’s context before inference.

The model then generates a response using the information available to it.

After the interaction, the system may identify useful new information and persist it for future use.

This creates a continuous lifecycle:

Retrieve → Build Context → Reason & Act → Store → Maintain

One important distinction: RAG is a retrieval technique, not memory itself. The same retrieval approach can be used to fetch historical memories or ordinary enterprise knowledge.

But What Happens When Conversations Get Too Long?

Why not send the entire conversation history with every request?

Because context windows have limits, and longer conversations increase cost, latency, and irrelevant information.

Architects use several techniques to manage this:

  • Sliding Window: Keep recent messages and discard older ones.
  • Summarization: Compress older conversations while preserving key information.
  • Selective Retrieval: Fetch only relevant history when needed.
  • Context Compaction: Reduce lengthy conversations and tool outputs into essential context.
  • Structured State: Store important facts, decisions, and task progress separately.

But there’s a catch. Summaries can lose details, retrieval can miss information, and outdated memories can lead to incorrect decisions.

That’s why managing context is just as important as storing memory.

The Architect’s Real Challenge

Building agent memory isn’t just about connecting an LLM to a vector database. Architects must make some critical decisions:

  • What to remember: Identify information worth retaining.
  • Where to store: Choose between databases, vector stores, and knowledge graphs.
  • When to retrieve: Bring in relevant memories without overloading context.
  • How to trust: Validate accuracy, freshness, and source of information.
  • How to secure: Enforce access controls, privacy, and retention policies.
  • How to measure: Evaluate whether memory improves task success and reduces errors.

For example, a support agent may remember how an incident was resolved last month, but it must verify current conditions before recommending the same fix.

The challenge isn’t just making agents remember. It’s ensuring they remember the right information and use it responsibly.

The Last Sip 🥃

LLMs are stateless. Agent memory is what allows applications built around them to maintain continuity, recall experiences, and reuse knowledge over time.

But memory isn’t about storing every conversation.

The real architectural challenge is knowing what to remember, what to retrieve, what to trust, and when to forget.


THE DAILY DRAM

Small sips. Big ideas.

One concept. Two minutes. A little wiser.

Enjoyed this Dram? Get the next one in your inbox.

Tags: , ,

Leave a Reply