Agent Memory System Design: Implementation of Short-Term and Long-Term Memory

This article explores the design of an Agent memory system, dividing it into short-term and long-term memory layers, and details their respective implementation approaches and considerations.

Framework and Core Points:

1. Overall Design Principle: Split the Agent's memory system into two layers:

   • Short-term memory: Serves the current session, controlling context length through technical means while maintaining semantic coherence.
   • Long-term memory: Serves cross-session scenarios, retrieving relevant memories from historical information on demand via a retrieval mechanism.

2. Two Main Approaches for Short-Term Memory:

   • Fixed Window Truncation: Keep only the most recent N rounds of dialogue or tokens, discarding anything beyond. Advantages: simple implementation, low cost, stable length, suitable for casual chat or simple customer service; disadvantages: may lose early key information due to a "one-size-fits-all" approach, causing the Agent to "forget".
   • Rolling Summary: When the dialogue history is about to exceed the window, summarize the early dialogue content into a shorter summary to replace the original record. Advantages: compresses length while retaining high-value information such as task objectives and style requirements, and alleviates attention dilution caused by long contexts, making it more suitable for long tasks like project planning or long-form content creation; cost: requires additional model calls, and summary quality directly affects subsequent performance.

3. Construction Approach for Long-Term Memory: A general approach using vector databases to build a knowledge base.

   • Core Idea: Process past dialogues into retrievable memory fragments, recalled by relevance when needed.
   • Key Three-Step Process:
     • Storage: Vectorize the dialogue and store it together with the original text in the long-term memory store.
     • Retrieval: Perform similarity search based on the user's new question.
     • Combination: Input the most relevant historical fragments together with the current question to the model.
   • Advantages: Breaks the limitation of context windows, enabling precise extraction of relevant information from vast history, forming the basis for long-term interactive systems like personalized assistants and enterprise knowledge bases.
   • Disadvantages: High system complexity, requiring introduction of embedding models, vector databases, and a complete retrieval logic.

4. Important Considerations in Practice:

   • Memory Writing Guidelines: Do not store all content by default; instead, set admission conditions for long-term memory, e.g., only store long-term user preferences, core task objectives, confirmed important facts, and reusable conclusions.
   • Memory Governance: Emphasize that memory is a dynamic data asset requiring regular cleaning, merging, updating, and fact-checking, and provide user management interfaces to ensure stable operation of the long-term memory system.