Agent Memory System Design: Implementation Solutions for Short-Term and Long-Term Memory

This article explores the design of Agent memory systems, dividing them into two levels: short-term memory and long-term memory, and details their respective implementation solutions and considerations.

Framework and Core Points:

1. Overall Design Principle: Split the Agent's memory system into two layers:

   • Short-Term Memory: Serves the current session, controlling context length through technical means while maintaining semantic coherence.
   • Long-Term Memory: Serves cross-session scenarios, retrieving relevant memories from historical information on demand via a retrieval mechanism.

2. Two Main Solutions for Short-Term Memory:

   • Fixed Window Truncation: Keep only the most recent N rounds of conversation or tokens, discarding anything beyond. Advantages: simple implementation, low cost, stable length, suitable for casual chat or simple customer service scenarios; disadvantages: may lose early key information due to a "one-size-fits-all" approach, causing the Agent to "forget".
   • Rolling Summary: When the conversation history is about to exceed the window, summarize the early conversation content into a shorter summary to replace the original records. Advantages: compresses length while retaining high-value information such as task goals and style requirements, alleviating attention dilution caused by long contexts, better suited for long tasks like project planning and long-form creation; cost: requires additional model calls, and summary quality directly affects subsequent performance.

3. Construction Solution for Long-Term Memory: A general solution using vector databases to build a knowledge base.

   • Core Idea: Process past conversations into retrievable memory fragments, recalling them by relevance when needed.
   • Key Three-Step Process:
     • Storage: Vectorize the conversation and store it together with the original text in the long-term memory repository.
     • Retrieval: Perform similarity search based on the user's new question.
     • Combination: Input the most relevant historical fragments together with the current question to the model.
   • Advantages: Breaks the limitation of the context window, accurately extracting relevant information from massive history, forming the foundation for long-term interactive systems such as personalized assistants and enterprise knowledge bases.
   • Disadvantages: High system complexity, requiring the introduction of embedding models, vector databases, and a complete retrieval logic.

4. Important Considerations in Practice:

   • Memory Writing Criteria: Do not store all content by default; instead, set admission conditions for long-term memory, such as only writing long-term user preferences, core task goals, confirmed important facts, and reusable conclusions.
   • Memory Governance: Emphasize that memory is a dynamic data asset that requires regular cleaning, merging, updating, and fact verification, and provide users with management interfaces to ensure the stable operation of the long-term memory system.