How Much Memory Does Your Agent Actually Need?

In our previous analysis, we explored the mechanics of ALTK-Evolve in comparison to ACE, demonstrating that the method of delivering an agent’s self-distilled guidelines—whether through targeted retrieval or full-context injection—is a primary driver of both performance accuracy and operational cost. Today, we shift our focus to the fundamental question that precedes implementation: What is the optimal volume of memory to provide an agent?

The concept of "agentic memory" is intuitively appealing. The logic suggests that by distilling lessons from past performance and re-injecting them into the agent’s current context, we should see a linear improvement in capability. However, our latest research across eight distinct models—ranging from 30B dense architectures to massive, proprietary frontier systems—reveals a more nuanced reality. Agentic memory is not a binary feature to be toggled on; it is a precise dosage that must be calibrated to the specific capabilities of the underlying model.

The Core Insight: Capability Dictates Dosage

Our evaluation indicates that not every model derives the same benefit from a uniform memory strategy. Across the spectrum of tested models, we identified three distinct behavioral patterns:

• Strong Models with Headroom: These systems possess the cognitive capacity to synthesize vast amounts of information. They benefit most from receiving the full set of distilled guidelines, including rare edge-case scenarios. For instance, DeepSeek-V3.2 (671B MoE) saw a 9.5 percentage point increase in task completion when provided with its complete self-mined guideline library.
• Smaller or Weaker Models: These systems are easily overwhelmed by excessive context. For these models, a "less is more" approach is superior. A compact, high-confidence core of guidelines, supplemented by a few task-specific retrievals, yields the best results. The gpt-oss-120b (117B MoE) model, for example, achieved a 16.1 percentage point gain using this selective strategy, whereas the full guideline set resulted in lower performance at a 50% higher token cost.
• Saturated Models: Some models show no measurable improvement regardless of the memory configuration. Whether due to reaching a performance ceiling, an inability to apply guidance effectively, or a failure of the guidelines to address specific remaining bottlenecks, these models—such as GLM-5 (745B MoE)—remain static.

"The right dose of memory depends on the model, and we can calibrate it. Learning happens around the model, not inside it."

The Mechanics of Externalized Learning

It is important to clarify that "memory" in this context does not refer to the replay of historical transcripts. Instead, it represents a curated set of behavioral guidelines—strategies for success, common pitfalls to avoid, and edge-case resolutions—distilled from the agent’s own prior trajectories.

The ALTK-Evolve loop operates as follows: 1. Execution: The agent performs tasks, generating a series of trajectories. 2. Extraction: ALTK-Evolve analyzes both successful and failed runs to distill actionable behavioral guidelines. 3. Consolidation: These insights are aggregated into a reusable, structured library. 4. Inference: At runtime, the agent receives either the full library or a task-relevant subset.

Crucially, this process requires no weight updates and no human annotation. Because the learning loop modifies the guidance provided to the agent rather than the underlying model parameters, the system remains highly portable and cost-effective across diverse architectures.

Benchmarking Performance: The AppWorld Evaluation

To validate these findings, we utilized AppWorld, a rigorous benchmark comprising 585 multi-step tasks across nine simulated applications, including messaging, calendar management, and payment processing. We measured performance using two specific metrics:

• TGC (Task Goal Completion): The percentage of individual tasks completed fully and correctly.
• SGC (Scenario Goal Completion): A stricter, all-or-nothing metric that requires the agent to succeed on every variant of a scenario.

#### Comparative Performance Table

| Model | Pattern | Baseline TGC/SGC | Best-Memory TGC/SGC | Best Config | Δ TGC | Δ SGC | | :--- | :--- | :--- | :--- | :--- | :--- | :--- | | gpt-oss-120b | Weak/Selective | 39.9 / 21.4 | 56.0 / 37.5 | Curated | +16.1 | +16.1 | | DeepSeek-V3.2 | Strong/Headroom | 79.8 / 64.3 | 89.3 / 80.4 | Full Set | +9.5 | +16.1 | | Claude Opus 4.6 | Strong/Headroom | 90.5 / 87.5 | 94.6 / 94.6 | Full Set | +4.1 | +7.1 | | GPT-5.5 | Strong/Near-Ceiling | 92.3 / 82.1 | 95.2 / 89.3 | Full Set | +2.9 | +7.2 | | GLM-5 | Saturated | 87.5 / 80.4 | 87.5 / 80.4 | Full Set | 0.0 | 0.0 |

The data reveals that the stricter SGC metric often shows more significant gains than the headline TGC. This suggests that high-quality guidelines are particularly effective at helping agents navigate the complexities of scenario variants, rather than just the "average" case. Even for top-tier models like GPT-5.5 and Claude Opus, memory continues to provide value as long as there are remaining failure modes to address.

Efficiency and the Economics of Inference

A common concern regarding memory injection is the inflation of input tokens, as guidelines are typically re-sent at every turn of the ReAct loop. However, our analysis suggests that this cost can be effectively managed.

For the gpt-oss-120b model, utilizing the "curated retrieval" strategy resulted in a mere 5% increase in token usage while delivering a 16.1 percentage point gain in task completion. This represents the "best of both worlds"—significant performance improvements without the overhead of massive token consumption.

Furthermore, production environments can leverage prompt caching. Since the static portion of the guideline set remains identical across steps, it can be cached, drastically reducing the effective cost. Engineering for cache-aware prompt design—maintaining a stable prefix for the shared guideline set—is a highly recommended practice for any production-grade agentic system.

Key Takeaways for Developers

• Calibrate, Don't Accumulate: Do not blindly feed an agent every piece of information it has ever generated. Match the volume of memory to the model's capacity.
• Prioritize Curated Retrieval: For smaller models, a selective, high-confidence core is both more accurate and significantly cheaper than full-set injection.
• Leverage Prompt Caching: Use caching to mitigate the costs of injecting large guideline sets for stronger models.
• Focus on Failure Modes: Memory is most effective when it specifically targets the remaining weaknesses of a model that is otherwise near its performance ceiling.

Future Directions

While these results provide a clear roadmap for memory management, our work is ongoing. We are currently developing a learned selector to replace the current cosine-similarity-based retrieval, aiming for a system that better predicts which specific guidelines will assist in a given task.

We are also investigating teacher-distilled memory for extremely weak models where self-distillation may lack sufficient signal, and we are conducting broader evaluations beyond the AppWorld benchmark to ensure these findings hold in real-world, high-stakes deployments.

As we continue to refine the ALTK-Evolve library, our goal remains the same: to provide a transparent, efficient, and highly effective framework for agentic learning that works for every model, regardless of its size or architecture. We invite you to explore the full technical report and the ALTK-Evolve pipeline to begin calibrating your own agentic memory systems today.