Agent-memory-bench exposes token cost scaling across multi-turn LLM memory strategies

Priyesh Dave's agent-memory-bench tool measures token consumption across full context, selective recall, and external memory across 30 reasoning steps on GPT-4o, highlighting long-tail cost bottlenecks.

For teams deploying multi-turn AI agents, agent-memory-bench provides a minimal harness to measure how memory architectures drive LLM token consumption. Use it if you are designing state management for long-running workflows and need to forecast API expenses before reaching production. Skip it if you are evaluating single-turn prompt quality or offloading context management entirely to managed platforms like LangChain or LlamaIndex. The bottom line: full context persistence scales token costs linearly per turn, while external vector-based recall caps token growth at a predictable flat baseline.

Testing methodology and limitations

This review draws on author Priyesh Dave's published claims and public code artifact at github.com/nradawg/agent-memory-bench (accessed October 2026). Independent benchmarks on Founderr Pulse hardware remain pending, and we will update this profile as claims diverge from observed production behavior.

The benchmark setup evaluates three memory strategies over a 30-turn stepwise reasoning chain: full context retention (persisting entire history), selective recall (sliding window context pruning), and external memory (vector DB retrieval). The tests used OpenAI's GPT-4o model (gpt-4o-2024-05-13) evaluated at June 2024 pricing rates of $5 per million input tokens and $15 per million output tokens.

Covered in this review: token growth curves, architectural trade-offs, and cost projections across 30 reasoning turns documented in the repository. Not covered: long-term database retrieval latency, vector index indexing overhead, accuracy decay during extreme context compression, or edge-case behavior across non-OpenAI provider APIs.

Quantifying token growth per turn

The agent-memory-bench Python package measures cumulative input and output token consumption at each turn of a multi-step agent workflow. Standard academic leaderboards evaluate task completion rates, but agent-memory-bench tracks context window expansion across repeated LLM calls.

In full context retention, context size grows linearly with every turn. By turn 30, the cumulative token count inflates total request volume, creating severe cost acceleration. For long-running agents, this naive context accumulation causes context window exhaustion or prohibitive billing spikes.

Comparing selective versus external recall

To mitigate token creep, the benchmark evaluates two structural alternatives to naive context accumulation. Selective recall implements sliding-window context pruning or summarization. This slows token inflation compared to full retention, but input prompt size still grows steadily as conversation history accumulates.

External memory detaches history from the active context window entirely, persisting conversation state in a vector index or search database. During each turn, the agent queries the index and injects only top-k retrieved snippets into the prompt. As demonstrated in the benchmark runs, external memory keeps input token volume practically flat across all 30 turns.

Engineering trade-offs and omitted variables

What makes agent-memory-bench valuable is its deliberate focus on infrastructure economics rather than prompt accuracy. Less than 10% of published agent benchmarks report token usage or context inflation. By treating token consumption as a first-class metric, the benchmark highlights why prototypes fail when transition to production.

However, the benchmark makes trade-offs that teams must account for. By focusing strictly on token counts, it omits the accuracy trade-offs inherent in context pruning. External memory keeps prompt costs low, but retrieval miss rates can degrade reasoning quality in complex multi-step tasks. Furthermore, the tool's published runs evaluate a single model (gpt-4o-2024-05-13) at fixed pricing tiers ($5/$15 per M tokens). It does not factor in vector database infrastructure costs, embedding generation expenses, or retrieval latency.

Pricing and availability

Agent-memory-bench is open-source under the MIT license and available via GitHub (nradawg/agent-memory-bench) and PyPI. As of October 2026, the benchmarking package is free to self-host. API execution costs depend on the underlying LLM provider; benchmark runs referenced here utilized OpenAI GPT-4o pricing of $5.00 per 1M input tokens and $15.00 per 1M output tokens.

Final verdict

Agent-memory-bench is an essential diagnostic tool for backend engineers building long-running AI workflows. If your agent requires more than 15 sequential turns, full context persistence is financially unviable. Adopting external retrieval fixes token expenditure per turn, keeping operational costs linear relative to execution steps rather than exponential. We recommend running agent-memory-bench during early architecture planning to set hard budget guardrails before shipping multi-turn workflows to production.

Future test matrix

In subsequent testing on our own evaluation rig, we will measure reasoning accuracy retention across all three memory strategies to quantify the precision loss of vector retrieval. We also plan to benchmark prompt caching performance on Anthropic Claude 3.5 Sonnet and DeepSeek v3, evaluating whether API-level context caching changes the economic calculus for full and selective context strategies.

The investor read

The findings from agent-memory-bench point to a broader architectural shift in AI infrastructure spend. As agent workflows extend from short chat turns to autonomous multi-step operations, unmanaged context growth turns LLM API bills into the primary unit economic blocker. This trend favors middleware vector databases, retrieval infrastructure, and intelligent context-compaction startups over pure prompt-engineering frameworks. Investors looking at agent tooling should watch companies that optimize context-window state and prompt caching. Startup teams that fail to implement external or selective context memory will face unsustainable gross margins as user engagement scales.

Pull quote: “Less than 10% of published agent benchmarks report token usage or context inflation.”