IWE Uses LOCOMO Benchmark to Validate Local-First AI Memory

IWE offers a local-first, markdown-based knowledge graph for AI agent memory, validated by the LOCOMO benchmark. It contrasts with vector databases, emphasizing human-editable, greppable data.

The Answer Up Front

IWE is for developers building AI agents who prioritize data ownership, local control, and human-readable memory. It offers a compelling alternative to hosted vector databases for agent context. Teams needing a fully managed, hosted memory solution with minimal setup should look elsewhere. The bottom line is that IWE demonstrates a viable, high-performance approach to agent memory using plain text and graph structures, achieving retrieval accuracy comparable to optimized systems while keeping data local and editable.

Methodology

This v0 review draws exclusively on the founder's published claims at https://dev.to/gimalay/the-benchmark-that-built-the-tools-d14, accessed on July 13, 2026. Independent benchmarks and long-term workflow evaluations are pending. We will re-test when claims diverge from observed behavior or when new versions are released. This review covers the founder's description of IWE (no specific version number is provided), its core architectural premise, and the results of its internal LOCOMO benchmark testing.

The core of IWE's validation is the LOCOMO (Maharana et al., 2024) benchmark, a standard instrument in agent-memory system evaluations. The corpus consists of ten very long fictional conversations, each spanning months and split into 19 to 35 sessions. For development, IWE iterated on two conversations: Caroline and Melanie, and Jon and Gina. The task involves ingesting an entire conversation chronologically, then answering 80 to 200 questions without additional context. Questions are categorized into four scored flavors: Single-hop (one fact from one session), Multi-hop (synthesis across sessions or speakers), and Temporal (date-specific questions). This methodology aims to produce numbers comparable to published agent-memory literature, including Mem0's.

What It Does

Local-first knowledge graph

IWE is a local-first knowledge tool designed to serve as an AI agent's memory. Its fundamental premise is that an agent's memories should be stored as plain markdown files within a local directory. These files are interconnected by links, forming a knowledge graph that is readable by any standard text editor and greppable by any command-line tool. This approach eliminates the need for embeddings, hosted pipelines, or per-retrieval API metering.

Agent memory and retrieval

Agents using IWE are designed to write their memories as linked markdown pages. Subsequently, they read these memories back using graph-aware tools. A key feature is the human-editability of these memories; users can open and correct agent-written content by hand directly in their preferred markdown editor. The system's development was driven by its own benchmark, which led to the reshaping of IWE's search engine and the motivation for a block-level editing language.

Performance and cost claims

The founder claims that IWE's store, curated by a specific model, reads back at 96% of a hand-built ceiling in a single retrieval call. This curation process reportedly costs $4.50. Despite this high performance, the founder notes that a baseline grep over raw transcripts remains competitive on accuracy alone, highlighting the challenge of surpassing simple text search for certain tasks.

What's Interesting / What's Not

The most interesting aspect of IWE is its explicit rejection of the prevailing vector database paradigm for AI agent memory. By opting for a local-first, plain markdown, graph-linked structure, IWE addresses growing concerns about data ownership, privacy, and the opaque nature of embedded knowledge. The commitment to human-readable and editable memories is a significant differentiator, offering a direct path for developers to inspect and correct agent behavior without complex tooling or proprietary formats. The narrative of benchmark-driven development, where the LOCOMO benchmark repeatedly exposed flaws and guided architectural decisions, lends credibility to the claimed performance.

What's less clear are the specifics of the

The investor read

IWE signals a potential shift in AI agent memory infrastructure, moving away from centralized, vector-embedding-based solutions towards local-first, human-readable knowledge graphs. This trend could be driven by increasing costs of hosted vector databases, privacy concerns, and a desire for greater control over agent data. While tools like Mem0 offer managed memory layers, IWE's approach targets developers who prioritize self-hosting and direct data access. For investors, IWE could be an interesting play if it establishes itself as a foundational open-source layer for local AI agents, or if its 'curator' model can be productized as a highly efficient, low-cost component. The challenge lies in monetizing a local-first tool that explicitly avoids hosted pipelines, potentially favoring a bootstrapped or open-source model over venture-scale SaaS.