SuperCompress claims 100% recall at 65% token savings with a CPU pre-filter

A new open-source prompt compressor uses a tiny CPU-based policy to evict irrelevant context before LLM inference. The creator's benchmarks claim perfect recall with significant cost savings.

The Answer Up Front

SuperCompress is for engineering teams building long-context LLM applications, especially RAG systems or agents, where both inference cost and answer quality are critical. Teams doing simple, one-shot prompts or those unable to add a pre-processing step to their inference pipeline should skip it. The bottom line is this: if the creator's benchmark claims of near-perfect recall at a 65% token reduction hold up under real-world conditions, SuperCompress represents a powerful, low-latency, and cost-effective alternative to naive truncation or more complex context management techniques.

Methodology

This is a v0 review based exclusively on the creator's technical blog post, published on dev.to. Independent benchmarks are pending. We will re-evaluate when we can test the tool against a standardized dataset or when public, reproducible benchmarks become available.

• Tool: SuperCompress
• Version: Unspecified
• Date Observed: June 26, 2026
• Source Signal: "How I Built a Prompt Compressor That Saves 65% on LLM Costs" (dev.to/arjunkshah/how-i-built-a-prompt-compressor-that-saves-65-on-llm-costs-3m80)

This review covers the tool's stated architecture, its claimed performance from the creator's own benchmarks, and its proposed mechanism of action. What is not covered is independently verified performance, behavior on out-of-distribution data, edge cases in production environments, or the practical overhead of integrating and running the CPU policy. All performance metrics cited here are claims made by the author.

What It Does

SuperCompress is designed to reduce the number of tokens sent to an LLM without sacrificing the information needed to generate a correct answer. It does this by acting as an intelligent pre-filter.

A CPU-first eviction policy

The core insight is that deciding which context is relevant doesn't require an expensive GPU. SuperCompress uses a very small neural network (claimed to be ~5,000 parameters) that runs on a CPU. This policy intercepts the full context and the user's query before they reach the main LLM. Its sole job is to decide what to keep and what to discard.

Line-by-line relevance scoring

The mechanism is straightforward. The policy examines each line of the provided context in relation to the user's question. It assigns a relevance score to every line. Any line scoring below a predetermined threshold is evicted from the context. The remaining, higher-relevance lines are then passed to the GPU-based LLM for processing. The author's example shows a 1,247-token context being reduced to 437 tokens.

Trained for recall and savings

The policy was reportedly trained on a dataset of long-form texts, user questions, and ground-truth relevance labels generated by an oracle LLM. The training process was designed to optimize for three factors simultaneously: maximizing the reduction in tokens, preserving all context lines necessary for a correct answer (recall), and keeping the CPU-side processing latency under 100ms.

What's Interesting / What's Not

The most interesting aspect of SuperCompress is its architectural simplicity and the corresponding performance claims. Using a tiny, specialized CPU model to gatekeep for a large, general-purpose GPU model is an elegant way to manage resources. It avoids the trap of using a second, smaller LLM for summarization, which can introduce its own latency and cost.

The headline claim of 100% oracle recall and 98% entity recall while discarding 65% of tokens is the key differentiator. The author provides a direct comparison to H2O, a known attention-based context management technique. SuperCompress claims to outperform H2O (98% oracle recall) while maintaining a similar latency profile (~60ms). If these numbers are reproducible, this is a significant improvement over the status quo.

What's missing is crucial context for verification. The blog post does not link to the open-source repository, the training dataset, or the specific benchmark used. The metrics themselves, "Oracle Recall" and "Entity Recall," are not formally defined, leaving their precise meaning open to interpretation. The robustness of the ~5K parameter model is also an open question. A policy trained on documentation and books might not perform as well on novel domains like legal text or multi-file source code repositories without retraining. These are the gaps that separate a compelling blog post from a production-ready tool.

Pricing

As of June 2026, SuperCompress is described as open source and available under an MIT license. This means it is free to use, modify, and distribute, including for commercial purposes.

Verdict

SuperCompress presents a compelling solution to the very real problem of expensive, long-context LLM inference. Its claimed ability to slash token counts by 65% with virtually no loss in answer quality makes it an immediate object of interest for any team operating RAG or agentic systems at scale. The CPU-first architecture is a smart, pragmatic design choice.

However, these impressive results are, for now, just claims from a single blog post. Without access to the code or a reproducible benchmark, they cannot be treated as verified facts. For engineering teams, SuperCompress is a high-potential tool that warrants immediate investigation. The first step is finding the repository and running it against your own data and workloads. It is a tool to test, not one to adopt based on the post alone.

What We'd Test Next

A v2 review would require hands-on testing. First, we would seek to reproduce the creator's benchmark claims on a standard long-context question-answering dataset, such as NarrativeQA. Second, we would measure the end-to-end latency impact on a typical inference stack. Third, we would test for robustness by evaluating the provided policy on out-of-domain content (for example, testing on legal documents if it was trained on technical manuals). Finally, we would investigate the process for tuning the relevance threshold to understand the trade-off between token savings and recall for different use cases.

The investor read

SuperCompress is a classic 'picks and shovels' play in the AI infrastructure market, targeting the significant operational cost of LLM inference. The market for cost-optimization tools is large and growing. As a standalone open-source project, it's a feature, not a product, and unlikely to be a venture-scale business on its own. Its value lies in its potential integration into a larger MLOps, observability, or inference platform. Companies like Anyscale, Databricks, or even model providers could see this as a valuable acquisition to improve the efficiency of their offerings. For the project to become investable, it would need to demonstrate a clear path to commercialization, likely through a managed cloud service, enterprise support contracts, or by becoming the core technology of a broader cost-management platform.

Pull quote: “If the creator's benchmark claims of near-perfect recall at a 65% token reduction hold up under real-world conditions, SuperCompress represents a powerful, low-latency, and cost-effective alternative.”