A Redis and Edge Function Playbook for Capping AI User Costs

After two failed attempts, founder Sean McC built a low-latency architecture using Redis and Cloudflare Workers to prevent runaway user API costs. Here is the specific implementation.

An individual user running an AI feature in a loop can generate hundreds of dollars in API costs before the founder wakes up. OpenAI and Anthropic provide organization-level spending caps, but offer no native tools to limit costs on a per-user basis. This leaves a critical gap for SaaS products built on their models.

After experiencing this problem twice, founder Sean McC developed a specific architecture to solve it. The challenge was not just capping spend, but doing so without adding user-facing latency to every AI call.

The latency trap of database checks

McC’s first attempt involved a common approach: logging token counts to a Postgres database after each API call and checking the user's cumulative spend before the next one. This works functionally but fails on performance. McC reports that a database read before every AI call, on top of the AI call itself, in a serverless environment adds 200-400ms to every request. Users notice this delay immediately, degrading the product experience.

Redis for fast counters

The core of the working solution is replacing Postgres with Redis for the spend-tracking component. Redis is designed for low-latency operations and can safely handle concurrent requests from the same user without race conditions. A Redis read from a Cloudflare Worker in the same region, McC claims, typically completes in under 20ms.

The data structure is simple. A key tracks monthly spend per user, like spend:{user_id}:{YYYY-MM}. This key resets automatically each month as a new one is created, eliminating the need for cron jobs to clear old data. A second key, blocked:{user_id}, can flag users who have exceeded their limit.

Edge functions for the intercept

The check itself runs in a Cloudflare Worker, an edge function that executes geographically close to the application server. This placement is critical for minimizing latency. The logic in the worker is minimal for allowed requests. It reads the user's current spend from Redis, compares it to their plan's limit, and returns an allow or block decision. The path for an allowed user involves only the fast Redis read.

Deferring the write

To keep the user's request unblocked, the cost logging happens after the response has already been sent. In a Cloudflare Worker, this is accomplished with the waitUntil method. This post-response function performs the heavier tasks. It calculates the true cost from token counts, increments the user's spend in Redis using INCRBYFLOAT, and logs the full event to a primary database for long-term analytics.

What We'd Change

The Redis and edge function architecture is a robust solution for a product with demonstrated usage and runaway cost risk. It is not, however, a day-one requirement for a new AI product.

Implementing this stack introduces operational complexity and cost. Services like Upstash for Redis and Cloudflare Workers are not free at scale. For a product with fewer than 100 users, a slower, simpler check against a primary database might be an acceptable trade-off. The 200-400ms latency penalty, while not ideal, may be preferable to premature optimization.

This playbook is also a tactical response to a current limitation in major AI platforms. If OpenAI or Anthropic introduce reliable, fast, per-user cost controls, this custom architecture could become technical debt. Founders should view this as a necessary infrastructure patch for the current environment, not a permanent fixture of their stack. The principles, however, remain valuable for managing any metered resource.

Landing

McC’s architecture effectively trades a small degree of infrastructure complexity for financial predictability and a performant user experience. It addresses a direct and significant business risk for any company building on top of large language models. While the specific tools may change, the pattern of using a fast in-memory store for pre-flight checks and deferring heavier logging is a durable one for managing resource consumption at scale.

The investor read

This tactic highlights a key operational risk in the 'AI wrapper' SaaS category: unpredictable COGS tied directly to user behavior. Founders implementing this level of control demonstrate maturity, shifting from 'can we build it?' to 'can we run it profitably and safely?'. For an investor, seeing this architecture in a due diligence deck is a positive signal about the team's handle on unit economics and risk. It's becoming table stakes for any AI-based SaaS product seeking to scale beyond a small user base. Its absence is a red flag.

Pull quote: “McC reports that a database read before every AI call, on top of the AI call itself, in a serverless environment adds 200-400ms to every request.”