Temporal Worker Detects LLM Cost Spikes with Adaptive EWMA Baselines

This review examines a self-hosted Temporal worker designed for real-time LLM cost anomaly detection, leveraging an Exponentially Weighted Moving Average (EWMA) to identify unusual spend patterns.

The Answer Up Front

Teams operating LLMs at scale, particularly those already invested in a robust data infrastructure and Temporal, should consider this worker. It offers a sophisticated, adaptive approach to cost anomaly detection that static thresholds cannot match. If your organization is small, lacks a complex data stack (ClickHouse, Kafka, Redis, PostgreSQL), or is not using Temporal, the operational overhead will likely outweigh the benefits. The bottom line is a powerful, self-hostable component for granular, real-time LLM cost monitoring, but it demands significant infrastructure commitment.

Methodology

This v0 review draws on the founder's published claims at the specified Reddit thread and the accompanying GitHub repository. Independent benchmarks are pending. Update cadence: re-tested when claims diverge from observed behavior.

• Tool Name: Temporal EWMA Cost Anomaly Worker (part of a larger LLM observability platform)
• Version: chiefj/temporal-ewma-worker:stable (as observed on 2026-05-26)
• Source Signal URL: https://www.reddit.com/r/selfhosted/comments/1to38cr/i_got_tired_of_finding_out_about_llm_cost_spikes/
• GitHub Repository: https://github.com/Chief-Strategist-J/llm-observability-platform
• Covered in this review: The founder's architectural claims, the EWMA algorithm's application, the rationale for using Temporal, and the specified technology stack. Technical details from the linked GitHub repository provide the core artifact for this assessment.
• Not covered: Independent performance benchmarks under various loads, long-term operational costs beyond initial setup, specific integration challenges with diverse LLM providers, or production stability over extended periods. This review does not evaluate the broader “LLM observability platform” but focuses solely on the cost anomaly detection worker.

What It Does

The Temporal worker addresses the common problem of discovering LLM cost overruns only after receiving a billing invoice. It automates the detection of anomalous spend in real time, providing actionable alerts.

Real-time Anomaly Detection

The system operates on an hourly cron schedule. Each hour, it computes the current LLM spend for specific (service x model) pairs. If the actual cost for a given hour exceeds three times its established baseline, an alert is triggered. This alert is published to Kafka, providing immediate notification of a potential issue.

Adaptive Baselining with EWMA

Instead of static thresholds, which chief_jaydeep notes often lead to false positives or missed spikes, the worker uses an Exponentially Weighted Moving Average (EWMA). This EWMA is keyed by (service x model x hour_of_week). This granular keying allows the system to compare current spend against historical spend for the same service, model, and hour of the week in previous weeks. The founder reports an α value of 0.1, implying a history window of approximately four weeks for the baseline calculation.

Granular Drill-Down and Cold Start Solution

Upon detecting an anomaly, the worker queries ClickHouse to pinpoint the exact Kubernetes cluster responsible for the cost spike. The founder claims this provides specific context, such as "payments-service on cluster-prod-us-east-2 burned $400 in the last hour." To mitigate the "cold start problem" for new (service, model) pairs, where initial requests could erroneously trigger 3x anomalies, the EWMA is seeded with the global average cost for that model across all existing services, providing a reasonable prior.

Architectural Stack

The system is built on a robust, distributed stack. Temporal orchestrates the hourly cron workflow, ensuring reliability and retries. PostgreSQL stores active (service, model) pairs. Redis caches the EWMA baselines for sub-millisecond access, crucial for the hot path. ClickHouse serves as the analytical database for detailed cost drill-downs, and Kafka is used for publishing anomaly events.

What's Interesting / What's Not

The most compelling aspect of this worker is its intelligent application of EWMA with hour_of_week keying. This design directly tackles the inherent variability of LLM workloads, where normal spend can fluctuate wildly based on time of day and day of week. Static thresholds are notoriously brittle for such patterns, making this adaptive approach a significant improvement for practical, low-noise alerting. The founder's solution to the cold start problem, seeding with a global average, is also a pragmatic detail that enhances usability from day one.

The choice of Temporal for orchestration is a strong signal for reliability. For critical monitoring tasks, Temporal's guarantees around workflow execution, retries, and state management are well-suited. This ensures that cost anomaly detection itself is resilient to infrastructure failures. The granular drill-down capability to the Kubernetes cluster level is also a valuable feature, moving beyond generic alerts to actionable insights.

What's less interesting, or rather, a significant consideration, is the operational overhead. This is not a plug-and-play solution for teams without an existing, mature data infrastructure. The reliance on Temporal, ClickHouse, PostgreSQL, Redis, and Kafka means a substantial commitment to maintaining a complex distributed system. For smaller teams or those just beginning their LLM journey, the cost of operating this stack might overshadow the benefits of the anomaly detection itself. It is a powerful component, but it is a component that expects a sophisticated ecosystem around it.

Pricing

The Temporal EWMA Cost Anomaly Worker is open-source and self-hosted, implying no direct software licensing costs. Users incur costs for hosting the underlying infrastructure: Temporal, ClickHouse, PostgreSQL, Redis, Kafka, and the Python 3.11 runtime. Pricing snapshot date: 2026-05-26.

Verdict

For engineering teams running LLMs at scale and already managing a sophisticated data infrastructure, this Temporal worker is a strong pick. Its adaptive EWMA baselining, keyed by hour_of_week, provides a robust and intelligent solution to real-time LLM cost anomaly detection, significantly reducing false positives compared to static thresholds. The ability to drill down to specific Kubernetes clusters offers critical context for rapid incident response. Skip this if your team is small, lacks the operational expertise for a complex distributed stack, or is not already using Temporal. The value here is in its sophisticated approach to a specific, high-stakes problem for mature LLM operations.

What We'd Test Next

In a v2 review, we would establish a test environment to benchmark the system's performance under varying LLM traffic patterns and data volumes. Key areas for investigation include the actual latency from cost spike to alert notification, the false positive and false negative rates across different α values and threshold multipliers, and the scalability of the Redis EWMA store as the number of (service x model) pairs grows. We would also evaluate the ease of deployment and integration into diverse cloud environments, assessing the practical overhead for teams adopting this solution.

The investor read

The LLM cost management space is rapidly evolving, driven by the unpredictable nature of LLM usage and the high unit costs. This open-source Temporal worker signals a growing demand for sophisticated, real-time FinOps solutions tailored specifically for AI workloads. While many cloud cost management platforms exist, few offer the granular, adaptive anomaly detection specific to (service x model x hour_of_week) that this tool provides. Comparable tools might include FinOps platforms with custom alerting, but this worker's deep integration with a distributed stack (Temporal, ClickHouse, Kafka) and its focus on behavioral baselining is distinct. For investors, this project, if it were to productize, highlights the potential for a venture-backable company specializing in LLM observability and cost governance, potentially as a managed service on top of this open-source core. The founder's approach demonstrates a clear understanding of the operational challenges, suggesting a strong product-market fit for a specialized offering, rather than a generalist cloud cost tool.