The database battle for the future of observability

A widely circulated blog post argues that ClickHouse is unseating Elasticsearch as the backend for logs and metrics, prompting a re-evaluation of the modern observability stack.

Where it happened

The conversation crystallized around a July 2026 blog post by engineer Mat Duggan titled "Clickhouse is winning the Observability Wars." The post gained significant traction on technical aggregators like Lobsters and Hacker News, serving as a focal point for a long-simmering debate among infrastructure engineers and SaaS founders about the underlying architecture of modern monitoring tools.

Side A: The challenger's architectural supremacy

This position, articulated by Duggan, argues that ClickHouse is fundamentally better suited for observability workloads than its predecessors. The core of the argument is architectural. ClickHouse is a columnar database, meaning it stores data by column rather than by row. This structure is exceptionally efficient for the analytical queries typical of observability, such as calculating averages or finding p99 latencies over vast time-series datasets. Proponents claim this leads to query speeds that are orders of magnitude faster than document-based databases like Elasticsearch.

Duggan points to major players like Grafana, Cloudflare, and Sentry adopting ClickHouse as evidence of a market-wide shift. He contends the performance gains and significant data compression, which lowers storage costs, create a "gravitational force" pulling the ecosystem in its direction. For this side, the incumbents are general-purpose tools being outcompeted by a specialized solution designed specifically for the high-volume, analytical nature of modern logs, metrics, and traces.

Side B: The incumbent's proven ecosystem

The counterargument rests on the maturity, flexibility, and vast ecosystem surrounding Elasticsearch. For over a decade, the ELK Stack (Elasticsearch, Logstash, Kibana) has been the de facto open-source solution for log management. This has produced a large pool of engineers with deep operational expertise and an extensive ecosystem of integrations, tutorials, and community support. For many teams, this operational stability is paramount.

Proponents of this view argue that Elasticsearch's strength in full-text search provides a versatility that pure columnar stores lack, which is valuable for debugging complex issues from unstructured logs. Furthermore, managed offerings from Elastic and major cloud providers abstract away much of the scaling complexity. The argument is that for many organizations, the known quantity of a mature, flexible, and well-supported platform represents a lower total cost of ownership than migrating to a new technology, even if that technology offers superior performance on specific benchmarks.

What's underneath

This is a classic debate between a specialized, best-in-class tool and a general-purpose, good-enough incumbent. The tension is not just about query speed but about the point at which a quantitative improvement in performance becomes a qualitative change in capability. The explosive growth in data volume from microservices and distributed systems has pushed the cost and performance of general-purpose tools past a breaking point for a growing segment of the market. This debate signals that observability is no longer a niche sub-problem of "search" but a mature infrastructure category with its own distinct technical and economic requirements. Both sides are reacting to the same pressure: observability data is becoming too voluminous and expensive to manage with yesterday's architecture.

The investor read

The growing adoption of ClickHouse signals a potential commoditization of the observability backend. This threatens the integrated, high-margin model of incumbents like Datadog and Splunk by enabling a modular, best-of-breed stack (e.g., open-source frontend, ClickHouse backend). This shift could lower customer switching costs and create opportunities for new companies building tooling on the emerging ClickHouse ecosystem. It suggests the market may be moving from all-in-one platforms toward a more disaggregated, cost-optimized model, putting pressure on incumbent pricing power.

Pull quote: “The explosive growth in data volume from microservices and distributed systems has pushed the cost and performance of general-purpose tools past a breaking point for a growing segment of the market.”