How airCloset's CTO reshapes observability data for AI agents

To make production logs useful for AI, airCloset's CTO split observability into four distinct surfaces. Each surface has a data shape tailored to the questions an AI would ask.

At airCloset, a 46-repository codebase was successfully mapped for semantic search by an internal AI platform. But CTO Ryan Tsuji identified a critical gap: the map showed static connections, not what was happening in production at that moment. Raw observability data was too voluminous and noisy for an AI to interpret.

To solve this, Tsuji’s team applied the same data-shaping discipline used for static code analysis to their live production monitoring. The approach treats AI as the primary consumer of observability data, which requires structuring that data before it ever reaches a model’s context window.

The problem: AI cannot drink from a firehose

Feeding raw production logs directly to a large language model is ineffective. The sheer volume overwhelms the context window, and the model cannot distinguish meaningful error signals from routine operational noise. Standard logs, metrics, and traces are often disconnected, making it impossible for an AI to form a coherent picture of system health.

According to Tsuji, this is a data-shaping problem, not a data-volume problem. The core lesson was that data must be shaped before an AI can consume it. The internal AI platform at airCloset, codenamed "cortex," needed a structured input layer to answer questions about production status, resource allocation, or build failures.

Four monitoring surfaces, four data shapes

Instead of a single, unified observability pipeline, Tsuji’s team designed four distinct surfaces. Each target is shaped according to the primary question an AI agent is expected to answer for that domain.

1. Application: To answer "What's happening in production right now?", the system combines logs and traces. This shape is designed for open-ended exploration and incident diagnosis, allowing an agent to follow a request's path through the system.
2. Infrastructure: For questions like "Do we have enough resources?", the data is shaped as time-series metrics. This is the classic domain of tools like Prometheus, optimized for tracking resource utilization and availability over time.
3. Continuous Integration (CI): To determine "What broke and when?", the platform uses a combination of logs and alerts. This shape focuses on discrete events, such as a failed build or a security alert, providing a clear history for root-cause analysis.
4. LLM Usage: Answering "How much are we spending?" requires its own data shape. This involves tracking token counts, costs per user or team, and API latency, which are distinct from traditional application or infrastructure metrics.

What we'd change

This architecture is a custom in-house build, representing a significant engineering investment. For most companies, replicating airCloset's "cortex" platform is not feasible. The primary prerequisite is a team with deep expertise in both observability and applied AI, which is a rare combination. The value of this investment only materializes at a scale where AI agents are already integrated into core engineering workflows.

The playbook presented is also explicitly incomplete. The source article is the first of a two-part series, deferring critical topics like PII handling, integration with other systems, and the implementation of "Self-Healing" capabilities. Any team attempting to follow this model would need to solve these complex problems independently. The lack of detail on PII redaction is a major operational hurdle for anyone handling customer data.

Finally, the approach is tailored for a future that has not yet fully arrived for most organizations. While AI-driven operations are a compelling vision, the immediate return on building such a sophisticated data-shaping layer is questionable for teams not already bottlenecked by an AI's inability to parse production data.

Landing

The specific four-axis model used by airCloset is less important than the underlying principle. As engineering teams delegate more diagnostic and operational tasks to AI agents, the focus of observability must shift from human-readable dashboards to machine-consumable data structures. This requires treating the AI as the end-user and designing data pipelines accordingly. The work at airCloset provides a detailed schematic for what that future might look like, where observability is no longer just about monitoring, but about creating structured, AI-ready knowledge from live systems.

The investor read

This pattern signals the emergence of a new infrastructure layer: AI-native observability. While Datadog, New Relic, and Grafana serve human operators, airCloset's internal 'cortex' platform is built for AI agents. This suggests a market opportunity for startups focused on 'data shaping as a service' for AI. An investable company in this space would not just collect logs, but transform them into structured, queryable formats tailored for LLMs. The primary challenge is the market timing. The tactic is currently relevant for sophisticated tech companies operating AI at scale. Its broader applicability depends on how quickly smaller companies adopt AI agents for core operational tasks. This is a bet on the second-order effects of AI adoption, where the tooling to manage AI becomes as critical as the models themselves.

Pull quote: “The core lesson was that data must be shaped before an AI can consume it.”