How to run an engineering health audit in your first week

A new engineering leader's first week is critical. Raleigh Schickel's framework uses ticketing data not for answers, but to find the right questions about product quality and team execution.

The first week in a new engineering leadership role is defined by pressure. Superiors need to see they made the right hire, and new reports are assessing whether their work is about to be upended. A structured framework can cut through this uncertainty.

Raleigh Schickel, in a post on Dev.to, outlines a diagnostic process for assessing an engineering organization. It centers on answering two questions: what is the state of product quality, and how healthy are the individual teams? The data from ticketing systems provides the starting point, but the answers come from the people.

Start with the tooling, not the data

Before calculating any metrics, the first step is a qualitative review of the organization's ticketing system. Schickel describes this as an archaeological survey. The structure of projects, the consistency of labels, and the history of custom dashboards reveal how the organization actually works, or has tried to work in the past.

An engineering organization with carefully structured tooling that reflects its team structure is different from one using a single monolithic project. The tooling shows the history of decisions made and abandoned. This initial, non-quantitative assessment provides context for any numbers that follow.

Collapse severity and priority

Schickel holds a strong opinion on bug classification: organizations should use either severity or priority, but not both. Using both creates a combinatorics problem. A bug classified as high severity but low priority, or vice versa, makes it harder for teams to reason about what to fix next.

The proposed solution is to collapse these into a single priority axis. A typical structure is urgent (P0), high (P1), medium (P2), and low (P3). This forces clarity and makes prioritization a more straightforward exercise for the entire team.

Track weekly bug trends by priority

The quantitative part of the quality assessment focuses on week-over-week trends. The key metrics are the total number of open bugs, the number opened in the last week, and the number resolved. This yields a net delta, showing whether the bug backlog is growing or shrinking.

This analysis is then broken down by priority. The most critical question, Schickel argues, is which priorities are actually getting worked on. It is a common organizational failure to close out numerous low-priority issues while urgent and high-priority bugs remain open for weeks. This single chart can reveal a deep misalignment between stated priorities and actual engineering effort.

What We'd Change

The framework as presented is incomplete. Schickel introduces team health as the second core pillar of the assessment but the source material does not detail the methodology for it. This leaves a founder with a playbook for auditing product quality debt but not execution or morale debt, which can be just as fatal.

This diagnostic is also tuned for an incoming leader at an established organization. A founder-CTO at a five-person startup will find weekly trend analysis of a P0-P3 backlog to be premature optimization. The principles of tracking quality and prioritizing fixes still apply, but the formal process requires significant scaling down for an early-stage context where the entire team sits in the same virtual room.

Finally, the framework's output is a set of questions, not a score. Its success depends entirely on the leader's ability to conduct qualitative interviews with the team. The data just tells the leader where to look and what to ask about. This is a feature, not a bug, but it means the framework is a tool for inquiry, not a self-diagnosing system.

Landing

Schickel's framework is a method for structuring the initial chaos of a new leadership role. It uses artifacts and data not to generate mandates, but to start informed conversations. By analyzing the tools, the classification systems, and the flow of work on defects, a leader can move from generic check-ins to specific, targeted questions about what is preventing the team from shipping a higher-quality product. The goal is not a perfect dashboard, but a more accurate map of the territory.

The investor read

An engineering leader implementing a structured audit like this is a positive signal for operational maturity. It indicates a shift from a pure 'build and ship' mentality to a focus on sustainable, scalable engineering practices. For investors, this is a de-risking event. It suggests the company is building the capacity to manage technical debt, maintain product quality, and scale its development team without collapsing under its own weight. A founder-CTO proactively adopting such a framework, even in a scaled-down form, demonstrates foresight about the challenges of growth beyond the initial product-market fit stage. This is a leading indicator of a team that can execute through scale.

Pull quote: “The data just tells the leader where to look and what to ask about.”