paulserban.eu

Writing Edition

Paul Serban

Writing, snippets & book notes

Vol. 2026

February 03, 2025

DORA Metrics Explained: The Four Key Metrics Every Fullstack Engineer and Architect Should Master

A practical deep dive into Deployment Frequency, Lead Time for Changes, Change Failure Rate, and Time to Restore Service - and how to use them to build better systems and teams

Introduction

If you ask ten engineering leaders how to measure software delivery performance, you'll likely get ten different answers - deployment counts, story points completed, lines of code, number of bugs closed. Most of these metrics are either easy to game, disconnected from business outcomes, or actively harmful when used to evaluate individuals. The DORA metrics, developed through years of research by the DevOps Research and Assessment (DORA) team, offer something rarer: a small set of measures that correlate with organizational performance and that are hard to game without actually improving the underlying system.

For an engineer moving toward a lead or staff role, or an aspiring software architect, understanding DORA metrics is not optional trivia - it's a lens for reasoning about system design, deployment pipelines, and team structure simultaneously. These metrics sit at the intersection of engineering practice and organizational behavior, which is exactly the intersection that senior technical roles are expected to navigate. This article walks through the origins of the metrics, what each one actually measures, how to instrument them in a real system, where teams commonly misuse them, and how to think about them as an architect rather than just an individual contributor.

We'll move from the conceptual to the concrete: you'll see the research behind the metrics, a technical breakdown of each one, code examples for instrumentation, and a discussion of trade-offs that most introductory articles skip entirely.

Where DORA Metrics Come From

The DORA metrics emerged from a multi-year research program run by the DevOps Research and Assessment group, founded by Nicole Forsgren, Jez Humble, and Gene Kim. Their research, surveying tens of thousands of engineering professionals annually, culminated in the book Accelerate: The Science of Lean Software and DevOps (2018), which remains the primary citable source for the methodology behind these metrics. DORA was later acquired by Google Cloud, and the research continues today in the annual "State of DevOps" report published by Google Cloud.

The key insight from this research wasn't just "measure deployment speed." It was that four specific metrics - two focused on throughput (tempo) and two focused on stability (quality) - together predict organizational performance, including profitability, market share, and customer satisfaction, without requiring teams to trade speed for stability or vice versa. This was a genuinely important finding because prior industry wisdom often assumed a hard trade-off between moving fast and maintaining reliability. Elite performing teams, according to the research, achieve both simultaneously, largely because of architectural and process decisions that reduce the cost of change: small batch sizes, strong automated testing, trunk-based development, and loosely coupled architectures.

It's worth noting that DORA later introduced a fifth metric, Reliability, in more recent iterations of their research to better capture operational health beyond just recovery time. However, the original "four keys" remain the canonical starting point and the most widely referenced framework in the industry, so this article focuses on those four in depth while acknowledging the broader context they sit within.

The Four Key Metrics in Depth

Deployment Frequency

Deployment Frequency measures how often an organization successfully releases to production. This is not about how often code is merged, or how often a build passes CI - it's specifically about production releases. Elite performers, per DORA's benchmarks, deploy on demand, potentially multiple times per day. Low performers might deploy once every few months.

The reason this metric matters architecturally is that deployment frequency is a proxy for batch size. Small, frequent deployments mean smaller diffs between production states, which makes failures easier to isolate and roll back. A monolithic architecture with a lengthy manual QA process structurally caps how high this metric can go, regardless of team discipline. This is why deployment frequency is often the metric that surfaces architectural debt fastest - you cannot fake your way to high-frequency deployment without addressing coupling, test automation, and release tooling.

Lead Time for Changes

Lead Time for Changes measures the time from code commit to that code running successfully in production. This is distinct from "cycle time" in the agile sense (which often starts from when work begins) - DORA's definition starts the clock at commit, which makes it a purer signal of the technical pipeline's efficiency rather than upstream planning overhead.

Lead time is where CI/CD pipeline design, test suite performance, and deployment automation become directly visible. A team with excellent code review practices but a four-hour test suite and manual approval gates will have a lead time measured in days, not minutes. This metric tends to expose bottlenecks that are invisible in day-to-day work: a flaky integration test, a manual security sign-off step, or a shared staging environment that creates a queue.

Change Failure Rate

Change Failure Rate is the percentage of deployments to production that result in a degraded service requiring remediation - a hotfix, rollback, or incident. This is the metric most directly tied to quality, and it's the one most vulnerable to definitional disputes inside an organization. What counts as a "failure"? Does a minor UI bug count the same as an outage? Most mature organizations define this against their incident management taxonomy (e.g., only incidents above a certain severity threshold count).

What makes Change Failure Rate interesting from an architecture standpoint is that it doesn't merely measure code quality - it measures the effectiveness of your entire safety net: automated testing, canary deployments, feature flags, and progressive rollout strategies. Elite performers keep this rate in the 0–15% range, according to DORA's published benchmarks, while achieving high deployment frequency simultaneously, which is only possible with strong automated guardrails rather than manual gatekeeping.

Time to Restore Service

Time to Restore Service (also called Mean Time to Restore, or MTTR) measures how long it takes to recover from a production incident. This is arguably the metric most sensitive to observability infrastructure. You cannot restore what you cannot detect, and you cannot detect what you cannot measure. Teams with poor logging, tracing, or alerting will have long restore times even if their engineers are technically excellent, simply because the time-to-detect component dominates the total.

This metric also reveals organizational maturity around incident response: do you have clear on-call rotations, runbooks, and rollback mechanisms that don't require a senior engineer to be paged at 2 a.m.? Architecturally, this is where patterns like circuit breakers, feature flags for instant rollback, and blue-green deployments pay off - they convert what would be a code-revert-and-redeploy cycle (potentially lead-time-length) into a near-instant mitigation.

Instrumenting DORA Metrics in Practice

Talking about DORA metrics conceptually is one thing; actually computing them from your systems is where most teams stumble. Let's walk through a realistic instrumentation approach using data most organizations already have: Git history, CI/CD pipeline events, and incident tracking.

The first step is establishing reliable event sources. Deployment Frequency and Lead Time for Changes require you to know, unambiguously, when a deployment happened and which commit triggered it. Most CI/CD systems (GitHub Actions, GitLab CI, CircleCI, Buildkite) emit webhook events for deployment completion that can be captured. Below is a simplified TypeScript example of a service that ingests deployment webhook events and computes lead time by correlating deployment timestamps with the commit's authored timestamp:

// deploymentMetrics.ts
import { Octokit } from "@octokit/rest";

interface DeploymentEvent {
  commitSha: string;
  environment: string;
  deployedAt: Date;
  status: "success" | "failure";
}

interface LeadTimeResult {
  commitSha: string;
  leadTimeMs: number;
}

const octokit = new Octokit({ auth: process.env.GITHUB_TOKEN });

async function getCommitTimestamp(
  owner: string,
  repo: string,
  sha: string
): Promise<Date> {
  const { data } = await octokit.repos.getCommit({ owner, repo, ref: sha });
  // Use the author date, not committer date, to capture actual "work done" time
  return new Date(data.commit.author?.date ?? data.commit.committer!.date!);
}

export async function computeLeadTime(
  owner: string,
  repo: string,
  event: DeploymentEvent
): Promise<LeadTimeResult | null> {
  if (event.status !== "success" || event.environment !== "production") {
    return null;
  }

  const commitTimestamp = await getCommitTimestamp(owner, repo, event.commitSha);
  const leadTimeMs = event.deployedAt.getTime() - commitTimestamp.getTime();

  if (leadTimeMs < 0) {
    // Guards against clock skew or misattributed commits corrupting the dataset
    console.warn(`Negative lead time detected for ${event.commitSha}, skipping`);
    return null;
  }

  return { commitSha: event.commitSha, leadTimeMs };
}

Change Failure Rate and Time to Restore Service require linking deployments to incidents, which is a harder data modeling problem because incidents don't always map cleanly to a single deployment. A pragmatic approach is to tag incidents with the deployment(s) suspected of causing them during the incident postmortem process, rather than trying to infer causation automatically. Here's a Python example that aggregates this data over a rolling window, which is a pattern commonly used for internal engineering dashboards:

from dataclasses import dataclass
from datetime import datetime, timedelta
from typing import Optional

@dataclass
class Deployment:
    id: str
    service: str
    deployed_at: datetime
    caused_incident: bool

@dataclass
class Incident:
    id: str
    detected_at: datetime
    resolved_at: Optional[datetime]
    caused_by_deployment_id: Optional[str]

def change_failure_rate(deployments: list[Deployment], window_days: int = 30) -> float:
    cutoff = datetime.utcnow() - timedelta(days=window_days)
    recent = [d for d in deployments if d.deployed_at >= cutoff]
    if not recent:
        return 0.0
    failures = sum(1 for d in recent if d.caused_incident)
    return failures / len(recent)

def mean_time_to_restore(incidents: list[Incident], window_days: int = 30) -> Optional[float]:
    cutoff = datetime.utcnow() - timedelta(days=window_days)
    resolved = [
        i for i in incidents
        if i.resolved_at and i.detected_at >= cutoff
    ]
    if not resolved:
        return None
    durations = [
        (i.resolved_at - i.detected_at).total_seconds() / 3600  # hours
        for i in resolved
    ]
    return sum(durations) / len(durations)

These examples are intentionally simplified, but they reflect the actual mechanics teams use: correlate Git and CI metadata for tempo metrics, and correlate incident metadata for stability metrics. Tools like Google Cloud's own DORA-focused tooling, or open-source options like Four Keys (Google's reference implementation), automate much of this, but understanding the underlying data model matters even if you adopt a tool, because you'll need to reason about edge cases like hotfixes, rollbacks, and multi-service deployments.

Trade-offs, Pitfalls, and Common Misuses

The single most damaging mistake organizations make with DORA metrics is using them to evaluate individual engineers or teams competitively. DORA metrics were designed and validated as organizational-level indicators, not individual performance metrics. The moment a team knows its deployment frequency is being used in performance reviews, you introduce strong incentives to game the number - for instance, splitting a single logical change into many small deployments purely to inflate frequency, or reclassifying incidents to avoid counting against change failure rate. Goodhart's Law applies directly here: once a measure becomes a target, it ceases to be a good measure.

A second common pitfall is treating the four metrics as independent targets to optimize in isolation. In reality, they're meant to be read together. A team can trivially achieve a low Change Failure Rate by deploying rarely and conservatively - but that will tank their Deployment Frequency and likely inflate Lead Time as changes queue up. Conversely, a team chasing high Deployment Frequency without investing in automated testing will see Change Failure Rate climb. The research explicitly found that elite performers do well on all four simultaneously, which is the actual signal worth pursuing - not excellence on one metric at the expense of others.

There's also a data quality trap that's easy to underestimate: without clear, consistent definitions of "deployment," "incident," and "production," teams comparing their numbers across squads or over time are often comparing apples to oranges. One team might count a feature-flag toggle as a deployment; another might not. One team might only log incidents above a P1 severity; another logs everything. Before drawing conclusions from trends, it's worth auditing whether the underlying event definitions have stayed consistent - a sudden "improvement" in Change Failure Rate is just as likely to reflect a change in incident classification as an actual quality improvement.

Finally, DORA metrics say nothing directly about product outcomes. A team could hit elite benchmarks on all four metrics while shipping features nobody uses, or while quietly accumulating technical debt in areas the metrics don't touch, like code maintainability or security posture. They are necessary but not sufficient signals of engineering health.

Best Practices for Adopting DORA Metrics

Start by nailing down consistent definitions before you start measuring anything. Decide, in writing, what counts as a "production deployment" (does a canary release to 5% of traffic count? does a config-only change count?) and what severity threshold makes something an "incident" for Change Failure Rate purposes. This sounds bureaucratic, but skipping it guarantees that your metrics will be disputed and eventually ignored the first time someone doesn't like what the numbers say.

Instrument at the platform level, not the team level, wherever possible. If every team has to manually report their own deployment events, you'll get inconsistent, gamed, or simply missing data. Building DORA metric collection into your CI/CD platform and incident management tooling (PagerDuty, Opsgenie, or similar) as a byproduct of normal operations - rather than an extra reporting burden - produces far more trustworthy data.

Pair the metrics with qualitative context when presenting them to leadership or using them to drive investment decisions. A dip in Lead Time for Changes might be entirely explained by a large, necessary refactor, not a regression in team capability. Numbers without narrative invite misinterpretation, especially from stakeholders who aren't close to the day-to-day engineering reality.

Use the metrics to guide architectural investment, not to shame teams. If Change Failure Rate is high across the organization, that's a signal to invest in better automated testing infrastructure, canary deployment tooling, or feature flag systems - not to mandate more manual QA sign-offs, which will simply tank Deployment Frequency and Lead Time in exchange for marginal stability gains. As a future architect, this is precisely the kind of systemic reasoning you'll be expected to bring: seeing a metric trend and identifying the structural, not individual, cause.

Finally, revisit your benchmarks periodically against DORA's published research (the annual State of DevOps report) rather than against arbitrary internal targets. Benchmarks shift as the industry's tooling and practices mature, and comparing your team only against its own historical performance can create a false sense of progress if the entire industry has moved faster.

Key Takeaways

Here are five practical steps you can apply starting this week:

  1. Audit your current deployment pipeline and write down, precisely, what would count as a "deployment" for measurement purposes - this single step surfaces most definitional disagreements before they cause problems.
  2. Instrument Lead Time for Changes first, since it usually requires the least new tooling (Git commit timestamps plus CI/CD deployment events) and gives immediate visibility into pipeline bottlenecks.
  3. Tag incidents with their suspected causal deployment during postmortems, even informally, to start building the dataset needed for Change Failure Rate and MTTR.
  4. Never present these metrics as individual or team performance scores - frame them explicitly as system health indicators owned collectively by engineering leadership.
  5. Re-read the four metrics together monthly, not in isolation, and ask which architectural or process investment would move more than one metric simultaneously (automated testing and feature flags are usually the highest-leverage answers).

Mental Model: The Four Metrics as a Feedback Loop

A useful mental model is to think of the four DORA metrics as two paired feedback loops rather than four separate numbers. Deployment Frequency and Lead Time for Changes form the "how fast can we learn" loop - every deployment is an opportunity to get real-world feedback on whether a change works, so shrinking this loop means the organization learns faster. Change Failure Rate and Time to Restore Service form the "how safely can we learn" loop - they measure whether that fast learning comes at the cost of frequent, prolonged breakage.

Thinking of it this way clarifies why elite performers don't trade one loop for the other: a fast learning loop with a broken safety loop just means you break production quickly and often, which is not actually valuable. And a very safe but slow learning loop means you're shielded from failure but also shielded from useful feedback about what actually works for users. Good architecture - loose coupling, strong automated testing, observability, and safe deployment mechanisms like feature flags - is what allows both loops to run fast simultaneously, which is the entire point of the research.

Conclusion

DORA metrics endure as an industry reference point because they were derived empirically rather than invented intuitively - they measure what the research actually found correlates with organizational performance, not what sounds good in a retrospective. For an engineer growing into a lead or architect role, their real value isn't in the four numbers themselves but in the systemic thinking they force: every one of these metrics is downstream of architectural decisions about coupling, testability, deployment safety, and observability.

As you move into more senior roles, you'll increasingly be the person asked to explain why a metric moved, and more importantly, what to actually do about it. The teams and architects who get the most out of DORA metrics are the ones who resist the temptation to treat them as a scoreboard, and instead treat them as a diagnostic tool pointing toward specific, addressable weaknesses in the technical system. Used that way, the four keys become less about measuring your team and more about understanding it well enough to make it genuinely better.

References