Most Heads of Data we talk to are tracking sprint velocity and treating that as the job done on measurement. Velocity says how many story points a team completed against what it estimated - a useful signal for whether a team's planning is calibrated, and almost entirely silent on the question that actually matters to the business: does anyone downstream trust and use what the team shipped? A team can hit its velocity target every sprint while stakeholders quietly route around its dashboards in favour of their own spreadsheets. Velocity wouldn't catch that. The right metric set would.

This is Part 5 of our series on incremental delivery for data and analytics teams. Software engineering has DORA - deployment frequency, lead time for changes, change failure rate, and time to restore - as its industry-standard measure of delivery performance. This post adapts that same framework for data work, then adds the data-specific signals DORA was never designed to capture: whether the data itself can be trusted, and whether the business actually uses it.

Why Sprint Velocity Isn't the Right Metric

Velocity measures estimation accuracy, not delivery performance, and the difference matters more than it first appears. A team can inflate velocity by padding estimates, by shipping trivial work that was never at risk of slipping, or by quietly deprioritising the hard, high-value work that resists confident estimation - all while the number on the sprint report keeps climbing. None of that shows up as a problem in a velocity chart. It shows up as stakeholders who've stopped trusting the dashboard, which is a completely different signal that velocity has no way to surface.

Four Data-Team Equivalents of DORA

DORA's four metrics split into two pairs: throughput (how fast value moves) and stability (what happens when something breaks). The same split translates cleanly to data work, once the unit of "deployment" is redefined as a data product reaching a real consumer rather than code reaching production.

The stability pair deserves particular attention for data teams specifically, because the underlying failure mode is different from software. A software bug usually fails loudly and immediately; a data quality issue often fails silently and late. Time to detect is frequently the larger number of the two, and the one worth investing in reducing first - a fast fix means little if the incident already ran undetected for three weeks and fed a board-level decision.

Data-Specific Metrics DORA Doesn't Cover

DORA was built for software delivery and doesn't have a native concept of data quality or business adoption. Two additional categories fill that gap.

The Metric Theater Trap

Every metric in this framework can be gamed, and the risk is worth naming directly rather than pretending a good scorecard is automatically a good team. Data delivery frequency rises if a team ships trivial, low-value pipeline tweaks to inflate the count - the number improves while the metric stops meaning anything. Data incident rate looks artificially clean if a team simply isn't monitoring closely enough to catch the incidents that are actually happening. And adoption numbers can be inflated by counting page loads instead of meaningful, repeated use.

A Sample Head-of-Data Dashboard

A workable starting dashboard doesn't need every metric above running simultaneously from day one. A realistic first version tracks data delivery frequency and cycle time weekly, data incident rate and time-to-detect monthly (since incidents are rarer and monthly gives a more stable signal), and adoption metrics on a rolling 30-day basis reviewed at the same cadence as sprint retrospectives - which Part 6 of this series covers in detail.

How to Start Without Overloading the Team

Introducing eight new metrics simultaneously is its own failure mode - the team spends more time instrumenting measurement than doing the work the measurement is supposed to track. Start with three: data delivery frequency, data incident rate, and one adoption metric specific to your highest-visibility data product. That combination already answers the two questions that matter most - is the team shipping, and does anyone trust what it ships - and it's a foundation the rest of the framework can be layered onto once the first three are running cleanly.

Numlytics' data strategy consulting team helps Heads of Data design and instrument a metric set that reflects real delivery performance, not just estimation accuracy - including the data quality and adoption signals most teams aren't tracking yet.