Matt Park

Engineering Leadership

We shipped more code. We did not ship less risk.

By 6 minute read

Editorial illustration of AI-generated pull requests entering a verification gate, with verified modules passing through and rework looping back.

PR volume is the metric AI improved first. Rework is where the hidden cost appears next.

Teams can now produce more mergeable diffs than before. Dashboards celebrate deployment frequency and closed tickets. Meanwhile the same organisations quietly spend more time fixing, reverting and rewriting changes that looked finished on the way in. Throughput rose. Risk did not fall to match.

This is the operational sequel to Practical AI in engineering — judgment over volume and The review bar after AI. Leaders also need a signal after merge: when does “more shipped” become “more to clean up”?

Why throughput misleads after AI

Classic delivery dashboards were built for a world where writing code was expensive. Deployment frequency and lead time were useful proxies for flow. They still matter. They are incomplete when drafting is cheap.

AI compresses the cost of producing a change that looks intentional. It does not compress verification against a real system — checkout rules, promotions, accessibility, payment edge cases, inventory contention, third-party scripts. The gap between “looks done” and “understood” widens. Reviewers feel it first. On-call feels it next.

DORA’s 2025 research into AI-assisted software development makes the pattern explicit at organisational scale: higher AI adoption is associated with higher software delivery throughput and higher software delivery instability.

Treat that as an association, not a claim of simple causation. DORA’s interpretation is that faster change exposes weaknesses downstream when testing, architecture and feedback loops cannot absorb the added volume.

DORA’s later qualitative work names the hidden cost: a verification tax. Time saved writing is often re-spent auditing. Authors move faster; reviewers absorb cognitive load; production absorbs what neither caught. If leaders only praise merges, the organisation learns to optimise for appearance.

Name the signal: rework, churn and durability

You do not need a perfect academic definition to start. You need language the team can act on.

  • Deployment rework — unplanned deployments that exist to fix what a previous change broke or left incomplete: hotfixes, rollbacks, “follow-up” releases that should not have been necessary. DORA treats deployment rework rate as an instability signal alongside change fail rate.
  • Near-term churn — for this purpose, the proportion of newly added code materially changed or removed within 14 days of merge. Exclude planned experiments, generated files, lockfiles and routine dependency updates. It is a prompt to investigate avoidable rework, not a verdict on quality.
  • Durability — whether recent work still exists in a recognisable form weeks later, or whether the team is quietly living in a loop of generate → merge → rewrite.

Industry code-change research in 2026 points the same direction. GitClear’s maintainability analysis across hundreds of millions of changes reports rising duplication, collapsing refactoring, weaker cross-file reuse and declining long-term maintenance of older code — while throughput climbs. GitClear is a vendor analysis rather than a universal benchmark. The useful signal is the direction of travel; teams should establish their own repository baseline before setting thresholds. The headline is not “AI writes unusable code.” It is that default AI workflows favour atomic, ticket-shaped output while taxing the invisible work that keeps a codebase cheap to own.

What you measure, celebrate and staff becomes the system’s true review bar after the merge button.

Ecommerce and frontend make the cost visible

Storefront and platform work punish shallow confidence. A fluent product-grid refactor can preserve screenshots and unit tests while changing DOM order, focus behaviour or analytics events. A “small” checkout helper can touch payment, tax and inventory paths that only show under concurrent load.

In that environment, rework is not abstract. It is hotfixes on a Thursday evening, abandoned baskets attributed to “creative fatigue,” and peak-season incidents that started as an August merge nobody could explain.

What to stop optimising

Some metrics become theatre when drafting is cheap:

  • PR count and lines merged as proof of productivity.
  • AI acceptance rate as proof of value.
  • Deployment frequency without an instability counterpart.
  • “Green CI” as proof the change was understood.
  • Cycle time that improves only because review became rubber-stamping.

None of those numbers are worthless. They are incomplete. Treat them as context, not the scoreboard.

What leaders should watch instead

Measure these signals at team, service or value-stream level — not per developer. Once a diagnostic becomes an individual target, people optimise the record instead of the system: hotfixes go untagged, reviews accumulate performative comments and healthy experimentation looks like failure. Use the measures to decide where to investigate, not whom to judge.

Start with a small set you can explain in a leadership forum:

  • Deployment rework rate — share of deployments that are unplanned remediation.
  • Change fail rate — still essential; do not let rework replace it.
  • Near-term churn on critical paths — checkout, pricing, inventory, auth, payments: how often newly merged code is materially changed or removed within the 14-day window above.
  • Incident explainability — can the owning team narrate intent and blast radius without archaeology?
  • Review queue health — time to first substantive review and share of merges with zero meaningful review; AI volume that outruns review capacity is a system failure, not a heroics opportunity.

Pair the numbers with the review bar you already need: authors who can explain intent in their own words, reviewers who can independently judge risk, and leaders who refuse to confuse fluency with readiness.

A practical 30-day move

You do not need a new platform to start:

  • Baseline last month’s deployment rework and change fails on the services that touch revenue.
  • Tag hotfixes and reverts consistently so the signal is not buried in release notes.
  • Sample ten recent merges involving material AI assistance, using author disclosure where available. Review both the original change and its downstream trail: follow-up fixes, reversals, incident links and whether the original risk was understood.
  • Publish a one-page expectation: throughput is welcome; unexplained rework on critical paths is a stop-ship conversation.
  • Protect review capacity when assistant usage rises — smaller batches, risk-tiered scrutiny, no weekend hero merges that skip judgment.
  • Budget explicit time for consolidation and legacy repair, or accept that AI will grow the “perpetual V1” surface area.

Keep judgment ahead of volume

AI did not invent weak delivery metrics. It made weak metrics look like success. The response is not nostalgia for slower typing. It is a clearer scoreboard: enough evidence for the risk being taken, enough durability that last week’s merge is not this week’s archaeology, and enough honesty to admit when speed is buying instability.

Use assistants to get to a reviewable artefact faster. Keep humans responsible for deciding what deserves to ship — and for noticing when what shipped did not stick. That is how teams stay close to the craft when the diff already looks done.

Currency note: DORA and GitClear findings referenced here reflect 2025–2026 research available as of 6 September 2026.

Sources and further reading