The question has moved
For two years the AI-coding question was whether the tools worked. That question is settled. Coding agents ship real changes, review real diffs, and pass real continuous integration (CI). The question that replaced it is harder: does any of that translate into an engineering organization delivering more value, or does it just make code generation cheaper while everything downstream of generation absorbs the difference?
Two datasets published in 2026 answer that question with more precision than the debate has had before, and they don’t agree with the optimistic version of the story.
| Signal | What it measures | Source |
|---|---|---|
| 28× | Increase in median quarterly AI spend in the technology sector, over 12 months ($1,500 → $44,000) | DX, State of AI Impact in Engineering, Q2 2026 |
| 57–58% | Share of engineering time going to new features, essentially unchanged over the same period | DX, Q2 2026 |
| 6% | Executives sure they have clear examples of organization-wide AI ROI | Atlassian, State of Teams, 2026 |
| 240% → 30% | Cumulative gain in coding activity from autonomous agents (240%), against a separately measured gain in releases (30%) | NBER Working Paper 35275, revised Sep 2026 |
Read together: spend on AI coding tools rose roughly 28-fold in a year. The fraction of engineering time spent building new things didn’t move. And the clearest available estimate of what autonomous agents do to raw coding activity, a 240% increase, shrinks to a 30% increase by the time it reaches an actual release. Something is absorbing the gap between what agents produce and what organizations ship.
AI has made software generation dramatically cheaper. It has not made the rest of the software-production system three times faster. That gap, not model quality, is now the central AI-engineering-ROI question.
Coding productivity is not engineering ROI
The confusion at the center of most AI-ROI conversations is treating these as the same measurement.
| Local productivity | Organizational ROI |
|---|---|
| Time to generate code | Time to production |
| PRs opened | Changes that ship and stay shipped |
| Lines generated | Delivered product capability |
| Developer-reported time saved | Capacity actually redeployed to new work |
A pull request (PR) produced in half the time but requiring twice the reconciliation is not twice as valuable. It may not be more valuable at all. This is not an argument that developer-productivity metrics are worthless; task completion time and AI-assisted work rate are useful leading indicators of adoption. They are just not sufficient to answer whether the organization is getting a return, and treating them as if they were is how a company ends up 28× deeper into AI spend with an unchanged feature-delivery rate and no way to explain why.
Where the value leaks
The National Bureau of Economic Research (NBER) attenuation, 240% down to 30%, is not evidence that AI coding doesn’t work. It is a map of where a system built for human-paced contribution starts to strain once contribution gets three times cheaper. Generation gets faster. Everything downstream, verification, integration, operational stability, was sized for the old rate.
Put it as a chain and the leak points are visible:
AI investment → generation gain → verification cost → rework cost → integration cost → operational cost → delivered value
Two of those stages carry a specific, architectural failure mode worth naming directly, because it is the one this site exists to address. An agent does not just generate code. It generates decisions: a dependency, an API boundary, a persistence choice, a framework, a migration path. When one of those decisions conflicts with something the organization already decided, the cost of reconciling it doesn’t disappear. It moves downstream, into review, into rework, sometimes into production, where it is considerably more expensive to find.
That is measurable. Faros AI’s 2026 telemetry, comparing each organization’s lowest and highest periods of AI adoption, found bugs per developer rising 54% and code churn rising 861% alongside the throughput gains. And it is exactly what the verification tax describes at the review stage specifically: time saved in generation reappearing as time spent checking whether the generated change was safe to merge.
The reconciliation case is worth stating precisely, because it is easy to conflate with ordinary bugs and it isn’t one. An agent asked to add a reporting endpoint chooses direct SQL because it is the fastest path to a working diff. The organization decided months earlier that all database access goes through repository abstractions. Nothing about the direct-SQL version is functionally wrong; it may pass every test written for it.
The cost shows up later, in a reviewer who has to notice the pattern, explain the constraint, and wait for a rewrite, or, if nobody notices, in the next engineer who has to reconcile two incompatible patterns in the same codebase. Multiply that by however many decisions a codebase depends on and however fast agents are now proposing changes, and the reconciliation cost stops being a rounding error.
The reconciliation cost is not evenly distributed
Two responses to rising AI-driven throughput are visible in the data, and they are not the same finding. Faros AI’s Engineering Report found throughput up alongside incidents, review times, and unreviewed merges: the delivery pipeline straining under volume. DevOps Research and Assessment (DORA) delivery metrics can register that strain without ever answering the separate question of whether what shipped is architecturally coherent. A change can clear CI, pass review, and deploy cleanly, and still be the thing that makes the next six changes harder to reason about. Throughput metrics and architectural-coherence metrics fail independently, which is why a team can look healthy on one and be accumulating debt on the other at the same time.
This is also why "just review more carefully" stops scaling before spend does. Review capacity is bounded by reviewer hours, so it grows at most in step with the number of people qualified to review. AI-driven generation capacity does not grow linearly with anything an organization directly controls; it scales with however fast the agent can iterate, which is a different curve entirely. A 28× increase in spend does not imply a 28× increase in the org’s capacity to check what that spend produced. Something has to close the widening part of that gap, and unreviewed merges are the visible symptom of nothing closing it yet.
Three layers, not one dashboard
Most AI-engineering dashboards stop at generation: tasks completed, code produced, adoption rate. A useful measurement model needs three layers, and the third is the one that is almost always missing.
| Layer | Question | Representative metrics |
|---|---|---|
| Generation | Is work being produced faster? | Task completion time, AI-assisted work rate, generation cost |
| Delivery | Does generated work become production-ready? | Cycle time, review rounds, rework rate, change failure rate |
| Governance | Does increased autonomy preserve the constraints the organization depends on? | Architectural-violation rate, decisions mechanically protected, repeat violations |
DORA-family metrics own the delivery layer well, and they are necessary. They were not built to answer whether autonomous development is preserving architectural intent, and they don’t claim to. The governance layer is where that question gets answered, and it is the layer missing from almost every AI-adoption dashboard published so far.
It is missing for a structural reason, not an oversight. Generation and delivery metrics are both derivable from systems every engineering org already instruments: the agent platform, the CI pipeline, the issue tracker. Governance metrics require a separate artifact that most organizations don’t yet have: an explicit, machine-checkable record of which decisions are supposed to constrain a change, and evidence of whether each proposed change was checked against it. Without that record, "architectural coherence" has no number attached to it, and what has no number attached to it tends not to make the dashboard, regardless of how much it's costing.
A five-stage ROI scorecard
The three layers above tell you where to look. A scorecard needs finer cuts, because two costs hide inside the delivery layer and the actual return sits beyond all three. Split the pipeline into five stages and give each one an owner and a number.
The governance row uses the four states that Mneme, an open-source tool that checks AI-written changes against rules derived from recorded architectural decisions, reports in its Architecture Protection Audit. Protected: an active rule enforces the decision. Mneme-ready: the audit has written a rule that waits for someone to activate it. Requires modelling: the decision needs breaking into smaller parts before a rule can express it. Guidance: no rule can express the decision, so it stays with reviewers.
| Stage | Question | Example metrics |
|---|---|---|
| Generation | Is work produced faster? | Task completion time, AI-assisted share of merged changes, tool spend per engineer |
| Delivery | Does produced work reach production? | Cycle time, review rounds per pull request, deployment frequency |
| Rework and quality | What does it cost to make the work acceptable? | First-pass PR acceptance, CI retry rate, senior time spent correcting AI output, change failure rate |
| Governance | Are the decisions the codebase depends on enforced? | Share of recorded decisions in each audit state: Protected, Mneme-ready, Requires modelling, Guidance |
| Business outcome | Did the organization ship more of what it wanted? | Share of engineering time on new features, request-to-release lead time, adopted features per quarter |
Three of the rework metrics need tight definitions, or each team will compute them differently.
- First-pass PR acceptance. The share of AI-assisted pull requests (PRs) merged without a changes-requested review and without a follow-up commit that changes the approach. Formatting fixes don't count against it. A falling rate means reviewers are absorbing the generation gain.
- CI retry rate. Failed CI runs per merged PR, counting both reruns of an unchanged commit and new pushes made only to get the pipeline green. Track flaky-test reruns as a separate line, so a fragile test suite doesn't get billed to the agent.
- Senior time spent correcting AI output. Hours that staff and principal engineers spend redirecting an AI-generated choice: review threads about a dependency or boundary, rewrites, design discussions reopened inside a PR. Few teams can pull this from existing tools. A two-week sample, with seniors tagging review comments as correction or ordinary feedback, gives you a usable baseline.
The governance row measures coverage, and you can read it directly from the audit: how many recorded decisions an active rule protects, how many wait on one activation step, how many need modelling first, and how many are guidance that a rule can’t express. Coverage carries no ROI number of its own. It tells you which of the senior correction hours a rule could have prevented, and which ones only a reviewer could have caught.
The business-outcome row anchors the rest. DX’s Q2 2026 report tracks this as an innovation ratio, time on new feature work versus maintenance, and found it held at roughly 57–58% while median quarterly AI spend in the tech sector grew from about $1.5K to $44K in a year. If your own ratio sits still while generation metrics climb, the gain is leaking somewhere in stages two to four.
To turn the stages into one figure, charge each downstream hour back against the hours saved. An illustrative model, with hypothetical numbers:
Net value = (hours redeployed to delivered work × loaded rate) − tool spend − (added review + rework + CI recovery hours) × loaded rate − senior correction hours × senior rate
Suppose a team redeploys 400 hours in a quarter, at a loaded rate of $100 an hour: $40,000. It spends $10,000 on tools, adds 200 hours of review, rework and CI recovery ($20,000), and spends a further 40 senior hours at $150 on corrections ($6,000), counted separately from the 200. The vendor dashboard reports 400 hours saved. The net is $4,000, a tenth of the gross. Count only hours you can show went to delivered work. Self-reported time saved overstates the first term.
Our six-level ladder runs 0 spend, 1 adoption, 2 productivity, 3 delivery, 4 quality, 5 governed ROI. In the vendor and internal dashboards we have seen, most stop between Levels 1 and 2; we have no survey behind that. The five stages give you a number at each level above that. The AI adoption maturity model is a different scale: five levels from the Carnegie Mellon Software Engineering Institute and Accenture, covering the organizational side of adoption.
What to check before renewing an AI coding budget
A short diagnostic, in order:
- Has code-generation throughput increased?
- Has production throughput increased by a comparable amount, or does the NBER-style attenuation show up in your own data?
- Has review time changed, and in which direction?
- Has rework or churn changed?
- How much senior engineering time goes to correcting AI-generated architectural decisions specifically, as distinct from ordinary bugs?
- Can the architectural decisions that matter be checked automatically before a change reaches review, or does every one of them depend on a reviewer remembering it?
The last question is the one an Architecture Protection Audit answers directly. It doesn’t estimate ROI. It measures something narrower and more actionable: of the architectural decisions a codebase depends on, how many are Protected by an active rule, how many could be with one activation step, how many need further modelling before they can become a rule, and how many are properly Guidance, left to review by deliberate choice rather than by gap.
The claim this page is not making
Architectural drift is one contributor to the gap between AI spend and delivered value. It is not the only one, and this page does not argue otherwise. Requirements churn, organizational structure, security review, and the underlying difficulty of the work all shape the same numbers. What can be said with more confidence is narrower and, for readers evaluating governance tooling specifically, more useful: architectural reconciliation is a downstream cost that generic AI-productivity measurement does not expose, and it is a cost engineering systems can increasingly prevent rather than merely detect later.
The other articles on AI engineering ROI follow that thread. The next piece takes the NBER attenuation and asks the sharper version of this page’s question directly: if raw coding activity is up 240% and releases are up 30%, what is the actual unit of AI coding productivity, and why has the industry been measuring the wrong one.