PreviewYou're previewing CTO AI Forge. Sign in to use it.Sign in
← All lessons
L2 · Measuring engineering impact7 min read

Adoption is not impact

The easiest AI number to report is adoption: seats bought, weekly active users, suggestions accepted. It is also the least meaningful on its own.

What large datasets show

DX, an engineering-intelligence company, publishes data across hundreds of organizations. Its Q4 2025 report (135,000+ developers, 435 companies, July–October 2025) found:

  • overall AI adoption of 91%;
  • developers report saving 3.6 hours a week on average (4.1 hours for daily users);
  • daily AI users ship 60% more pull requests than non-users (median 2.3 vs 1.4 per week);
  • results are uneven: some organizations see quality improve as AI use grows, others see serious degradation;
  • teams that pair rollout with training, governance and measurement outperform those treating AI as plug-and-play.

By DX's Q1 2026 report (April 2026), adoption had reached 93%. When nearly everyone uses AI, adoption no longer tells you who is getting value from it.

Why adoption and impact diverge

  1. Correlation, not causation. Daily users may already be the most productive engineers.
  2. The constraint moves. Faster writing doesn't help if review, CI or release is slow.
  3. Time saved leaks. Saved hours go into other tasks, meetings or rework unless the system is designed to capture them.
  4. Quality costs come later. Rework and incidents show up periods after the code ships.

What to report instead

A layered view: adoption (are people using it?), engagement (how deeply?), impact (did delivery outcomes move?), cost (what did it cost per unit of outcome?). DX's own advice is that there is no "average" AI experience and industry benchmarks should not be used as your targets.

What this means for the CTO

Stop reporting seat counts as success. Give your CEO and CFO a layered view — adoption, engagement, delivery impact and cost — and say clearly which layer each number belongs to.

Most importantly, invest in what turns adoption into impact: enablement and training, fast CI, good documentation and review capacity. That is where DX found winners differ, not in which tool they bought.

When someone presents an AI success story, ask which layer it proves. 'Everyone has a licence' is adoption; 'lead time fell while change failures held' is impact. Only the second justifies more investment, and only with its cost next to it.

Real example

DX's Q4 2025 impact report (published November 4, 2025; 135,000+ developers at 435 companies) found overall adoption of 91%, an average of 3.6 hours saved per week, and daily AI users shipping 60% more PRs than non-users, while noting that some organizations see quality improve and others serious degradation, and that structured enablement outperforms plug-and-play rollout. Source: DX — AI-assisted engineering: Q4 impact report (Nov 4, 2025) ↗

Use it

Quick check · 3 questions

  1. 1.Why is '91% adoption' not proof of impact?

  2. 2.Daily AI users ship 60% more PRs than non-users. What is a key caution?

  3. 3.What did DX find distinguishes better outcomes?

0 of 3 answered

Sources

Last reviewed 2026-10-22.