PreviewYou're previewing CTO AI Forge. Sign in to use it.Sign in
← All lessons
L1 · AI across the SDLC8 min read

AI for testing and code review

Testing and review are where AI-written code is caught — and they are also places where AI itself can help.

Test generation

LLMs can propose unit tests, extend existing ones and suggest edge cases. The hard part is trusting generated tests: a test can pass while asserting the wrong behaviour. Good systems filter generated tests automatically — does it build, does it pass reliably, does it add coverage — before a human sees them.

Meta's TestGen-LLM is a well-documented example. It extends existing human-written tests and keeps only candidates that pass a set of filters. In Meta's evaluation, 75% of its test cases built correctly, 57% passed reliably and 25% increased coverage; in Instagram and Facebook test-a-thons it improved 11.5% of the classes it was applied to, and engineers accepted 73% of its recommendations for production.

Review assistance

AI can help in two ways:

  1. First-pass review: flag likely bugs, security issues, missing tests and style problems before a human looks.
  2. Resolving comments: propose the edit that addresses a reviewer's comment.

Google built the second into its internal review tool. Its research blog (May 2023) reported that the model addressed 52% of comments with a target precision of 50%, that 40–50% of previewed suggested edits were applied, and that Google expected it to save hundreds of thousands of hours of review time a year.

The review trap

If AI writes more code and AI also reviews it, human reviewers may start skimming. Keep a named human approver for every change, and decide which changes need deeper review: security-sensitive code, data migrations, public APIs, infrastructure. Track review time — if it climbs as AI adoption rises, review has become your bottleneck.

What this means for the CTO

Fund testing and review capacity before you scale code generation. Generated tests need automatic filters (build, pass reliably, add coverage); AI review should support humans, not replace the approver.

Ask your teams for two numbers per period: median review time and change-failure rate. If either rises as AI use rises, slow down code generation and invest in tests, smaller PRs and review assistance.

Also make reviewer capacity visible in planning: if the same few senior engineers approve most changes, AI will overload them first. Spreading review, and using AI for a first pass, protects both speed and quality.

Real example

Meta's TestGen-LLM paper (February 2024) reports that 75% of its test cases built correctly, 57% passed reliably and 25% increased coverage; in Instagram and Facebook test-a-thons it improved 11.5% of classes it was applied to, and Meta engineers accepted 73% of its recommendations for production. Google Research reported in May 2023 that ML-suggested edits addressed 52% of reviewer comments at a target precision of 50%. Source: Alshahwan et al. — Automated Unit Test Improvement using Large Language Models at Meta (arXiv 2402.09171, 2024) ↗ · Google Research — Resolving code review comments with ML (May 23, 2023) ↗

Use it

Quick check · 3 questions

  1. 1.Why should generated tests pass automatic filters before humans see them?

  2. 2.What is the 'review trap'?

  3. 3.Which signal suggests review has become the bottleneck?

0 of 3 answered

Sources

Last reviewed 2026-10-01.