When to kill an AI pilot — CFO AI Ledger

When it comes up

At a pilot's end date or review, when results are mixed, when costs rise faster than value, or when a sponsor asks for 'just one more quarter'.

Your options

Scale

Fits when: The pilot met its success metric against the baseline, adoption is real, run costs are understood and controls work.

Watch out: Scaling exposes data, cost and control problems hidden at pilot size; re-forecast before committing.

Fix and extend once, with a new end date

Fits when: Results are close, the cause of the gap is specific (data, workflow, training) and fixable, and the break-even adoption is reachable.

Watch out: Endless extensions; set one new date and one metric, and stop if missed.

Stop

Fits when: No measurable value against the baseline, break-even is not reachable, costs are escalating, or risk controls are inadequate.

Watch out: Sunk-cost bias and sponsor pressure; capture the lessons and reuse what works (data, components).

How to decide

Set kill criteria before the pilot starts: metric, baseline, end date, minimum result.
Compare against the baseline or holdout, not against expectations.
Re-run the business case with pilot data: adoption, unit costs, run costs. Check break-even adoption.
Check control evidence: can it pass audit at scale?
Decide scale, fix once, or stop, and document why.
Reallocate the budget from stopped pilots to proven ones.

Trade-offs

Stopping early saves money and focus but may abandon an idea that needed one more fix; extending keeps options open but feeds sunk-cost drift; scaling captures value but multiplies any hidden flaw.

Real examples

On November 2, 2021, Zillow's board decided to wind down Zillow Offers, its algorithm-supported home-buying business, citing home pricing unpredictability and operational challenges; Zillow recorded a $304.4 million inventory write-down in Q3 2021 for homes bought at prices above its estimates of future selling prices, and expected a roughly 25% workforce reduction. Source: Zillow Group — Form 8-K, wind-down of Zillow Offers (Nov 2, 2021) ↗
Gartner predicted in June 2025 that over 40% of agentic AI projects will be canceled by the end of 2027 due to escalating costs, unclear business value or inadequate risk controls. Source: Gartner — Over 40% of agentic AI projects will be canceled by end of 2027 (Jun 25, 2025) ↗
Gartner predicted in February 2025 that through 2026 organizations will abandon 60% of AI projects unsupported by AI-ready data. Source: Gartner — Lack of AI-ready data puts AI projects at risk (Feb 26, 2025) ↗

What to check first

Whether kill criteria and an end date were agreed at the start.
Pilot results against the baseline or a holdout.
Break-even adoption from the ROI builder, using pilot data.
Whether costs are escalating faster than adoption.

Learn and use

Sources

Last reviewed 2026-10-01.