The most visible legal AI failure is the fake citation: a brief or submission that cites cases that don't exist, or quotes words a court never wrote. It has happened in hundreds of cases worldwide, and courts treat it as a serious breach of the duty of candour, whatever tool caused it.
The landmark cases
- Mata v. Avianca (S.D.N.Y. 2023): lawyers filed an opposition citing non-existent decisions generated by ChatGPT, then defended them when questioned. Judge P. Kevin Castel sanctioned the two lawyers and their firm, jointly fining them $5,000, and found bad faith in how they responded.
- Ayinde v London Borough of Haringey (High Court of England and Wales, 2025): a Divisional Court led by the President of the King's Bench Division dealt with two cases of fictitious citations, made wasted-costs orders and referred lawyers to their regulators, stressing that lawyers who use AI must check its output against authoritative sources.
- Johnson v. Dunn (N.D. Ala. 2025): lawyers from a large firm were disqualified from the case after filings with AI-fabricated citations, despite the firm's internal AI warnings.
A verification workflow that works
- Pull every authority (case, statute, regulation, quotation) into a list.
- Check each one in a primary source: it exists, the citation is right, the quote is verbatim, the proposition is supported and it is still good law.
- Record who checked and when.
- Second-person check for filings to courts and regulators.
- Check local rules: some judges require certification of AI use or verification.
Scale the review to the risk
Not every AI output needs a cite-check. An internal summary for a lawyer's own use needs a spot-check; advice to the board needs full review; a court filing needs full review plus a recorded cite-check.