Healthcare & wellbeing

Regulated uses

• Diagnostic and clinical decision-support software (often a medical device)
• Patient-facing triage, symptom-checker and mental-health chatbots
• Ambient scribing and note generation from consultations
• Eligibility, prior-authorisation and benefits decisions
• Emotion recognition or monitoring of patients or staff

Laws by region

Obligations checklist

• ☐Check whether each tool is a medical device under FDA, Health Canada or EU MDR rules before any AI-law analysis.
• ☐Tell patients when AI is used in their care, no later than when the service is provided (Texas) and at the start of health or mental-health interactions (Utah).
• ☐Keep a clinician in the loop with authority to override, and log overrides.
• ☐Run a privacy/data-protection impact assessment for any tool processing health data.
• ☐Validate accuracy and bias across patient groups and monitor drift after deployment.
• ☐Set retention and confidentiality rules for prompts, recordings and generated notes.
• ☐For mental-health chatbots, document clinician involvement in design and testing and avoid advertising inside sessions.

Real cases

• Ban2023 · US-federal
  FTC v. Rite Aid — five-year ban on facial recognition surveillance

  Settlement announced 19 December 2023. Rite Aid is banned from using facial recognition for surveillance for five years, must delete the images and any models built from them, must notify consumers before enrolling them in any future biometric system, run a comprehensive security programme with independent assessments for 20 years, and have its CEO certify compliance annually.

• Ruling2024 · CA-BC
  Moffatt v. Air Canada — airline liable for its chatbot's wrong answer

  Decision 2024 BCCRT 149 (February 2024). Air Canada was ordered to pay C$812.02 in total: C$650.88 in damages (the fare difference) plus pre-judgment interest and tribunal fees. Small money, but the first widely reported ruling that a company answers for what its customer-facing AI says.