The FTC's complaint against Rite Aid repeated one phrase: the company "failed to test, assess, measure, document, or inquire". Documentation is not bureaucracy; it is the only way to prove you took care. This lesson covers the two kinds you need: a record file per system, and an impact assessment for the risky ones.
The record file
Keep one folder (digital is fine) per inventory entry that is high-risk, consequential, or customer-facing. Contents:
- Purpose and scope: what it does, for whom, and what it must not be used for.
- Risk classification and the reasoning behind it.
- Data: sources, legal basis, retention, the data-protection agreement.
- Vendor documentation: instructions for use, model card, audits.
- Testing: fairness results, accuracy, security tests, dates.
- Oversight: reviewer roster, override log, escalation path.
- Disclosures: the notices used and where they appear.
- Change log: model versions, configuration changes, who approved.
- Incidents and complaints, with what you did.
- Review dates: last and next.
Retention rules give you the minimum: Colorado SB 26-189 requires deployers to keep compliance documentation for at least three years; Ontario requires job postings and applications kept for three years; EU AI Act deployers must keep system logs for at least six months (Article 26). Keep the file for as long as the system runs plus the longest of these.
Impact assessments
An impact assessment is a structured "what could go wrong, for whom, and what we did about it" written before go-live and refreshed when things change. Several laws name one:
- Fundamental rights impact assessment (EU AI Act Article 27) for certain deployers of high-risk systems, including public bodies and those providing credit or insurance.
- Algorithmic Impact Assessment under Canada's Directive on Automated Decision-Making: a questionnaire that yields the impact level (I to IV) and the required controls, completed before production and published.
- Privacy impact assessment under Quebec's Law 25 for any project involving personal information, and a DPIA under GDPR Article 35 for high-risk processing.
- Colorado removed its impact-assessment mandate in SB 26-189, but notice, human review and records remain, and an assessment is the easiest way to generate them.
A one-page template
If you have no template, use these headings:
- System and owner
- Purpose and decision affected
- People affected and any vulnerable groups
- Data used and legal basis
- Risks: accuracy, bias, privacy, security, over-reliance, and for each the likelihood, severity and mitigation
- Oversight and appeal route
- Residual risk and sign-off: who accepted it and when
One page done honestly beats forty pages nobody reads.
Decisions about individuals
Where a tool influences a decision about a person, keep enough to reconstruct it: inputs, output, the human reviewer's action, and the notice sent. That is what Colorado's 30-day post-adverse explanation, Quebec's right to observations and the EU AI Act's Article 86 right to an explanation will require you to produce.