Discrimination law does not care whether a person or a model made the decision. If your tool produces worse outcomes for a protected group and you cannot justify it, you are liable. Fairness testing is how you find that out before a regulator or a plaintiff does.
The rules
- US federal: Title VII, the ADEA and the ADA apply to hiring tools. The EEOC's iTutorGroup settlement (2023, USD 365,000) involved software that rejected older applicants outright. The "four-fifths rule" from the Uniform Guidelines on Employee Selection Procedures is the usual first screen: a selection rate for any group below 80 percent of the highest group's rate is a flag.
- NYC Local Law 144: an independent bias audit within the past year for any automated employment decision tool used for NYC hiring or promotion, with impact ratios by sex and race/ethnicity published on your website. A December 2025 New York State Comptroller audit found the city's enforcement weak and the agency has committed to proactive review, so expect more scrutiny, not less. Penalties run to USD 1,500 per violation per day.
- Illinois HB 3773 (since 1 January 2026): using AI that has a discriminatory effect in employment decisions violates the Human Rights Act, and using zip codes as a proxy for protected classes is banned.
- Texas TRAIGA: bans developing or deploying AI with the intent to discriminate unlawfully; disparate impact alone is not enough under this statute, but other laws still apply.
- Colorado SB 26-189: dropped the original duty of care against algorithmic discrimination but kept notice, human review and records, and existing anti-discrimination law continues to apply.
- EU AI Act, Article 10: high-risk providers must examine training data for possible biases and take measures to detect and correct them; the Digital Omnibus extended the legal basis for processing special-category data for bias detection to all AI systems.
A minimum testing routine
For each consequential tool:
- Define the outcome you are testing (shortlisted, approved, flagged, priced).
- Pick the groups: at least sex and race/ethnicity where lawful to collect; age and disability where the tool could touch them.
- Compute selection or approval rates per group and the impact ratio against the most favoured group. Do it on your own applicants, not the vendor's benchmark.
- Look at error rates too: false positives and negatives by group. Rite Aid's system was not "biased" on average; its false matches clustered in Black and Asian neighbourhoods.
- Investigate any ratio below 0.8 or any material error gap. Investigate does not mean panic; it means find the cause, document it, and fix or justify it.
- Repeat at least annually and after any model change.
Vendor audits are not your audit
A vendor's audit typically tests on its own pooled data and its own definitions. NYC requires the audit to use your historical data where available. Ask the vendor for the audit, then run the same ratios on your data.
Proxies
Zip code, name, school, gaps in employment, commute distance, and typing speed can all stand in for protected characteristics. If a feature has no clear job-related rationale, remove it.
Keep the evidence
Record the method, the data window, the results, the decisions taken and who approved them. Some organisations run testing under legal privilege; discuss the pros and cons with counsel, because privilege can also stop you using the results as a defence.