What is Algorithmic Discrimination
A situation where an automated system unfairly makes people worse off because of attributes, data, or decision rules.
Definition
Algorithmic discrimination can occur if the data reflects past biases, the model uses indirect indicators, or the metric optimizes the business experience to the detriment of certain groups. It is dangerous because it appears neutral, although the consequences may be unfair.
Example
A scoring system may perform less well on people from areas with historically less access to financial services, even if it does not explicitly use a protected characteristic.
Why it matters
The term is important for assessing the risks of AI in hiring, lending, education, medicine, advertising and government services.
How it works
Discrimination is identified through data analysis, comparing errors across groups, auditing attributes, testing scenarios, and testing the consequences of decisions.
Where it is used
- fairness assessment
- regulated industries
- audit of automated solutions
Limitations
Not every difference in results is discrimination, but any meaningful difference requires explanation, verification, and context.