What is AI and Bias
A situation where an AI system systematically produces more favorable or unfavorable results for specific groups, cases or traits.
Definition
AI bias arises from data, problem definition, metrics, layout, or application context. The model may reproduce past mistakes, increase inequality, or perform worse for rare groups. This does not always mean malicious intent, but the consequences can be serious.
Example
If the hiring system was trained on the history of the company, where people of the same profile were previously hired more often, it may underestimate candidates with different experience.
Why it matters
The term is important for businesses and users: a beautiful AI service can produce unfair results if it is not tested on different groups and scenarios.
How it works
Bias is looked for through data audits, quality comparisons across segments, error analysis, feature testing, and tests for discriminatory effects.
Where it is used
- checking models for fairness
- data audit
- risk assessment in hiring, loans and moderation
Limitations
It is difficult to completely remove bias because the very definition of fairness depends on the task. It is important to explicitly select criteria and document trade-offs.