What is AUC
A classifier evaluation metric that shows how well the model separates positive and negative examples at different thresholds.
Definition
AUC is often used in conjunction with an ROC curve. The metric helps to understand how capable the model is of ranking examples: for example, assigning a higher risk to suspicious transactions than to normal ones. AUC is convenient when you need to evaluate quality not at one threshold, but over a whole range of solutions.
Example
The anti-fraud model assigns risk to transactions. A high AUC means that fraudulent transactions are more likely to receive a higher risk than normal ones.
Why it matters
The term is important for comparing models, especially when you need to select a solution threshold for a business problem after training.
How it works
The system builds a relationship between the proportion of positive examples found and the proportion of false alarms at different thresholds. AUC is equal to the area under this curve.
Where it is used
- antifraud
- medical diagnostics
- evaluation of scoring models
Limitations
AUC does not show how a model performs at a specific threshold and a specific cost of error. For practice, additional metrics and error analysis are needed.