Fairness explorer
About the data
The cases are hypothetical and the numbers come from a seeded random number generator. Each case is loosely based on a real system, linked in its description, and the numbers do not estimate how that system performs.
A risk score rates people from 0 to 100, and everyone at or above the cutoff is flagged. Groups A and B are scored by the same tool but have different base rates, as in the Week 3 reading. Nothing is saved or sent.
Case
Error costs
Confusion matrix
The counts cover all 1,200 people at the current cutoff.
- Recall / sensitivity
- The share of everyone actually high-risk who are flagged: TP / (TP + FN).
- Precision / PPV
- The share of everyone flagged who are actually high-risk: TP / (TP + FP).
- False-positive rate (Type I)
- The share of everyone actually not high-risk who are wrongly flagged: FP / (FP + TN).
- False-negative rate (Type II)
- The share of everyone actually high-risk who are missed: FN / (FN + TP).
This adds a separate slider for group B.
Consequences at this cutoff
Which cutoff would you choose for this case?
Group A
Real base rate: 26% actually high-risk
Group B
Real base rate: 50% actually high-risk
Fairness tests
Each test compares one score between the groups and passes when they are within 5 percentage points.
Conflict between the tests
When groups have different base rates, no single cutoff or pair of cutoffs can equalize the flag rate, the false-positive rate and the predictive value at once, except at extremes where almost everyone or almost no one is flagged (Chouldechova, 2017; Kleinberg, Mullainathan, and Raghavan, 2016). Predictive value rises with the base rate, so equal error rates give the group with the higher base rate a higher predictive value and a higher flag rate. Choosing which test to meet and which error to accept is a decision about values.
Try separate cutoffs: can you get all three to pass?
These sliders move the same cutoffs as the ones above and turn on separate cutoffs.
More scores
| Measure | Group A | Group B | All |
|---|
Calculations
TP stands for true positives, FP for false positives, FN for false negatives and TN for true negatives.
Score distributions
Bars count people in five-point score bands, with people who have the outcome above the line and people without it below. Dark bars left of the cutoff are misses and light bars right of it are false alarms. Dragging on either chart moves the cutoff.
has the outcome does not have the outcome
In practice
The bars overlap because some people with the outcome score low and some without it score high. Inside the overlap every cutoff trades misses against false alarms, and a tool whose scores overlap less forces a smaller trade.
ROC curve
Each point is one cutoff, plotting recall (vertical) against the false-positive rate (horizontal); the circle and square mark the current cutoffs for groups A and B. The dashed diagonal is chance, and curves nearer the top-left corner separate people with and without the outcome better.
The area under the curve (AUC) is the probability that a randomly chosen person with the outcome scores higher than a randomly chosen person without it. An AUC of 0.5 is chance, 1.0 is perfect ranking, and the cutoff does not change it.
group A (circle at its cutoff) group B (square at its cutoff)
In practice
A program acts at one cutoff, and two groups with nearly the same AUC can have very different error rates there. AUC also says nothing about whether a score of 70 means a 70 in 100 chance; calibration checks that.
Calibration
A score is calibrated when about 70 percent of people scored around 70 have the outcome. Each point is a ten-point score band, plotting average score (horizontal) against the share with the outcome (vertical), and points on the dashed diagonal are calibrated. Calibration does not change with the cutoff.
group A group B. Bands with fewer than 15 people are left out.
In practice
A caseworker who sees a score of 70 reads it as roughly 70 in 100. A tool can be well calibrated in one group and off in another, so the same number carries different risk depending on whom it describes.
Calibration calculation
Predicted share = average score in the band / 100. Observed share = people in the band with the outcome / people in the band. A score can be calibrated and still have poor recall.
| Score band | Group A predicted and observed | Group B predicted and observed |
|---|
Model training: how a score like this is learned from past records, and how overfitting happens.