Model training
A logistic regression learns a risk score from the past records of 1,200 synthetic people, 600 in group A and 600 in group B, for the same cases as the Fairness explorer. Thirty percent of people are held out as test cases the model never sees during training. The numbers are made up, and nothing is saved or sent.
Case
Training choices
Records available to the model
Choose one or more. The neighborhood record has no direct link to the outcome, but most group B people live in the neighborhoods it marks high. The day-of-week record is pure chance.
Training outcome
Past records for training
Model flexibility
The simple model has one weight per chosen record. The flexible model adds 25 columns of chance values, so it has many more weights it can tune to quirks of the training cases.
Results
Accuracy counts a score of 50 or more as a flag. Accuracy and AUC are measured against the real outcome, even when the model learns from a proxy. With a rare outcome, a model that flags almost no one can have high accuracy and low recall.
Overfitting
A model that fits the quirks of its training cases does worse on new ones. On the loss chart, overfitting shows up as a training loss that keeps falling while the test loss levels off or rises. A county model tuned on its own past records can look better in validation than it performs on next year's calls, so held-out test cases, and external validation on records from another year or place, come before a tool is used.
Try adding the neighborhood record, training again and watching group B’s false-positive rate.
Try the flexible model with 60 training records and comparing training and test accuracy.