source: kdnuggets: auditing model bias with balanced datasets with mimesis
level: technical
machine learning models can pick up biases from training data. a loan approval classifier trained on biased historical data may unfairly favor one group. this article shows how to audit such a model using mimesis, a python library for generating synthetic data. the process starts by creating a biased dataset where men are always approved and women only when income exceeds 80,000. a decision tree is then trained on this data.
mimesis generates three base financial profiles with random ids and moderate incomes between 40,000 and 70,000. for each profile, two counterfactual copies are made: one male and one female, with identical income and id. this creates a perfectly balanced test set where gender is the only variable. the model predicts on these pairs, and the results reveal clear discrimination. all male applicants are approved, while all female applicants with the same income are denied.
the audit exposes the model's gender bias without using real sensitive data. mimesis provides full statistical control, isolating the protected attribute. if bias is found, next steps include augmenting training data with balanced examples, applying model re-weighting, or using fairness toolkits like ai fairness 360. this approach helps data scientists detect and address unfair model behavior early.
why it matters: counterfactual testing with synthetic data lets data scientists detect model bias without exposing real personal information, supporting fairer ai systems.
source: kdnuggets: auditing model bias with balanced datasets with mimesis