Google researcher Moritz Hardt and colleagues have developed an approach for testing whether machine learning algorithms inject bias, such as gender or racial bias into their decisions. There has been worry for some time that ML algorithms might deliberately or inadvertently inject bias in applications spanning advertising, credit, employment, education, and criminal justice.
The paper, Equality of Opportunity in Supervised Learning, is written by Hardt and colleagues Eric Price (University of Texas, Austin), and Nathan Srebro (University of Chicago) and posted on the ARXiv.org site. Back in 2014, the Obama Administration’s Big Data Working Group released a report arguing that discrimination can sometimes “be the inadvertent outcome of the way big data technologies are structured and used” and pointed toward “the potential of encoding discrimination in automated decisions”.
The authors note, “Despite the demand, a vetted methodology for avoiding discrimination against protected attributes in machine learning is lacking. A naive approach might require that the algorithm should ignore all protected attributes such as race, color, religion, gender, disability, or family status. However, this idea of “fairness through unawareness” is ineffective due to the existence of redundant encodings, ways of predicting protected attributes from other features.
The group’s work “depends only on the joint statistics of the predictor, the target and the protected attribute, but not on interpretation of individual features. We study the inherent limits of defining and identifying biases based on such oblivious measures, outlining what can and cannot be inferred from different oblivious tests. We illustrate our notion using a case study of FICO credit scores.”
Details of the approach can be found in the paper: https://arxiv.org/abs/1610.02413