A General Framework for Fair Regression

Abstract

Fairness, through its many forms and definitions, has become an important issue facing the machine learning community. In this work, we consider how to incorporate group fairness constraints into kernel regression methods, applicable to Gaussian processes, support vector machines, neural network regression and decision tree regression. Further, we focus on examining the effect of incorporating these constraints in decision tree regression, with direct applications to random forests and boosted trees amongst other widespread popular inference techniques. We show that the order of complexity of memory and computation is preserved for such models and tightly binds the expected perturbations to the model in terms of the number of leaves of the trees. Importantly, the approach works on trained models and hence can be easily applied to models in current use and group labels are only required on training data.

Keywords: Gaussian process; algorithmic fairness; constrained learning; decision tree; kernel methods; machine learning; neural network.

Figure 1

This is a visualisation of a decision tree kernel matrix with marginal constraint, left in explicit representation and right in compressed representation. The dark cell in the upper left of the matrix is the double integrated kernel function with respect to the difference of input distributions, which constrain the process. The solid grey row and column are single integrals of the kernel function. White cells have zero values and the dashed (block) diagonals are the kernel matrix between observations or leaves of the tree. We can note that the above, compressed representation kernel matrix is an arrowhead matrix, which we exploit to create an efficient algorithm.

Figure 2

The figure shows synthetic data of two populations, $p_A\left(x\right)$ (blue) and $p_B\left(x\right)$ (orange). The main plots show the observations and the perturbation to the respective models. Purple functions identify the original inferred functions and green indicates the fair perturbed inferred functions. Below, the main plots show a normalised histogram of the observations for the $p_A\left(x\right)$ and $p_B\left(x\right)$ populations, respectively, along with the PDF of the Gaussian mixture model of their respective densities. To the right shows how the expected mean of the two populations have been perturbed to be equal.

Figure 3

The figure shows the output distribution of decile scores for African Americans and non-African Americans before (blue) and after (orange) the mean equality constraint was applied. We can see that the respective means (vertical lines) become approximately equal after the inclusion of the constraint using the empirical input distribution.

Figure 4

The above figure shows the effect of multiple GFE constraints acting on a single regression task. Blue signifies the original model and orange the perturbed model. The dashed horizontal lines signify the mean before and after perturbation.