AI systems can create, propagate, support, and automate bias in decision-making processes. To mitigate biased decisions, we both need to understand the origin of the bias and define what it means for an algorithm to make fair decisions. By Locating Unfairness through Canonical Inverse Design (LUCID), we generate a canonical set that shows the desired inputs for a model given a preferred output. The canonical set reveals the model's internal logic and exposes potential unethical biases by repeatedly interrogating the decision-making process.
LUCID-GAN extends on LUCID by generating canonical inputs via a conditional generative model instead of gradient-based inverse design. LUCID-GAN generates canonical inputs conditional on the predictions of the model under fairness evaluation. LUCID-GAN has several benefits, including that it applies to non-differentiable models, ensures that a canonical set consists of realistic inputs, and allows us to assess indirect discrimination and explicitly check for intersectional unfairness.
Read our paper on LUCID and LUCID-GAN for more details, or check out the documentation.
We encourage everyone to contribute to this project by submitting an issue or a pull request!
Install canonical_sets from PyPi.
pip install canonical_setsFor development install, see contribute. You can also check the documentation.
LUCID can be used for the gradient-based inverse design to generate canonical sets, and is available for both
PyTorch and Tensorflow models. It only requires a model, a preferred output, and an example input
(which is often a part of the training data). The results are stored in a pd.DataFrame, and can be accessed by
calling results. It's fully customizable, but can also be used out-of-the-box for a wide range of
applications by using its default settings:
import pandas as pd
from canonical_sets.data import Adult
from canonical_sets.models import ClassifierTF
from canonical_sets import LUCID
adult = Adult()
model = ClassifierTF(2)
outputs = pd.DataFrame([[0, 1]], columns=["<=50K", ">50K"])
example_data = adult.train_data
lucid = LUCID(model, outputs, example_data)
lucid.results.head()LUCIDGAN generates canonical sets by using conditional generative models (GANs). This approach has several benefits,
including that it applies to non--differentiable models, ensures that a canonical set consists of realistic inputs,
and allows us to assess indirect discrimination and explicitly check for intersectional unfairness. LUCID-GAN only
requires the input and predictions of a black-box model. It's fully customizable, but can also be used out-of-the-box
for a wide range of applications by using its default settings:
import pandas as pd
from canonical_sets.data import Adult
from canonical_sets.models import ClassifierTF
from canonical_sets import LUCIDGAN
model = ClassifierTF(2)
adult = Adult()
# we need original data as LUCID-GAN does some preprocessing
test_data = adult.inverse_preprocess(adult.test_data)
# we only require the predictions for the positive class
preds = model.predict(adult.test_data.to_numpy())[:, 1]
data = pd.concat([test_data, pd.DataFrame(preds, columns=["preds"])], axis=1)
lucidgan = LUCIDGAN(epochs=5)
lucidgan.fit(data, conditional=["preds"])
samples = lucidgan.sample(100, conditional=pd.DataFrame({"preds": [1]}))
samples.head()For detailed examples see examples and for the source code see canonical_sets. For LUCID, we advice to start with either the
tensorflow or pytorch example, and then the advanced example. For LUCIDGAN, you can replicate the experiments from the paper
with the GAN_adult and GAN_compas examples. You can also check the documentation for more details.
If you have any remaining questions, feel free to submit an issue or PR!
Most group fairness notions focus on the equality of outcome by computing statistical parity metrics on a model's output.
The two most prominent examples of these statistical output-based metrics are Demographic Parity (DP) and Equality Of Opportunity (EOP).
In DP, we compare the Positivity Rate (PR) of the subpopulations under fairness evaluation, and in EOP, we compare the True Positive Rate (TPR).
The choice between DP and EOP depends on the underlying assumptions and worldview of the evaluator.
The Metrics class allows you to compute these metrics for binary classification tasks given the predictions and ground truth:
from canonical_sets.data import Adult
from canonical_sets.models import ClassifierTF
from canonical_sets.group import Metrics
model = ClassifierTF(2)
adult = Adult()
preds = model.predict(adult.test_data.to_numpy()).argmax(axis=1)
targets = adult.test_labels[">50K"]
metrics = Metrics(preds, targets)
metrics.metricscanonical_sets contains some functionality to easily access commonly used data sets in the fairness literature:
from canonical_sets import Adult, Compas
adult = Adult()
adult.train_data.head()
compas = Compas()
compas.train_data.head()The default settings can be customized to change the pre-processing, splitting, etc. See examples for details. You can also check the documentation.
If you are interested in cross-disciplinary research related to machine learning, feel free to:
The package and the code is provided "as-is" and there is NO WARRANTY of any kind. Use it only if the content and output files make sense to you.
This project benefited from financial support from Innoviris.
LUCIDGAN is based on the CTGAN class from the ctgan package. It has been extended to fix
several bugs (see my PR on the CTGAN GitHub page) and to allow for the extension of the conditional
vector. A part of the code and comments is identical to the original CTGAN class.
@inproceedings{mazijn_lucid_2023,
title={{LUCID: Exposing Algorithmic Bias through Inverse Design}},
author={Mazijn, Carmen and Prunkl, Carina and Algaba, Andres and Danckaert, Jan and Ginis, Vincent},
booktitle={Thirty-Seventh AAAI Conference on Artificial Intelligence (accepted)},
year={2023},
}
@article{algaba_lucidgan_2022,
title={{LUCID-GAN: Conditional Generative Models to Locate Unfairness}},
author={Algaba, Andres and Mazijn, Carmen and Prunkl, Carina and Danckaert, Jan and Ginis, Vincent},
year={2022},
journal={Working paper}
}