Semi-automatic assessment of datasets fairness
FanFAIR is a rule-based approach based on fuzzy logic able to calculate some fairness metrics over a dataset and combine them into a single score, enabling a semi-automatic evaluation of a dataset in algorithmic fairness research.
We try to make FanFAIR as simple and automatic as possible, in order to make it very operational and streamlined. Nevertheless, a few analysis (notably, the legal compliance) cannot be entirely offloaded to algorithms and require the intervention of a domain expert human operator.
FanFAIR is designed to be as automatic as possible. However, two metrics (quality, compliance) require human intervention. Here is an example of analysis performed with FanFAIR:
from fanfair import FanFAIR
FF = FanFAIR(dataset="myfile.csv", output_column="output")
FF.set_compliance( {"data_protection_law": True,
"copyright_law": True,
"medical_law": True,
"non_discrimination_law": False,
"ethics": False})
FF.set_quality(0.9)
FF.produce_report()
The analysis is automatically performed by calling the produce_report method, which generates two main figures: the gauge with the overall fairness score (from 0% to 100%), and the plots of the linguistic variables of the fuzzy model, which provide a summary of the metrics for the dataset's fairenss features.
The user can also specify a set of sensitive variables and FanFAIR will automatically check any correlations with the output and assess the fairness ex post with respect to such features.
If you find FanFAIR useful for your research, please cite our project as follows:
Gallese, C., Scantamburlo, T., Manzoni, L., Giannerini, S., & Nobile, M. S. (2025). FanFAIR: sensitive data sets semi-automatic fairness assessment. BMC Medical Informatics and Decision Making, 25(Suppl 3), 329
Gallese C., Scantamburlo T., Manzoni L., Nobile M.S.: Investigating Semi-Automatic Assessment of Data Sets Fairness by Means of Fuzzy Logic, Proceedings of the 20th IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (IEEE CIBCB 2023), 2023
If you need additional information, or want to see additional metrics implemented in FanFAIR, please feel free to contact Dr. Chiara Gallese (c.gallese@tilburguniversity.edu).
