Change-Relaxed Active Fairness Auditing - Archive ouverte HAL
Communication Dans Un Congrès Année : 2023

Change-Relaxed Active Fairness Auditing

Résumé

The pervasive deployment of user-facing automated decisions systems raises concerns over their impact on society. The sheer amount of such online platforms and their growing complexity highlights the need for automated and robust audits to assess their impact on users. This paper focuses on a recent theoretical advance named manipulationproofness. It aims at guaranteeing successive audits of a platform cannot be gamed by the platform, provided the labels returned on the audit dataset do not change. While this constitutes a decisive step for reliable audits, it is too restrictive, as models naturally evolve with time in practice. This paper thus explores how manipulation-proofness can be adapted to better fit actual scenarios, by studying the effects of relaxing the constraint on the amount of change the remote model can operate while being audited. Our results on the COMPAS dataset demonstrate a request gain in one of the two models considered, while also noticing the surprisingly good performance of the random strawman approach. We believe this constitutes an interesting step for further attempts to improve reliable and manipulation proof audits.
Fichier principal
Vignette du fichier
RJCIA2023_paper_10.pdf (359.47 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04395914 , version 1 (15-01-2024)

Licence

Identifiants

  • HAL Id : hal-04395914 , version 1

Citer

Augustin Godinot, Erwan Le Merrer, Gilles Trédan, Camilla Penzo, François Taïani. Change-Relaxed Active Fairness Auditing. RJCIA 2023 - 21e Rencontres des Jeunes Chercheurs en Intelligence Artificiel, Association Française pour l'Intelligence Artificielle, Jul 2023, Strasbourg, France. pp.91-96. ⟨hal-04395914⟩
202 Consultations
90 Téléchargements

Partager

More