Periodic Reporting for period 1 - NoBIAS (Artificial Intelligence without Bias)
Reporting period: 2020-01-01 to 2021-12-31
Artificial Intelligence (AI) algorithms are widely employed by businesses, governments, and other organizations in order to optimize efficiency and effectiveness of operations. Decisions, once undertaken by humans, are now conducted by algorithms, increasingly derived through ML and AI powered by big data. Incidents of bias and unfairness in various real-world AI applications have led to an ever increasing public concern about the impact of AI in our lives. AI-based decision making may magnify pre-existing biases and have huge potential for new types of biases. If such issues are not carefully tackled, AI-based decision making may underperform. NoBIAS aims to be the answer in this respect. To achieve this objective, our challenges stem from the AI-based decision making process, which at high-level involves the following phases: data collection, AI algorithms, and results.
At each step in this process biases may arise, which need to be accounted for and countered in order to produce business benefits while addressing related legal and ethical concerns. In particular, the three core challenges are: (C1) Data can be biased; (C2) Algorithms can be biased; (C3) Results can be biased.
Research, development and training of early-stage researchers (ESRs) in NoBIAS is organized around the AI-based decision making pipeline and the identified C1-C3 core challenges and the corresponding O1-O3 objectives (as listed next) to ensure that crucial skills for AI-based decision making in industry and society are acquired and are well-aligned with business value creation.
Objective 1 [O1]: Understanding bias in data. The quality of the data provided as input to AI-decision making processes has a strong influence on the results. Understanding why and how bias is manifested in data is of paramount importance.
Objective 2 [O2]: Mitigating bias in algorithms. To account for bias in AI, we can improve the bias-related quality of the data or we can introduce extra constraints/costs in the utility measure of the model to “enforce” fairness. The former approach is independent of the algorithm whereas the latter depends on the algorithm per se. In the context of NoBIAS, we aim to tackle both model-independent and model-dependent challenges as well as connecting them with legal issues and contexts.
Objective 3 [O3]: Accounting for bias in results. Results of AI-decision making systems might be biased, even if the data has been corrected for bias and even if the algorithms have been modified to account for bias. Moreover, new sources of biases are introduced by the interpretation of the results and application context, when continuous model outputs are converted into binary decisions or when concept-drift arises over time. Therefore, an important challenge is to understand the models extracted from data and how certain decisions are taken and evolve over time.
At each step in this process biases may arise, which need to be accounted for and countered in order to produce business benefits while addressing related legal and ethical concerns. In particular, the three core challenges are: (C1) Data can be biased; (C2) Algorithms can be biased; (C3) Results can be biased.
Research, development and training of early-stage researchers (ESRs) in NoBIAS is organized around the AI-based decision making pipeline and the identified C1-C3 core challenges and the corresponding O1-O3 objectives (as listed next) to ensure that crucial skills for AI-based decision making in industry and society are acquired and are well-aligned with business value creation.
Objective 1 [O1]: Understanding bias in data. The quality of the data provided as input to AI-decision making processes has a strong influence on the results. Understanding why and how bias is manifested in data is of paramount importance.
Objective 2 [O2]: Mitigating bias in algorithms. To account for bias in AI, we can improve the bias-related quality of the data or we can introduce extra constraints/costs in the utility measure of the model to “enforce” fairness. The former approach is independent of the algorithm whereas the latter depends on the algorithm per se. In the context of NoBIAS, we aim to tackle both model-independent and model-dependent challenges as well as connecting them with legal issues and contexts.
Objective 3 [O3]: Accounting for bias in results. Results of AI-decision making systems might be biased, even if the data has been corrected for bias and even if the algorithms have been modified to account for bias. Moreover, new sources of biases are introduced by the interpretation of the results and application context, when continuous model outputs are converted into binary decisions or when concept-drift arises over time. Therefore, an important challenge is to understand the models extracted from data and how certain decisions are taken and evolve over time.
[O1]: Understanding bias in data.
Each ESR started out exploring the problem from a specific direction and each acquired a much deeper understanding of the nature of the manifestation of bias in data within their individual area. The important but common questions that O1 ESRs have been working on are:
(i) What is bias?
(ii) How is bias created?
(iii) How can we detect bias?
To answer these questions, long ethnographic fieldwork has been conducted by ESR Miriam Fahimi, suitable technical frameworks have been identified/ developed, methodological innovations and testing are currently underway for various use cases by ESRs Kristen Marie Scott, Jose Manuel Alvarez, Simone Fabbrizzi and Mayra Russo.
[O2]: Mitigating bias in algorithms.
O2 ESRs focus on the following multi-disciplinary research directions.
(i) Development of model-independent approaches for mitigating bias at the data level
(ii) Development of model-dependent approaches for mitigating bias at the model/algorithm and at the output level
(iii) Reconciling bias mitigation approaches with legal norms and legal theory
Legal exploration into the lawfulness of processing personal data for pre-processing debiasing purposes is about to be completed by ESR Ioanna Papageorgiou. Prior works on bias mitigation in different contexts have been surveyed by ESR Alaa Elobaid, and new techniques for NoBIAS’s ranking and classification use cases are currently being tested by ESRs Antonio Ferrara and Paula Reyero.
[O3]: Accounting for bias in results.
All ESRs have conducted extensive overviews of the literature in a rather large spectrum of multidisciplinary topics. Specifically the ESRs have looked into the following aspects;
(i) an ethical and legal perspective on accountability,
(ii) the field of eXplainable AI (XAI),
(iii) the issue of monitoring time-evolving AI models and their biases.
While a legal framework which encompasses the rights to information and an explanation is currently under development by ESR Alejandra Bringas Colmenarejo, the technical aspects of O3 are being explored through various ongoing use cases and experimental analyses on explainability and temporal bias mitigation by ESRs Xuan Zhao, Laura State, Ioannis Koulierakis, Carlos Mougan Navarro, and Seyed Siamak Ghodsi.
Other achievements of the network as of Dec’21 are listed next. The network has successfully organized various training programs including the onboarding week, a summer school, and the European AI regulation week, NoBIAS monthly colloquium talks. The ESRs have already published 7 research papers and attended 29 events including conferences, workshops, and panel discussions thereby contributing to dissemination and communication goals of NoBIAS. The network has garnered a lot of attention on its social media handle (#followers=267) and also on the NoBIAS website (#visitors=4974).
Each ESR started out exploring the problem from a specific direction and each acquired a much deeper understanding of the nature of the manifestation of bias in data within their individual area. The important but common questions that O1 ESRs have been working on are:
(i) What is bias?
(ii) How is bias created?
(iii) How can we detect bias?
To answer these questions, long ethnographic fieldwork has been conducted by ESR Miriam Fahimi, suitable technical frameworks have been identified/ developed, methodological innovations and testing are currently underway for various use cases by ESRs Kristen Marie Scott, Jose Manuel Alvarez, Simone Fabbrizzi and Mayra Russo.
[O2]: Mitigating bias in algorithms.
O2 ESRs focus on the following multi-disciplinary research directions.
(i) Development of model-independent approaches for mitigating bias at the data level
(ii) Development of model-dependent approaches for mitigating bias at the model/algorithm and at the output level
(iii) Reconciling bias mitigation approaches with legal norms and legal theory
Legal exploration into the lawfulness of processing personal data for pre-processing debiasing purposes is about to be completed by ESR Ioanna Papageorgiou. Prior works on bias mitigation in different contexts have been surveyed by ESR Alaa Elobaid, and new techniques for NoBIAS’s ranking and classification use cases are currently being tested by ESRs Antonio Ferrara and Paula Reyero.
[O3]: Accounting for bias in results.
All ESRs have conducted extensive overviews of the literature in a rather large spectrum of multidisciplinary topics. Specifically the ESRs have looked into the following aspects;
(i) an ethical and legal perspective on accountability,
(ii) the field of eXplainable AI (XAI),
(iii) the issue of monitoring time-evolving AI models and their biases.
While a legal framework which encompasses the rights to information and an explanation is currently under development by ESR Alejandra Bringas Colmenarejo, the technical aspects of O3 are being explored through various ongoing use cases and experimental analyses on explainability and temporal bias mitigation by ESRs Xuan Zhao, Laura State, Ioannis Koulierakis, Carlos Mougan Navarro, and Seyed Siamak Ghodsi.
Other achievements of the network as of Dec’21 are listed next. The network has successfully organized various training programs including the onboarding week, a summer school, and the European AI regulation week, NoBIAS monthly colloquium talks. The ESRs have already published 7 research papers and attended 29 events including conferences, workshops, and panel discussions thereby contributing to dissemination and communication goals of NoBIAS. The network has garnered a lot of attention on its social media handle (#followers=267) and also on the NoBIAS website (#visitors=4974).
Bias in AI-systems cannot be addressed only by computer scientists, but it rather requires an interdisciplinary approach and close collaboration with legal expert and social scientists, to make sure that societal origins of data and the legal limits of socio-technical systems are appropriately considered and new methods become usable and useful in practice. Society today lacks professionals and researchers that do not only vow for economic success, but have the capacity to embody ethics and legal behavior in AI algorithms that decide upon all our lives. NoBIAS will provide ESRs with such interdisciplinary training to instill in them such capacity. The manifestation of bias depends on the application per se, therefore our ESRs will be able to gain practical experience on real world applications spanning a variety of sectors from telecommunication, finance, pharmaceutical industry, marketing, media, software, and legal consultancy to broadly foster legal compliance and innovation.
Related ETN projects include analysis of Big Data (ETN Longpop, ID: 676060), privacy and usability (ETN Privacy.us 675730), risk analysis (ETN BigDataFinance, 675044), Big Data management (ETN BigStorage, 642963), or question answering from Big Data (ETN WDAqua, 642795). However, the focus in these projects lies on generating value from big data by new methods, and with some lesser focus on privacy. The issues arising with regard to bias, such as fairness, legality, or discrimination are not addressed.
Related ETN projects include analysis of Big Data (ETN Longpop, ID: 676060), privacy and usability (ETN Privacy.us 675730), risk analysis (ETN BigDataFinance, 675044), Big Data management (ETN BigStorage, 642963), or question answering from Big Data (ETN WDAqua, 642795). However, the focus in these projects lies on generating value from big data by new methods, and with some lesser focus on privacy. The issues arising with regard to bias, such as fairness, legality, or discrimination are not addressed.