IEEE Technology and Society Magazine - June 2021 - 72

Feature
Bias and Discrimination in
AI: A Cross-Disciplinary
Perspective
Xavier Ferrer, Tom van Nuenen,
Jose M. Such, Mark Coté, and Natalia Criado
King's College London, London WC2R 2LS, U.K.
 Operating at a large scale and impacting large
groups of people, automated systems can make consequential
and sometimes contestable decisions.
Automated decisions can impact a range of phenomena,
from credit scores to insurance payouts
to health evaluations. These forms of automation
can become problematic when they place certain
groups or people at a systematic disadvantage.
These are cases of discrimination-which is legally
defined as the unfair or unequal treatment of an individual
(or group) based on certain protected characteristics
(also known as protected attributes) such as
income, education, gender, or ethnicity. When the
unfair treatment is caused by automated decisions,
usually taken by intelligent agents or other AI-based
systems, the topic of digital discrimination arises.
Digital discrimination is prevalent in a diverse range
of fields, such as in risk assessment systems for policing
and credit scores [1], [2].
Digital discrimination is becoming a serious problem,
as more and more decisions are delegated to
systems increasingly based on artificial intelligence
(AI) techniques such as machine learning. Although
a significant amount of research has been undertaken
from different disciplinary angles to understand this
challenge-from computer science to law to sociology-
none of these fields have been able to resolve
the problem on their own terms. For instance, computational
methods to verify and certify bias-free data
sets and algorithms do not account for socio-cultural
or ethical complexities, and do not distinguish
between bias and discrimination. Both of these terms
have a technical inflection, but are predicated on
legal and ethical principles [3].
In this article, we propose a synergistic approach
that allows us to explore bias and discrimination in
AI by supplementing technical literature with social,
legal, and ethical perspectives. Through a critical survey
of a synthesis of related literature, we compare
and evaluate the sometimes contradictory priorities
within these fields, and discuss how disciplines
might collaborate to resolve the problem. We also
highlight a number of interdisciplinary challenges to
attest and address discrimination in AI.
Digital Object Identifier 10.1109/MTS.2021.3056293
Date of current version: 3 June 2021.
72
0278-0097/21©2021 IEEE
Bias and discrimination
Technical literature in the area of discrimination
typically refers to the related issue of bias. Yet,
despite playing an important role in discriminatory
processes, bias does not necessarily lead to discrimination.
Bias means a deviation from the standard,
sometimes necessary to identify the existence of
some statistical patterns in the data or language used
[4], [5]. Classifying and finding differences between
instances would be impossible without bias.
In this article, we follow the most common definition
of bias used in the literature and focus on the
problematic instances of bias that may lead to discrimination
by AI-based automated-decision making
systems. Three main, well-known causes for bias
have been distinguished [4] as follows:
IEEE Technology and Society Magazine
IEEE Technology and Society Magazine - June 2021

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - June 2021
