IEEE Technology and Society Magazine - June 2021 - 76

Feature
areas have specific demographics that play the role
of proxy variables for ethnicity). These approaches
can also be criticized because they alter the model
of the world that an AI makes use of, instead of altering
how that AI perceives and acts on bias [17].
On a broader level, debiasing an algorithm's
output requires a specific definition of its context
and, as such, is difficult to achieve from a technical
perspective only. A myriad of lingering questions
remains to be answered: how much bias does an
algorithm need to encode to consider its outputs discriminating?
How can we reflect on the peculiarity of
the data on which these algorithms are operating-
data which often reflects the inequities of its time?
In short, a clearer definition of the relation between
algorithmic biases and discrimination is needed. We
argue that such a definition can only be provided
by a cross-disciplinary approach that takes legal,
social, and ethical considerations into account. In
response, in the next sections, we will engage critically
with related work from legal, social, and ethical
perspectives.
Legal perspective
Legislation designed to prevent discrimination
against particular groups of people that share one
or more protected attributes-namely protected
groups-receives the name of antidiscrimination
law. Antidiscrimination laws vary across countries.
For instance, European antidiscrimination legislation
is organized in directives, such as Directive
2000/43/EC against discrimination on grounds of
race and ethnic origin, or Chapter 3 of the EU Charter
of fundamental rights. Antidiscrimination laws in
the US are described in the Title VII of the Civil Rights
Act of 1964 and other federal and state statutes, supplemented
by court decisions. For instance, Title VII
prohibits discrimination in employment on the basis
of race, sex, national origin, and religion; and The
Equal Pay Act prohibits wage disparity based on sex
by employers and unions.
The main issues in trials related to discrimination
consist of determining [20]: 1) the relevant
population affected by the discrimination case,
and to which groups it should be compared; 2)
the discrimination measure that formalizes group
under-representation, e.g., disparate treatment or
disparate impact [18], [21]; and 3) the threshold
that constitutes prima facie evidence of discrimination.
Note that the three issues coincide with the
76
problems explored in the technical approaches
presented earlier. With respect to the last point, no
strict threshold has been laid down by the European
Union. In the U.S., the fourth-fifth rule from
the Equal Employment Opportunity Commission
(1978), which states that a job selection rate for
the protected group of less than 4/5 of the selection
rate for the unprotected group, is sometimes
used a prima facie evidence of an adverse impact.
The Castaneda rule, which states that the number
of people of the protected group selected from a
relevant population cannot be smaller than 3 standard
deviations the number expected in a random
selection, is also used [21]. Although such laws
can relieve discriminatory issues, more complex
scenarios can arise. For instance, Hildebrandt and
Koops [22] mention the legally gray area of price
discrimination, where consumers in different geographical
areas can be offered different prices
based on differences in average income.
More recent regulations, such as the general
data protection regulation (GDPR), have been
offered as a framework to alleviate some of the
enforcement problems of antidiscrimination law,
and include clauses on automated decision-making
related to procedural regularity and accountability,
introducing a right of explanation for all
individuals to obtain meaningful explanations of
the logic involved when automated decision making
takes place. However, these solutions often
assume white-box scenarios, which, as we have
seen, may be difficult to achieve technically, and
even when they are achieved, they may not necessarily
provide the answers sought to assess whether
discrimination is present or not. Generally, current
laws are badly equipped to address algorithmic discrimination
[21]. Leese [23], for instance, notes that
antidiscrimination frameworks typically follow the
establishment of a causal chain between indicators
on the theoretical level (e.g., sex or race) and their
representation in the population under scrutiny.
Data-driven analytics, however, create aggregates
of individual profiles, and as such are prone to
the production of arbitrary categories instead of
real communities. As such, even if data subjects
are granted procedural and relational explanations,
the question remains at which point potential
biases can reasonably be considered forms of
discrimination.
IEEE Technology and Society Magazine
IEEE Technology and Society Magazine - June 2021

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - June 2021
