IEEE Technology and Society Magazine - June 2021 - 73

* Bias in modeling: Bias may be deliberately
introduced, e.g., through smoothing or regularization
parameters to mitigate or compensate
for bias in the data, which is called algorithmic
processing bias, or introduced while modeling
in cases with the usage of objective categories
to make subjective judgments, which is called
algorithmic focus bias.
* Bias in training: Algorithms learn to make decisions
or predictions based on data sets that
often contain past decisions. If a data set used
for training purposes reflects existing prejudices,
algorithms will very likely learn to make
the same biased decisions. Moreover, if the data
do not correctly represent the characteristics of
different populations, representing an unequal
ground truth, it may result in biased algorithmic
decisions.
* Bias in usage: Algorithms can result in bias
when they are used in a situation for which
they were not intended. An algorithm utilized
to predict a particular outcome in a given population
can lead to inaccurate results when
applied to a different population-a form of
transfer context bias. Further, the potential misinterpretation
of an algorithm's outputs can
lead to biased actions through what is called
interpretation bias.
A significant amount of literature focuses on
forms of bias that may or may not lead to discriminatory
outcomes, i.e., the relationship between bias
and discrimination is not always clear or understood.
Most literature assumes that systems free from
biases do not discriminate, hence, reducing or eliminating
biases reduces or eliminates the potential for
discrimination. However, whether an algorithm can
be considered discriminatory or not depends on the
context in which it is being deployed and the task it
is intended to perform. For instance, consider a possible
case of algorithmic bias in usage, in which an
algorithm is biased toward hiring young people. At
first glance, it can be considered that the algorithm
is discriminating against older people. However, this
(biased) algorithm should only be considered to discriminate
if the context in which it is intended to be
deployed does not justify hiring more young people
than older people. Therefore, statistically reductionist
approaches, such as estimating the ratio between
younger and older people hired, are insufficient to
June 2021
attest whether the algorithm is discriminating without
considering this socially and politically fraught
context; it remains ethically unclear where we need
to draw the line between biased and discriminating
outcomes. Therefore, AI and technical researchers
often: 1) use discrimination and bias as equivalent
or 2) focus on measuring biases without actually
attending to the problem of whether or not there is
discrimination. Our aim, in the below, is to disentangle
some of these issues.
Measuring biases
To assess whether an algorithm is
free from
biases, there is a need to analyze the entirety of the
algorithmic process. This entails
first confirming
that the algorithm's underlying assumptions and its
modeling are not biased; second, that its training
and test data do not include biases and prejudices;
and finally, that it is adequate to make decisions for
that specific context and task. More often than not,
however, we do not have access to this information.
A number of issues prevent such an analysis. The
data used to train a model, for instance, is typically
protected since it contains personal information,
rendering the task of attesting training bias impossible.
Access to the algorithm's source code might
also be restricted to the general public, removing
the possibility of identifying modeling biases. This is
common as algorithms are valuable private assets of
companies. Third, the specifics of where and how
the algorithm will be deployed might be unknown
to an auditor. Depending on what is available, different
types of bias attesting might be possible, both in
terms of the process and in terms of the metrics used
to measure it.
Procedural versus relational approaches
We can distinguish between two general
approaches to measure bias: 1) procedural
approaches, which focus on identifying biases in the
decision-making the process of an algorithm [6] and
2) relational approaches, which focus on identifying
(and preventing) biased decisions in the data set
or algorithmic output. Although ensuring unbiased
outcomes is useful to attest whether a specific algorithm
has a discriminatory impact on a population,
focusing on the algorithmic process itself can help
yield insights about the reason why it happened in
the first place.
73
IEEE Technology and Society Magazine - June 2021

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - June 2021
