IEEE Technology and Society Magazine - June 2018 - 82

automated software systems promises to notably in crease our quality of life. In fact, it will relieve us from
a number of boring physical and mental activities, and
enable us to do much more interesting things that, say,
driving and having to decide on a route to a destination. In other words, it will make it possible to satisfy
needs at the highest levels of the Maslow pyramid [9].
However, as we will elaborate in the following, algorithmic governance also comes with a number of potential
dangers, and raises the risk of actually making us "dumber" rather than smarter.

Possible Perils of Algorithmic Governance
The primary source of all peril related to algorithmic governance is that, very often, software and algorithms are
designed as "black boxes" with little understanding of
how they actually work. This lack of understanding not
only involves the final users (i.e., in case of smart cities,
all citizens) but quite often also the stakeholders (e.g.,
municipalities and decision makers), and the developers, if they do not care enough to understand how these
algorithms operate and have a discussion with designers
and stakeholders.
This limited understanding of algorithms, from what
we can see so far, does not prevent people from relying
on them for various activities. For instance, people fully
rely on Google's search results, even without knowing
anything about its underlying PageRank algorithms. This
is the key difference between algorithms and other classes of technologies: we are using them and relying on
their judgement and suggestions without knowing the
reasons we are being suggested something. In Smart Cities, a pervasive environment governed by algorithms will
make us apparently smarter in our capabilities, but to
some extent will also make us "dumber" in that our
actions will no longer be conscious. For instance, roaming in an unknown city with a paper map enables us to
absorb the basic fabric of the city and, on this basis, to
consciously decide what route to take. Conversely, roaming with a GPS navigator does not require knowing anything about the city structure, and we are willing to
accept route suggestions without investigating further the
reason a certain route has been selected. In other words,
our notion of "dumber" here is in the sense of potentially
losing the ability to make good judgements in some situations, reducing discernment, and lazily deferring to the
algorithm. The overreliance on algorithms, which are
capable (unlike other technologies) of making judgements or decisions on our behalf, without our questioning or thinking them through, will make us increasingly
depend on the algorithms in our everyday life, instead of
depending on human rationale and our own reasoning.
One might say that the above issue will not cause
any trouble because algorithms will be developed to

82

serve citizens, for their own good and in accord with
rules and policies of the municipality, and that such
rules will be made transparent. Yet, as we elaborate in
the following: 1) decisions made by algorithms may be
biased and have inaccurate information; 2) there is risk
that someone manage to make intentional misuse of
algorithms without no one noticing; 3) politics and decision makers themselves may end up having little clue as
to the actual working of algorithms, thus losing power.
With regard to the first issue, we normally trust the
developers of the technologies we use, knowing that
societal norms, regulations, and reputation largely help
to protect that trust. However, intentional or (more
often) unintentional bias, errors, or incomplete information, can subtly hide in algorithms, along with biases in
behavior due to values (right or wrong) that developers
or creators of the technology might have. This issue
could be exacerbated by the increasing influence and
role of technology in daily decision-making. For example, bias can creep into a machine learning algorithm
[10] due to wrong or inadequate datasets, rather than
any malice on the part of the developers (e.g., a system
that is racially biased against people with black-sounding names when judging potential for crime, or software
that could not do face recognition well for people with
dark skin, or unfair judgments on insurance or loans, all
due to issues with data used in training the algorithms).
Also, what is a "normal person" or "normal behavior" is
often contextual and hard to define - which can influence default settings in software [11], e.g., gender bias
in default character profiles in some games, or someone struggling to read a critical alert message in a nonnative language, or what an algorithm portrays as
"normal looking."
Algorithms also could have incomplete information
in decision-making or lack knowledge of exceptional
cases even if developers do endeavor to be comprehensive. In some cases, even if tempted to doubt the algorithm, profitability in following the algorithm's decision
can be high, but not necessarily by virtue of how good
the algorithm is. For instance, with smart algorithms
performing sentencing, or the "robotization of justice"
[12], by deciding against an algorithm's position for guilt,
one personally takes on the responsibility for a possible
second offense. In any case, even if the thought that
algorithms can do better than humans is becoming prevalent, and despite the recent breakthroughs in deep
learning, for neural networks to achieve the computational level of a human brain, neural networks still need
far more power [13].
With regard to the second issue, independently of the
reliability and trustability of algorithms, there is an issue
of who is devoted to managing such algorithms and
of how we can trust those people to manage them. For

IEEE Technology and Society Magazine

∕

JUNE 2018



Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - June 2018

Contents
IEEE Technology and Society Magazine - June 2018 - Cover1
IEEE Technology and Society Magazine - June 2018 - Cover2
IEEE Technology and Society Magazine - June 2018 - 1
IEEE Technology and Society Magazine - June 2018 - Contents
IEEE Technology and Society Magazine - June 2018 - 3
IEEE Technology and Society Magazine - June 2018 - 4
IEEE Technology and Society Magazine - June 2018 - 5
IEEE Technology and Society Magazine - June 2018 - 6
IEEE Technology and Society Magazine - June 2018 - 7
IEEE Technology and Society Magazine - June 2018 - 8
IEEE Technology and Society Magazine - June 2018 - 9
IEEE Technology and Society Magazine - June 2018 - 10
IEEE Technology and Society Magazine - June 2018 - 11
IEEE Technology and Society Magazine - June 2018 - 12
IEEE Technology and Society Magazine - June 2018 - 13
IEEE Technology and Society Magazine - June 2018 - 14
IEEE Technology and Society Magazine - June 2018 - 15
IEEE Technology and Society Magazine - June 2018 - 16
IEEE Technology and Society Magazine - June 2018 - 17
IEEE Technology and Society Magazine - June 2018 - 18
IEEE Technology and Society Magazine - June 2018 - 19
IEEE Technology and Society Magazine - June 2018 - 20
IEEE Technology and Society Magazine - June 2018 - 21
IEEE Technology and Society Magazine - June 2018 - 22
IEEE Technology and Society Magazine - June 2018 - 23
IEEE Technology and Society Magazine - June 2018 - 24
IEEE Technology and Society Magazine - June 2018 - 25
IEEE Technology and Society Magazine - June 2018 - 26
IEEE Technology and Society Magazine - June 2018 - 27
IEEE Technology and Society Magazine - June 2018 - 28
IEEE Technology and Society Magazine - June 2018 - 29
IEEE Technology and Society Magazine - June 2018 - 30
IEEE Technology and Society Magazine - June 2018 - 31
IEEE Technology and Society Magazine - June 2018 - 32
IEEE Technology and Society Magazine - June 2018 - 33
IEEE Technology and Society Magazine - June 2018 - 34
IEEE Technology and Society Magazine - June 2018 - 35
IEEE Technology and Society Magazine - June 2018 - 36
IEEE Technology and Society Magazine - June 2018 - 37
IEEE Technology and Society Magazine - June 2018 - 38
IEEE Technology and Society Magazine - June 2018 - 39
IEEE Technology and Society Magazine - June 2018 - 40
IEEE Technology and Society Magazine - June 2018 - 41
IEEE Technology and Society Magazine - June 2018 - 42
IEEE Technology and Society Magazine - June 2018 - 43
IEEE Technology and Society Magazine - June 2018 - 44
IEEE Technology and Society Magazine - June 2018 - 45
IEEE Technology and Society Magazine - June 2018 - 46
IEEE Technology and Society Magazine - June 2018 - 47
IEEE Technology and Society Magazine - June 2018 - 48
IEEE Technology and Society Magazine - June 2018 - 49
IEEE Technology and Society Magazine - June 2018 - 50
IEEE Technology and Society Magazine - June 2018 - 51
IEEE Technology and Society Magazine - June 2018 - 52
IEEE Technology and Society Magazine - June 2018 - 53
IEEE Technology and Society Magazine - June 2018 - 54
IEEE Technology and Society Magazine - June 2018 - 55
IEEE Technology and Society Magazine - June 2018 - 56
IEEE Technology and Society Magazine - June 2018 - 57
IEEE Technology and Society Magazine - June 2018 - 58
IEEE Technology and Society Magazine - June 2018 - 59
IEEE Technology and Society Magazine - June 2018 - 60
IEEE Technology and Society Magazine - June 2018 - 61
IEEE Technology and Society Magazine - June 2018 - 62
IEEE Technology and Society Magazine - June 2018 - 63
IEEE Technology and Society Magazine - June 2018 - 64
IEEE Technology and Society Magazine - June 2018 - 65
IEEE Technology and Society Magazine - June 2018 - 66
IEEE Technology and Society Magazine - June 2018 - 67
IEEE Technology and Society Magazine - June 2018 - 68
IEEE Technology and Society Magazine - June 2018 - 69
IEEE Technology and Society Magazine - June 2018 - 70
IEEE Technology and Society Magazine - June 2018 - 71
IEEE Technology and Society Magazine - June 2018 - 72
IEEE Technology and Society Magazine - June 2018 - 73
IEEE Technology and Society Magazine - June 2018 - 74
IEEE Technology and Society Magazine - June 2018 - 75
IEEE Technology and Society Magazine - June 2018 - 76
IEEE Technology and Society Magazine - June 2018 - 77
IEEE Technology and Society Magazine - June 2018 - 78
IEEE Technology and Society Magazine - June 2018 - 79
IEEE Technology and Society Magazine - June 2018 - 80
IEEE Technology and Society Magazine - June 2018 - 81
IEEE Technology and Society Magazine - June 2018 - 82
IEEE Technology and Society Magazine - June 2018 - 83
IEEE Technology and Society Magazine - June 2018 - 84
IEEE Technology and Society Magazine - June 2018 - 85
IEEE Technology and Society Magazine - June 2018 - 86
IEEE Technology and Society Magazine - June 2018 - 87
IEEE Technology and Society Magazine - June 2018 - 88
IEEE Technology and Society Magazine - June 2018 - Cover3
IEEE Technology and Society Magazine - June 2018 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2023
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2013
https://www.nxtbookmedia.com