IEEE Technology and Society Magazine - June 2017 - 70

worrisome; it is also the variety and uncertainty of situations that the system will face [30].
The complexity of a self-driving car's system architecture, including subsystems for the human interface,
route planning, environment perception and modeling,
and vehicle hardware actuators, all interconnected with
a coordination and control module [31], could generate
many outcomes that are difficult to anticipate. This is
further complicated by the variability of the design pathways that different car manufacturers are pursuing. The
volume of and interconnections between sensor data
that have to be processed may (arguably) be a bigger

Users already seem to have a
predisposition to develop an
over-reliance on digital outputs.

problem that the sheer amount of code, especially given
how such data must be processed in a short amount of
time in order for a car to react promptly enough. Smooth
and timely calibration across light detection and ranging
(LIDAR) or other sensors is essential and difficult [32].
The associated complexity is increased by potential variables such as vehicle-to-vehicle (V2V) communication,
cloud connectivity, and smart highways, all of which
could intensify the need to process vast quantities of
information almost instantaneously.
Some of the advantages anticipated for automated
cars are predicated on all vehicles being automated. If
cars with human drivers are allowed to mingle with
self-driving cars, an automated system will be much
more difficult to design and test. However, even if
human drivers are phased out, a collection of autonomous cars will still be part of a socio-technical system
of enormous complexity. The design, development,
and testing of swarms of robots is an area of research
that is relatively young [33], but it seems clear that
much progress will be required before a swarm of vehicles will be able to interact and operate safely [34]. The
emergence of "normal accidents" (also known as "system accidents") is likely unavoidable due to the interactive complexity and tight coupling of the involved
technical systems [35]. Intentional tampering (e.g.,
Volkswagen Diesel emissions tests) and hacking (e.g.,
Jeep Cherokee case) [36] are also legitimate sources
of concern.
Another consideration is whether and how the Eliza
Effect [37] might manifest itself; in other words, how

70

might users deceive themselves in terms of a self-driving
car's abilities? At times, an engineer's design choices
directly contribute to the likelihood of a user's self-deception; for example, utilizing human-like features on a robot
can lead users to inaccurately anthropomorphize the
technology [38]. Users already seem to have a predisposition to develop an over-reliance on digital outputs as, for
example, in the case of the Therac-25 radiation therapy
machine [39]. Significant harm, and even death, can
result from over-trust of computing technology, including
in cases where it has led to airplane crashes [40]. Along
these lines, a study by Robinette and colleagues indicates
that participants may place too much trust in a robot during a simulated emergency situation even when the robot
seems to be malfunctioning [41].
An added variable is that some companies are pursing design pathways that do not require the user to be
actively involved in the operation of the car [42]. The
underlying paternalistic logic of harm prevention may
be well-intended, but numerous unintended, and potentially disastrous, consequences could result. While the
list below highlights potential user-centered problems,
they have a direct bearing on the designer's decisions
and actions:
■ Will the lack of control over the car cause a user to
panic even when it is functioning normally?
■ Which types of important information might the
user miss? For example, will the user be attentive
enough to notice if someone is trying to hack into
the car?
■ Will the user's driving skill diminish over time [43]?
■ Will the user know how to respond if the car is "in
trouble" especially if there is no steering wheel or
other obvious means for intervening? Or if there is,
would grabbing the wheel place the user (and others) at greater risk than letting the system handle
the situation by itself?
With regard to the last point, a similar issue has emerged
for airline pilots when they are relying on or interacting
with an auto-pilot that may be malfunctioning [40], [44].
An overarching concern about normalization of deviance
with regard to user behavior can certainly emerge as
well [45]; in short, if they are not actively, cognitively
engaged in the vehicle's operation, users will become
less diligent about monitoring how it functions (to a
point where it can easily be imagined that users could
metaphorically if not literally become "asleep at the
wheel"). These and numerous other considerations,
integrally intertwined with the user's psychology and
behavior, must be taken into account by designers.

Relying on Standards
Many scholars, including De George, would stipulate that
the "standards of the time" can serve as a crucial means

IEEE TEchnology and SocIETy MagazInE

∕

june 2017



Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - June 2017

IEEE Technology and Society Magazine - June 2017 - Cover1
IEEE Technology and Society Magazine - June 2017 - Cover2
IEEE Technology and Society Magazine - June 2017 - 1
IEEE Technology and Society Magazine - June 2017 - 2
IEEE Technology and Society Magazine - June 2017 - 3
IEEE Technology and Society Magazine - June 2017 - 4
IEEE Technology and Society Magazine - June 2017 - 5
IEEE Technology and Society Magazine - June 2017 - 6
IEEE Technology and Society Magazine - June 2017 - 7
IEEE Technology and Society Magazine - June 2017 - 8
IEEE Technology and Society Magazine - June 2017 - 9
IEEE Technology and Society Magazine - June 2017 - 10
IEEE Technology and Society Magazine - June 2017 - 11
IEEE Technology and Society Magazine - June 2017 - 12
IEEE Technology and Society Magazine - June 2017 - 13
IEEE Technology and Society Magazine - June 2017 - 14
IEEE Technology and Society Magazine - June 2017 - 15
IEEE Technology and Society Magazine - June 2017 - 16
IEEE Technology and Society Magazine - June 2017 - 17
IEEE Technology and Society Magazine - June 2017 - 18
IEEE Technology and Society Magazine - June 2017 - 19
IEEE Technology and Society Magazine - June 2017 - 20
IEEE Technology and Society Magazine - June 2017 - 21
IEEE Technology and Society Magazine - June 2017 - 22
IEEE Technology and Society Magazine - June 2017 - 23
IEEE Technology and Society Magazine - June 2017 - 24
IEEE Technology and Society Magazine - June 2017 - 25
IEEE Technology and Society Magazine - June 2017 - 26
IEEE Technology and Society Magazine - June 2017 - 27
IEEE Technology and Society Magazine - June 2017 - 28
IEEE Technology and Society Magazine - June 2017 - 29
IEEE Technology and Society Magazine - June 2017 - 30
IEEE Technology and Society Magazine - June 2017 - 31
IEEE Technology and Society Magazine - June 2017 - 32
IEEE Technology and Society Magazine - June 2017 - 33
IEEE Technology and Society Magazine - June 2017 - 34
IEEE Technology and Society Magazine - June 2017 - 35
IEEE Technology and Society Magazine - June 2017 - 36
IEEE Technology and Society Magazine - June 2017 - 37
IEEE Technology and Society Magazine - June 2017 - 38
IEEE Technology and Society Magazine - June 2017 - 39
IEEE Technology and Society Magazine - June 2017 - 40
IEEE Technology and Society Magazine - June 2017 - 41
IEEE Technology and Society Magazine - June 2017 - 42
IEEE Technology and Society Magazine - June 2017 - 43
IEEE Technology and Society Magazine - June 2017 - 44
IEEE Technology and Society Magazine - June 2017 - 45
IEEE Technology and Society Magazine - June 2017 - 46
IEEE Technology and Society Magazine - June 2017 - 47
IEEE Technology and Society Magazine - June 2017 - 48
IEEE Technology and Society Magazine - June 2017 - 49
IEEE Technology and Society Magazine - June 2017 - 50
IEEE Technology and Society Magazine - June 2017 - 51
IEEE Technology and Society Magazine - June 2017 - 52
IEEE Technology and Society Magazine - June 2017 - 53
IEEE Technology and Society Magazine - June 2017 - 54
IEEE Technology and Society Magazine - June 2017 - 55
IEEE Technology and Society Magazine - June 2017 - 56
IEEE Technology and Society Magazine - June 2017 - 57
IEEE Technology and Society Magazine - June 2017 - 58
IEEE Technology and Society Magazine - June 2017 - 59
IEEE Technology and Society Magazine - June 2017 - 60
IEEE Technology and Society Magazine - June 2017 - 61
IEEE Technology and Society Magazine - June 2017 - 62
IEEE Technology and Society Magazine - June 2017 - 63
IEEE Technology and Society Magazine - June 2017 - 64
IEEE Technology and Society Magazine - June 2017 - 65
IEEE Technology and Society Magazine - June 2017 - 66
IEEE Technology and Society Magazine - June 2017 - 67
IEEE Technology and Society Magazine - June 2017 - 68
IEEE Technology and Society Magazine - June 2017 - 69
IEEE Technology and Society Magazine - June 2017 - 70
IEEE Technology and Society Magazine - June 2017 - 71
IEEE Technology and Society Magazine - June 2017 - 72
IEEE Technology and Society Magazine - June 2017 - 73
IEEE Technology and Society Magazine - June 2017 - 74
IEEE Technology and Society Magazine - June 2017 - 75
IEEE Technology and Society Magazine - June 2017 - 76
IEEE Technology and Society Magazine - June 2017 - 77
IEEE Technology and Society Magazine - June 2017 - 78
IEEE Technology and Society Magazine - June 2017 - 79
IEEE Technology and Society Magazine - June 2017 - 80
IEEE Technology and Society Magazine - June 2017 - 81
IEEE Technology and Society Magazine - June 2017 - 82
IEEE Technology and Society Magazine - June 2017 - 83
IEEE Technology and Society Magazine - June 2017 - 84
IEEE Technology and Society Magazine - June 2017 - 85
IEEE Technology and Society Magazine - June 2017 - 86
IEEE Technology and Society Magazine - June 2017 - 87
IEEE Technology and Society Magazine - June 2017 - 88
IEEE Technology and Society Magazine - June 2017 - Cover3
IEEE Technology and Society Magazine - June 2017 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2023
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2023
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2023
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2013
https://www.nxtbookmedia.com