IEEE Technology and Society Magazine - December 2019 - 26

for more diplomatic solutions,
which is why a focus on the stra-
tegic certification level is just as
important as the design certifica-
tion level.
Given the increasing complex-
ity of warfare and reduced time
scales, we need to recognize that
we are often asking warfighters to
make life and death decisions that
they are ill-suited for, with high prob-
abilities of errors. No combatant
who kills anyone accidentally, mili-
tary or civilian, ever truly recovers
from psychological suffering; nor do
the families of those killed. While
autonomous weapons technology
for dynamic environments is not
ready now, we owe it to both those
innocent people that could be killed
by human error and the pilots likely
to make mistakes in such settings
to keep working towards a balanced
role allocation in weapons systems
that at least makes fewer mistakes
than humans. The key to this future
is meaningful human certification of
autonomous weapons, not insisting
on an illusory concept of meaningful
human control.

REVIEW ESSAY

.bookdepository.com/Living-with-Robots
-Richard-Pak/9780128153673.
[6] I. Evtimov, K. Eykholt, E. Fernandes, T.
Kohno, B. Li et al., "Robust physical-world
attacks on deep learning models," arXiv
preprint 1707.08945, 2017.
[7] F uture of Life Institute, "Autono-
mous weapons: An open letter from AI
and robotics researchers," 2015; http://
futureoflife.org/open-letter-autonomous
-weapons/.
[8] GAO, Operation Provide Comfort:
Review of Air Force Investigation of Black
Hawk Fratricide Incident. (G AO/OSI-
9804). Washington, DC: U.S. Government
Accounting Office, 1997.
[9] R.J. Jagacinski and J.M. Flach, Control Theory for Humans: Quantitative
Approaches to Modeling Performance. New
Jersey, U.S.A.: Lawrence Erlbaum, 2003.
[10] G. Marcus, "Deep learning: A critical
appraisal," arXiv., 2018
[11] R. Parasuraman and D.H. Manzey, "Com-
placency and bias in human use of automa-
tion: An attentional integration," Human Factors, vol. 52, no. 3, pp. 381-410, 2010.
[12] R.E. Rasmussen, The Wrong Target. Nor-
folk, VA: Joint Forces Staff College, 2007.
[13] J.A. Ratches, (2011). Review of current
aided/automatic target acquisition technol-
ogy for military target acquisition tasks.
Optical Engineering, vol. 50, no. 7, 2011;
https://doi.org/10.1117/1.3601879.
[14] M. Sharif, S. Bhagavatula, L. Bauer,
and M.K. Reiter, "Accessorize to a crime:
Real and stealthy attacks on state-of-the-art
face recognition," presented at ACM SIG-
SAC Conf. Computer and Communications
Security (Vienna, Austria), 2016.

Deborah Johnson, Jason Borenstein,
and Charles Risio provided feedback
that substantially improved this paper.

Author Information
The author is Professor in the De-
partment of Electrical and Computer
Engineering at Duke University, Dur-
ham, NC. Email: mary.cummings@
duke.edu.

References
[1] M.A. Alcorn, Q. Li, Z. Gong, C. Wang,
L. Mair, W.S. Ku, and A. Nguyen, "Strike (with) a
Pose: Neural Networks Are Easily Fooled by
Strange Poses of Familiar Objects," arXiv.
org, 2018; https://arxiv.org/abs/1811.11553.
[2] M.L. Cummings, "Automation bias in
intelligent time critical decision support
systems," presented at AIAA 3rd Intelligent
Systems Conf., Chicago, IL, 2004.
[3] M.L. Cummings, "Artificial Intelligence and
the future of warfare," chathamhouse.org, 2017.
[4] M.L. Cummings, "Adaptation of licensing
examinations to the certification of autono-
mous systems," in Safe, Autonomous and
Intelligent Vehicles, H. Yu, X. Li, R. Murray, S.
Ramesh, and C.J. Tomlin, Eds. Basel, Switzer-
land: Springer, 2019, pp. 145-162.
[5] M.L. Cummings and D. Britton, "Liv-
ing with robots: Emerging issues on the
psychological and social implications
of robotics," R. Pak, E.d. Visser, and E.
Rovira, Eds. Elsevier, 2020; https://www

(continued from page 16)

As the Ghost Shirts rebel, the
protagonist-scientist Paul Proteus sits
near the banks of the Ilium river and
watches the "flush toilets [go] with
the automatic lathe machines." The
scene reminds of other responses to
the consolidation of power. Paul may
well have watched the Luddite tex-
tile-workers of 1810s Nottingham,
war-weary physicists in 1950s Ameri-
ca, or tech workers of 2010s Silicon
Valley [8]. After all, "eternal vigilance
is the price of efficiency" [5, pp. 73].

26

Acknowledgment

Author Information

Condition Have Failed. New Haven, CT:
Yale Univ. Press, 1998.
[5] K. Vonnegut, Player Piano. Dial, 1999.
[6] T. Wu, The Master Switch: The Rise
and Fall of Information Empires. Vintage,
2010.
[7] S. Zuboff, The Age of Surveillance
Capitalism: The Fight for the Future at
the New Frontier of Power. Profile, 2019.
[8] J. Moore, "Towards a more repre-
sentative politics in the ethics of com-
puter science" in Proc. Conf. Fairness,
Accountability, and Transparency - FAT*
'20, Barcelona, Spain, to be published,
2020.

The author is with the the University
of Washington, Seattle, WA. Email:
jared@jaredmoore.org.

References
[1] E. Anderson, "Private government: How
employers rule our lives (and why we don't
talk about it)," vol. 44. Princeton, NJ: Princ-
eton Univ. Press, 2019.
[2] A. Huxley, Brave New World. Ernst Klett
Sprachen, 2007.
[3] G. Orwell, 1984. HMH Books, 1983.
[4] J.C. Scott, Seeing Like a State: How
Certain Schemes to Improve the Human

IEEE TECHNOLOGY AND SOCIETY MAGAZINE

∕

DECEMBER 2019


https://www.bookdepository.com/es/Living-with-Robots-Richard-Pak/9780128153673 https://www.bookdepository.com/es/Living-with-Robots-Richard-Pak/9780128153673 https://futureoflife.org/open-letter-autonomous-weapons/ https://futureoflife.org/open-letter-autonomous-weapons/ https://futureoflife.org/open-letter-autonomous-weapons/ https://www.arxiv.org/abs/1811.11553 http://www.chathamhouse.org https://www.doi.org/10.1117/1.3601879 https://www.bookdepository.com/es/Living-with-Robots-Richard-Pak/9780128153673

IEEE Technology and Society Magazine - December 2019

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - December 2019

Contents
IEEE Technology and Society Magazine - December 2019 - Cover1
IEEE Technology and Society Magazine - December 2019 - Cover2
IEEE Technology and Society Magazine - December 2019 - 1
IEEE Technology and Society Magazine - December 2019 - Contents
IEEE Technology and Society Magazine - December 2019 - 3
IEEE Technology and Society Magazine - December 2019 - 4
IEEE Technology and Society Magazine - December 2019 - 5
IEEE Technology and Society Magazine - December 2019 - 6
IEEE Technology and Society Magazine - December 2019 - 7
IEEE Technology and Society Magazine - December 2019 - 8
IEEE Technology and Society Magazine - December 2019 - 9
IEEE Technology and Society Magazine - December 2019 - 10
IEEE Technology and Society Magazine - December 2019 - 11
IEEE Technology and Society Magazine - December 2019 - 12
IEEE Technology and Society Magazine - December 2019 - 13
IEEE Technology and Society Magazine - December 2019 - 14
IEEE Technology and Society Magazine - December 2019 - 15
IEEE Technology and Society Magazine - December 2019 - 16
IEEE Technology and Society Magazine - December 2019 - 17
IEEE Technology and Society Magazine - December 2019 - 18
IEEE Technology and Society Magazine - December 2019 - 19
IEEE Technology and Society Magazine - December 2019 - 20
IEEE Technology and Society Magazine - December 2019 - 21
IEEE Technology and Society Magazine - December 2019 - 22
IEEE Technology and Society Magazine - December 2019 - 23
IEEE Technology and Society Magazine - December 2019 - 24
IEEE Technology and Society Magazine - December 2019 - 25
IEEE Technology and Society Magazine - December 2019 - 26
IEEE Technology and Society Magazine - December 2019 - 27
IEEE Technology and Society Magazine - December 2019 - 28
IEEE Technology and Society Magazine - December 2019 - 29
IEEE Technology and Society Magazine - December 2019 - 30
IEEE Technology and Society Magazine - December 2019 - 31
IEEE Technology and Society Magazine - December 2019 - 32
IEEE Technology and Society Magazine - December 2019 - 33
IEEE Technology and Society Magazine - December 2019 - 34
IEEE Technology and Society Magazine - December 2019 - 35
IEEE Technology and Society Magazine - December 2019 - 36
IEEE Technology and Society Magazine - December 2019 - 37
IEEE Technology and Society Magazine - December 2019 - 38
IEEE Technology and Society Magazine - December 2019 - 39
IEEE Technology and Society Magazine - December 2019 - 40
IEEE Technology and Society Magazine - December 2019 - 41
IEEE Technology and Society Magazine - December 2019 - 42
IEEE Technology and Society Magazine - December 2019 - 43
IEEE Technology and Society Magazine - December 2019 - 44
IEEE Technology and Society Magazine - December 2019 - 45
IEEE Technology and Society Magazine - December 2019 - 46
IEEE Technology and Society Magazine - December 2019 - 47
IEEE Technology and Society Magazine - December 2019 - 48
IEEE Technology and Society Magazine - December 2019 - 49
IEEE Technology and Society Magazine - December 2019 - 50
IEEE Technology and Society Magazine - December 2019 - 51
IEEE Technology and Society Magazine - December 2019 - 52
IEEE Technology and Society Magazine - December 2019 - 53
IEEE Technology and Society Magazine - December 2019 - 54
IEEE Technology and Society Magazine - December 2019 - 55
IEEE Technology and Society Magazine - December 2019 - 56
IEEE Technology and Society Magazine - December 2019 - 57
IEEE Technology and Society Magazine - December 2019 - 58
IEEE Technology and Society Magazine - December 2019 - 59
IEEE Technology and Society Magazine - December 2019 - 60
IEEE Technology and Society Magazine - December 2019 - 61
IEEE Technology and Society Magazine - December 2019 - 62
IEEE Technology and Society Magazine - December 2019 - 63
IEEE Technology and Society Magazine - December 2019 - 64
IEEE Technology and Society Magazine - December 2019 - 65
IEEE Technology and Society Magazine - December 2019 - 66
IEEE Technology and Society Magazine - December 2019 - 67
IEEE Technology and Society Magazine - December 2019 - 68
IEEE Technology and Society Magazine - December 2019 - 69
IEEE Technology and Society Magazine - December 2019 - 70
IEEE Technology and Society Magazine - December 2019 - 71
IEEE Technology and Society Magazine - December 2019 - 72
IEEE Technology and Society Magazine - December 2019 - 73
IEEE Technology and Society Magazine - December 2019 - 74
IEEE Technology and Society Magazine - December 2019 - 75
IEEE Technology and Society Magazine - December 2019 - 76
IEEE Technology and Society Magazine - December 2019 - 77
IEEE Technology and Society Magazine - December 2019 - 78
IEEE Technology and Society Magazine - December 2019 - 79
IEEE Technology and Society Magazine - December 2019 - 80
IEEE Technology and Society Magazine - December 2019 - 81
IEEE Technology and Society Magazine - December 2019 - 82
IEEE Technology and Society Magazine - December 2019 - 83
IEEE Technology and Society Magazine - December 2019 - 84
IEEE Technology and Society Magazine - December 2019 - Cover3
IEEE Technology and Society Magazine - December 2019 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2013
https://www.nxtbookmedia.com