IEEE Technology and Society Magazine - December 2019 - 21

on MHC tasks that have high likeli-
hoods of human error with deadly
unintended consequences.
Fundamentally this question of
meaningful control aligns with a
question faced every day by con-
trols engineers who must determine
which functions and roles automa-
tion/autonomy should have versus
what should be allocated to humans
when designing a complex system.
To answer such questions, engi-
neers and designers must know
the strengths and limitations of
humans as well as of the technolo-
gies, and such boundaries are not
always clear. It is often not clear
whether a technology or a human
should or could be in control from a
design standpoint, such as whether
self-driving cars are actually better
than humans across a number of
driving scenarios [4]. Such design
gaps inevitably lead to discussions
of ethical and social impacts of the
technology, especially in safety criti-
cal systems like transportation and
medical systems, as well as autono-
mous weapons.
To shed more light on these is-
sues, two vignettes of actual mili-
tary engagements are provided to
frame the MHC debate, which is
really a discussion about role alloca-
tion between humans and autono-
mous systems. I suggest that what is
needed is not a call for meaningful
human control of autonomous weap-
ons but rather a focus on meaningful
human certification of such systems.

The original target was a Yugo-
slav supply warehouse, about
500 yards from coordinates
were provided to the Air Force
by the U.S. Central Intelligence
Agency, in part due to faulty
maps and a flawed review pro-
cess that failed to catch the
error [12].

Operation Provide Comfort
In 1994, the United States
sought to stop Iraqi attacks
on Kurds in northern Iraq by
enforcing a no-fly zone through
Operation Provide Comfort.
The no-fly zone was patrolled
by U.S. Air Force F-15 fighters,
supported by an A i r b o r n e
Warning and Control System
(AWACS) aircraft. On April 14,
two U.S. Army Black Hawk
helicopters transported U.S.,
French, British and Turkish
commanders, as well as Kurd-
ish personnel across this
zone when the F-15 fighters
misidentified them as Iraqi
Hind helicopters. The fighters
launched air-to-air missiles,
killing all 26 onboard the
Black Hawks. One person on
the AWACS plane gave the
Black Hawks permission to
enter the no-fly zone. Howev-
er, due to a communication
breakdown, this information
was not shared with another
controller on the same AWACS
communicating with the F-15s.
One government accident
investigation found 130 caus-
al human errors for this single
accident [8].
These two case studies exemplify
the kinds of missions faced by mili-
tary operators in deciding when and
where to launch an airborne weap-
on with likely lethal consequenc-
es. The Chinese Embassy incident
represents what should have been
a relatively low uncertainty engage-

The Chinese Embassy
Bombing
During NATO's Operations Allied
Force campaign in Yugosla-
via in May of 1999, the Chinese
embassy in Belgrade was acci-
dentally bombed with a U.S. Air
Force GPS-guided Joint Direct
Attack Munition (JDAM). Three
Chinese journalists were killed,
with more than 20 people injured.
DECEMBER 2019

∕

IEEE TECHNOLOGY AND SOCIETY MAGAZINE

ment given the intended target was
static and known, while the Operation Provide Comfort case was one
of high uncertainty with a dynamic
and unknown potential threat. Un-
derstanding the relative level of
uncertainty is key to understanding
how much human control could and
should be exerted in these scenari-
os, and what the implications are for
the use of autonomous weapons in
these settings [3].

High Uncertainty
Engagements Are Outside
the Realm of Autonomous
Weapons - For Now
Unfortunately, the problems sur-
rounding the Operation Provide
Comfort incident are not new to the
military and even today are sadly all
too common. There are many very
similar incidents where a suspected
target emerges on or near a battle-
field, information about this target
is misinterpreted, and because of
the "fog of war" which includes in-
complete information, psychological
biases, and human error, a weapon
is launched, killing innocent civil-
ians or friendly forces [12].
One of the biggest problems with
high uncertainty targeting situa-
tions is time pressure. Identification
and classification of such targets is
always the highest priority, but when
targets are moving and perceived
as a threat, there is limited time for
those humans in control (the F-15
pilots in the Provide Comfort exam-
ple) to make a high-quality decision.
Research has shown that warfight-
ers under time pressure in the air
and on the ground are predisposed
to a number of psychological bias-
es that cause them not to evaluate
all relevant pieces of information,
which in many cases leads to disas-
trous outcomes [2], [11].
Autonomous weapons could, in
theory, aid human decision makers in
these time-critical targeting scenarios

21



IEEE Technology and Society Magazine - December 2019

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - December 2019

Contents
IEEE Technology and Society Magazine - December 2019 - Cover1
IEEE Technology and Society Magazine - December 2019 - Cover2
IEEE Technology and Society Magazine - December 2019 - 1
IEEE Technology and Society Magazine - December 2019 - Contents
IEEE Technology and Society Magazine - December 2019 - 3
IEEE Technology and Society Magazine - December 2019 - 4
IEEE Technology and Society Magazine - December 2019 - 5
IEEE Technology and Society Magazine - December 2019 - 6
IEEE Technology and Society Magazine - December 2019 - 7
IEEE Technology and Society Magazine - December 2019 - 8
IEEE Technology and Society Magazine - December 2019 - 9
IEEE Technology and Society Magazine - December 2019 - 10
IEEE Technology and Society Magazine - December 2019 - 11
IEEE Technology and Society Magazine - December 2019 - 12
IEEE Technology and Society Magazine - December 2019 - 13
IEEE Technology and Society Magazine - December 2019 - 14
IEEE Technology and Society Magazine - December 2019 - 15
IEEE Technology and Society Magazine - December 2019 - 16
IEEE Technology and Society Magazine - December 2019 - 17
IEEE Technology and Society Magazine - December 2019 - 18
IEEE Technology and Society Magazine - December 2019 - 19
IEEE Technology and Society Magazine - December 2019 - 20
IEEE Technology and Society Magazine - December 2019 - 21
IEEE Technology and Society Magazine - December 2019 - 22
IEEE Technology and Society Magazine - December 2019 - 23
IEEE Technology and Society Magazine - December 2019 - 24
IEEE Technology and Society Magazine - December 2019 - 25
IEEE Technology and Society Magazine - December 2019 - 26
IEEE Technology and Society Magazine - December 2019 - 27
IEEE Technology and Society Magazine - December 2019 - 28
IEEE Technology and Society Magazine - December 2019 - 29
IEEE Technology and Society Magazine - December 2019 - 30
IEEE Technology and Society Magazine - December 2019 - 31
IEEE Technology and Society Magazine - December 2019 - 32
IEEE Technology and Society Magazine - December 2019 - 33
IEEE Technology and Society Magazine - December 2019 - 34
IEEE Technology and Society Magazine - December 2019 - 35
IEEE Technology and Society Magazine - December 2019 - 36
IEEE Technology and Society Magazine - December 2019 - 37
IEEE Technology and Society Magazine - December 2019 - 38
IEEE Technology and Society Magazine - December 2019 - 39
IEEE Technology and Society Magazine - December 2019 - 40
IEEE Technology and Society Magazine - December 2019 - 41
IEEE Technology and Society Magazine - December 2019 - 42
IEEE Technology and Society Magazine - December 2019 - 43
IEEE Technology and Society Magazine - December 2019 - 44
IEEE Technology and Society Magazine - December 2019 - 45
IEEE Technology and Society Magazine - December 2019 - 46
IEEE Technology and Society Magazine - December 2019 - 47
IEEE Technology and Society Magazine - December 2019 - 48
IEEE Technology and Society Magazine - December 2019 - 49
IEEE Technology and Society Magazine - December 2019 - 50
IEEE Technology and Society Magazine - December 2019 - 51
IEEE Technology and Society Magazine - December 2019 - 52
IEEE Technology and Society Magazine - December 2019 - 53
IEEE Technology and Society Magazine - December 2019 - 54
IEEE Technology and Society Magazine - December 2019 - 55
IEEE Technology and Society Magazine - December 2019 - 56
IEEE Technology and Society Magazine - December 2019 - 57
IEEE Technology and Society Magazine - December 2019 - 58
IEEE Technology and Society Magazine - December 2019 - 59
IEEE Technology and Society Magazine - December 2019 - 60
IEEE Technology and Society Magazine - December 2019 - 61
IEEE Technology and Society Magazine - December 2019 - 62
IEEE Technology and Society Magazine - December 2019 - 63
IEEE Technology and Society Magazine - December 2019 - 64
IEEE Technology and Society Magazine - December 2019 - 65
IEEE Technology and Society Magazine - December 2019 - 66
IEEE Technology and Society Magazine - December 2019 - 67
IEEE Technology and Society Magazine - December 2019 - 68
IEEE Technology and Society Magazine - December 2019 - 69
IEEE Technology and Society Magazine - December 2019 - 70
IEEE Technology and Society Magazine - December 2019 - 71
IEEE Technology and Society Magazine - December 2019 - 72
IEEE Technology and Society Magazine - December 2019 - 73
IEEE Technology and Society Magazine - December 2019 - 74
IEEE Technology and Society Magazine - December 2019 - 75
IEEE Technology and Society Magazine - December 2019 - 76
IEEE Technology and Society Magazine - December 2019 - 77
IEEE Technology and Society Magazine - December 2019 - 78
IEEE Technology and Society Magazine - December 2019 - 79
IEEE Technology and Society Magazine - December 2019 - 80
IEEE Technology and Society Magazine - December 2019 - 81
IEEE Technology and Society Magazine - December 2019 - 82
IEEE Technology and Society Magazine - December 2019 - 83
IEEE Technology and Society Magazine - December 2019 - 84
IEEE Technology and Society Magazine - December 2019 - Cover3
IEEE Technology and Society Magazine - December 2019 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2013
https://www.nxtbookmedia.com