IEEE Technology and Society Magazine - December 2019 - 40

by graphs (see Figures). Each graph shows the mean
of the dependent variable on the y axis and the different scenarios on the x axis to point out the differences
between the scenarios.

Sample
To determine the number of scenarios for the two studies, we performed power calculations to estimate the
total number of participants that we would need based
on the results of the pilot studies. Based on the power
calculations (effect size 0.4, a desired statistical power
of 0.8, and a probability level of 0.05) we aimed for a
total of 200 responses and determined that we could
run 3 scenarios.

Appendix B

Study 1 was distributed via the snowball method by
e-mail with an anonymous link to approximately 40 military personnel who further distributed the survey. This
method was used because we were not allowed to collect any personal information, such as e-mail or IP
addresses. Study 1 resulted in 327 responses of which
239 were complete valid responses and usable after the
data preprocessing. The 239 responses (227 male) of
study 1 consist of Dutch military (149 respondents) and
civilian (90 respondents) personnel working at the Dutch
Ministry of Defense (MOD). The number of respondents
per scenario ranged between 64 and 96. Study 2 was
distributed via the crowdsourcing platform Amazon
Mechanical Turk and 294 valid responses were collected. The 294 responses consist of civilian respondents
(168 male) and the number of respondents per scenario
ranged between 110 and 91.

Results

A military convoy is on its way to deliver supplies to
one of their units at a camp near Mosul in Iraq. The
commander has ordered an autonomous drone to
support the convoy in the air. The autonomous drone
scans the surroundings for enemy threats and carries
weapons for the defence of the convoy. When the
convoy is at a three-mile distance from the camp, the
autonomous drone detects a vehicle behind a mountain
range that is approaching the convoy at high speed.
The autonomous drone detects four people in the
car with large weapon-shaped objects and identifies
the driver of the vehicle as a known member of an
insurgency group. The autonomous drone attacks the
approaching vehicle which results in the death of all
four passengers, but also causes collateral damage by
killing five children that were playing nearby the road.
The above scenario represents the neutral agency condition in
which we do not provide any extra information on the weapon. In
the human operated scenarios, we replace words autonomous
drone with the words Human Operated drone and provided
no extra information on the weapon (note that the words are
highlighted in blue to show the distinction in this paper and
the respondents in the survey were shown scenarios in black
wording). In the high agency condition, we added the following
phrase to describe the agency characteristics: "The autonomous
drone independently deliberates between a series of options,
weighs the pros and cons, and decides to attack the approaching
vehicle,...." We purposely kept the changes to the scenarios to
a minimum so that we can attribute different results to those
changes and measure their effect.

We will report the results of both studies by describing
the results of the values blame, trust, harm, human dignity, confidence, expectations, support, fairness, and
anxiety for both studies in a descriptive manner. The
results of the values of human dignity, trust, anxiety,
and blame can be viewed in Figs. 1-8.

Study 1
The actions of Human Operated drones are perceived
as having more respect for human dignity than Autonomous Weapon Systems by military personnel and civilians working at the Dutch Ministry of Defense (Figure 1).
The respondents in Study 1 are more anxious about the
actions of Autonomous Weapon Systems than the
actions of human operated drones (Figure 2). We also
found that military personnel and civilians working at
the Dutch Ministry of Defense have more trust (Figure 3)

Mean Human Dignity Value

The default scenario that we used in the study reads as
follows:

50
40
30
20

36
28

10
0

Human
Operated
Weapon

Error Bars: +/- 1 SE

Neutral Agency High Agency
Autonomous Autonomous
Weapon
Weapon
Scenario

FIGURE 1. Human Dignity value study 1.

IEEE TECHNOLOGY AND SOCIETY MAGAZINE

∕

DECEMBER 2019

IEEE Technology and Society Magazine - December 2019

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - December 2019