IEEE Technology and Society Magazine - March 2018 - 57

On the other extreme, we find existing semi-autonomous weapons such as the unmanned aerial vehicles
like the Reaper drones. Although these aircraft are highly automated, "they are not considered to be autonomous because they are still operated under human
supervision and direct control" [11]. They are automated
and not fully autonomous because, without human
intervention, they can only perform highly structured
tasks and require human intervention for tasks that are
more dynamic and less well-defined.
However, it would only take a small engineering
tweak to transform a drone and take the human out of
the loop replacing it with a computer and thus enabling
the autonomous side of the technology. One such a system might already exist: the Samsung SGR-A1, a sentry
gun used to monitor the Korean demilitarized zone. This
gun - using technology like the one used in videogames - recognizes human shapes, and orders them to
stop and surrender. Currently, the gun is only used in
autonomous surveillance mode, but several reports confirm that the gun is equipped to deliver lethal or nonlethal force without human intervention [12], [13].
In short, neither the Reaper drones nor the Terminator are the killer robots I am discussing in this paper.
The locus of this discussion is systems such as the SGRA1, but only when their autonomous capacity is fully
functional, i.e., when they can be considered a fully
autonomous weapon system.

The Human and the Loop
Evidently, all robotic weapons have some degree of
autonomy based on how their software is programmed.
A further clarification might be required regarding the
level of human control in the targeting process, i.e., the
decision-making process. This human involvement can
be of three types [9]:
■■ Human-in-the-Loop Weapons: Robots that can
select targets and deliver force only with a human
command;
■■ Human-on-the-Loop Weapons: Robots that can
select targets and deliver force under the oversight
of a human operator who can override the robots'
actions; and
■■ Human-Out-of -the-Loop Weapons: Robots that
are capable of selecting targets and delivering force
without any human input or interaction.
Strictly speaking, AWSs fall into the third type
(human-out-of-the-loop). However, in some cases, weapons of the second type (human-on-the-loop) could be
considered de-facto out-of-the-loop weapons because of
lack of adequate or sufficient human supervision: "[t]he
ability of a single operator to have effective oversight of
dozens or even hundreds of aircrafts seems implausible
to many experts" [9].
MARCH 2018

∕

AI and robotics have reached a point
where it is technically feasible to
deploy a killer robot that hunts and
kills enemy combatants without
human intervention.
Definitions
There is a wide spectrum of current conceptions of
AWSs. According to Lewis et al. [4], "on one end of the
spectrum, [an] AWS is an automated component of an
existing weapon. On the other, it is a platform that is
itself capable of sensing, learning, and launching resulting attacks." Asaro [11] defines an AWS as "any system
that is capable of targeting and initiating the use of
potentially lethal force without direct human supervision
and direct human involvement in lethal decision-making."2
A similar definition is offered by the International Committee of the Red Cross [14]. I will use this broad definition of an AWS.
A further distinction can be drawn between defensive and offensive systems. Defensive systems are used
to protect facilities or areas from incoming attacks.
Many defensive systems already exist: one of them is
the Phalanx CIWS, which offers defense against antiship missiles. These systems can be highly automated,
but their autonomy is restricted by severe operational
constraints such as a limited envelop of operations, e.g.,
the area around a ship - or restrictions of the task they
carry out, e.g., they offer specific protection against
incoming supersonic missiles. Although arguments
against these weapons could be made, for example, on
absolute pacifist grounds, I will not argue against these
types of systems because of their limited autonomous
behavior and their acceptable degree of predictability -
even without human intervention after the system has
been activated.
The arguments I will present below apply to offensive
weapon systems, whose behavior might become less
and less predictable because of a series of developments related to their autonomous behavior: increased
mobility over greater periods of time, adaptability in
functioning and goal setting, self-learning, and
increased interaction of multiple weapon systems in
self-organizing formations [14].

2
Targeting and initiating the use of force are part of the so-called kill chain,
"defined in the US Air Force as containing six steps: find, fix, track, target,
engage and assess" [11].

IEEE Technology and Society Magazine

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - March 2018