IEEE Technology and Society Magazine - December 2020 - 37

human-centric approach. Several IEEE projects, series
and publications [4] are dedicated to this prioritization,
especially regarding civilian use. As autonomous technology is increasingly at the center of contemporary military
innovations, questions of (human) agency and responsibility in warfare have become even more pressing [5]. As
stressed by the United Nations Institute for Disarmament
Research (UNIDIR), the concept of MHC may prove useful
in the context of development and use of (semi-) autonomous weaponry [6].
Acknowledging the need for a multidisciplinary
approach [7], we present our analysis of the respective
CCW discourse as a first step towards answering the
question of how to ensure MHC in the interaction with
LAWS. Asking for factors supporting the implementation
of MHC, we look for values and underlying, more
abstract discourses which may converge with the idea
of MHC. The identification of such values and dominant
discursive narratives may contribute to a better understanding of the political, ethical, legal, and technological requirements for MHC, a concept which has been
introduced in both political and legal debates to allow
for improved regulation of the use of force in armed
conflicts [8]. The regulation of certain conventional
weapons considered particularly injurious or indiscriminative, such as landmines and blinding lasers, is part of
the protocols of the UN CCW. However, in the past, the
effectiveness of the CCW has been called into question
in light of Cold War politics [9], consensual decisionmaking, or difficulties in advancing arms control in the
context of humanitarian and military arguments [10].
Beside these difficulties, the CCW has helped set relevant norms of arms control in the past and may do so
with regards to the regulation of LAWS [11].
First, we give an overview of related work and identify research gaps. In the following, we elaborate our theoretical perspective, inspired by Value-Sensitive Design
(VSD). Further, we offer insights into the research
design, i.e., our sample of 43 CCW documents and discourse analysis as the method used. Subsequently, the
results of the analysis are presented. Our focus lies on
the CCW's prevalent understanding of autonomy and
LAWS as well as on control, particularly concerning MHC
in human-computer interaction. These conceptual clarifications are necessary foundational work for retaining
human control of LAWS. Then, we give an overview of
identified values with respect to the interaction with
LAWS technology, putting them into relation with MHC
as well as with each other. To gain a deeper understanding of the crucial values, we illustrate correlations
between respective values and dominant discourses,
which may not always support an implementation of
MHC. The work is concluded with a formulation of implications, a discussion of results, and an outlook.
DECEMBER 2020

∕

Related Work
Scholars of various disciplines are dedicated to LAWS
and the question of human control. Most work focuses
on autonomous weapons from a legal perspective with
regard to international humanitarian law (IHL). Crootof
[12], focusing on the applicability of international
humanitarian law and accountability, reflects on the
inherent imprecision of the concept of MHC, while
stressing the need to interpret the evolving norm as convergent with existing international law. Anderson [13]-
[15] elaborates on the applicability of the law of armed
conflict on emerging technologies and explicitly argues
against a ban of LAWS. Walker-Smith [16], concentrating
on MHC, criticizes a human-biased view by pointing out
that autonomous weapon systems can potentially limit
lethal human behavior. Walsh [17] notices a concomitant shift of accountability with the development of
LAWS, increasingly including designers and programmers. Yet, these legal debates are often characterized
by repetitive arguments, echoing difficulties regarding
legal accountability while stressing the merit of existing
international humanitarian law principles.1
Ethical perspectives stress that the technological
advances towards more autonomous functions and systems, which are increasingly interacting with humans,
need to be designed with more attention towards
responsibility [18], [19], control [20], and effects on
human dignity [21], thus arguing to limit the possible
consequences on the lives of affected people [22], [23].
Scholars of ethics agree that LAWS run counter the principle of human dignity as only humans can be moral
agents that can take life-affecting decisions and be held
accountable [21], [24]. Sharkey [25] introduces a classification system of human supervisory control, which is
adapted by Amoroso and Tamburrini [24], putting it into
relation with the concept of MHC, as well as by Suchman and Weber [26], who focus on autonomy of humanmachine configurations. Ekelhof [27], [28] asks for the
feasibility of operational implementation of MHC, suggesting other ideas like " distributed control " to be more
practical. It can be identified that the discourse has
moved towards discussing autonomous functions, especially regarding the target selection and target engaging
process [24], [29].
Scholars of international relations and strategic security [30]-[33] have discussed the strategic consequences of
autonomy. On one side, autonomous systems are a central part of the network-centric warfare (NCW) doctrine,
which plans for autonomous weapon systems to increasingly assist humans as human-machine teaming [34], [35].
However, the autonomy of armed systems is increasing
1

In contrast, Sehrawat [86] takes a minor position in the debate, arguing
IHL principles to be ineffective for regulating LAWS.

IEEE TECHNOLOGY AND SOCIETY MAGAZINE

IEEE Technology and Society Magazine - December 2020

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - December 2020