IEEE Technology and Society Magazine - December 2020 - 49

several procedural measures may prove helpful. Software
developers, certainly aware that surveillance, detection, or
identification are crucial parts of the targeting process,
need to intensify sensitivity towards biases [39].
Still ongoing is the discussion whether it is possible
to design autonomous weapon systems that can adhere
to the principles of international humanitarian law, e.g.,
to protect civilians and to ensure the use of appropriate
and necessary force only [21], [81]. Cummings [36] has
argued with regards to the strategic target selection by
commanders that in case the precision of the system
performs better than the human operators, autonomous
target selection follows the IHL [36]. She [36] suggests
that autonomous systems are potentially better in performing and preventing unnecessary suffering due to
the stress-bias commanders face in a combat situation.
Thus, a research culture that fosters open communication about systemic errors (e.g., in imagery analysis) as
well as responsibility in R&D needs to be initiated
urgently [82]. This culture may be enhanced by recording design processes [83].
To summarize, control of autonomous systems matters, however, with divergent implications on how, what
(regarding the function and situation), and in which circumstances. As we asked for supporting conditions of
MHC within the respective CCW forum, tense associations between predictability and reliability, on the one
hand, and precision and efficiency on the other hand,
become visible. While precision may prove crucial for
reliability and predictability, it is neither sufficient to aim
solely for precision. Additionally, as reflected by the
value of efficiency as well as various references to time
as an important factor, developers need to weigh
options for timely interventions against rapidity of
actions. This supports prioritizing situational awareness
of the human operator within the design process [26].
Lastly, and with respect to our finding of economic
and military discourses not necessarily supporting
MHC, restrictions of software development may neither
be grounded in economic interests nor in a military
advantage ratio. Paying special attention to humanLAWS perception among CCW participants, it is revealed
that MHC heavily relies on a human-centric hierarchy.
This may contradict a decentralized network-centric
warfare. Yet, as commercial industries as well as the
military are relevant (if not the most relevant) actors,
deliberation among involved actors is necessary.

Limitations and Outlook
In reacting to the legal [12] and technical [84] debates of
LAWS and human control, we tried to put them into relation. Our work offers a first conduct on the question of
MHC within the CCW GGE discourse. Naturally, this
study yields several limitations. Parallel coding by
DECEMBER 2020

∕

several researchers and measurement of inter-rater reliability may improve the robustness of the findings. We
presented a list of LAWS-incorporated values and
sketched tendencies of MHC-supportive values and rather competitive relationships. Future work may focus on
other relationships or ambivalences of values like " precision " or " time. " At the same time, we tried to grasp discursive patterns, norms, and practices forming
participants' perceptions of LAWS features by using VSD
as a theoretical starting point. Crediting the initiative,
one may consider that while the sample tried to represent existing power relations, it does not pay, in contrast
to other work, special attention to racism or sexism [50],
[51]. Future studies may contribute by investigating these
issues, in particular with regards to stakeholders' positions on MHC. Because of purposive inclusion of indirectly involved actors, the quantitative results of coding
frequencies and word stem occurrences should not be
taken to be absolute but rather be seen as an indication
of relationships. To complement the focus on discourse,
future analysis may shed light on material capabilities,
long-term national interests, as well as innovation and
defense politics. Our study may be accompanied by
analyses comprising the entire body of the CCW expert
group's documents to increase representativeness and
validity of our findings, including statements of all actors
and of more recent meetings. At the same time, focusing
specifically on the different understandings of sociotechnological values like accountability may prove valuable insights. Further, it may prove valuable to dig
deeper into technological specifics, which CCW participants did not communicate in a very detailed way.

Acknowledgment
This work was supported by the German Federal Ministry for Education and Research in the project KontiKat
(13N14351), by the German Federal Ministry of Education and Research, the Hessen State Ministry for Higher
Education, Research and the Arts within their joint support of the National Research Center for Applied Cybersecurity ATHENE and by the Forum for Interdisciplinary
Research (FIF) in the IANUS research fund for interdisciplinary research within the research project " IT
research and development of concern. " A two-page presentation of the motivation for this research was presented at Science Peace Security '19 in Darmstadt [85].

Author Information
Thea Riebe is a doctoral student and research associate
at PEASEC, 64289 Darmstadt, Germany; email: riebe@
peasec.tu-darmstadt.de.
Stefka Schmid is a doctoral student and research
associate at PEASEC, 64289 Darmstadt, Germany; email:
schmid@peasec.tu-darmstadt.de.

IEEE TECHNOLOGY AND SOCIETY MAGAZINE

IEEE Technology and Society Magazine - December 2020

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - December 2020