IEEE Technology and Society Magazine - December 2020 - 44

assuming that language shapes actors' perceptions of
human-LAWS interaction [3].
Speaking in relative terms, statements indicating
hierarchical relationships of humans and subordinated
technology were found almost twice as often as statements supporting the autonomy of technology (49% versus 27%). At the same time, " interaction, " as a rather
neutral or non-hierarchical position, made up roughly a
quarter (27%) of the respective human-LAWS interaction
statements. This indicates that perceiving humans to be

Control of autonomous systems
matters, however, with divergent
implications on how, what (regarding
the function and situation), and in
which circumstances.
superior to technology is the dominant view within the
CCW GGE forum. These relative frequencies yield several implications: First, it becomes clear that explicit MHCrelated statements are very rare (in absolute as well as
relative terms). This reflects that despite its initial trendiness as an ethical-legal buzzword [6], [75], CCW GGE
participants are rather uninterested in highlighting MHC.
In cases of referring to MHC, it is either done by a nongovernmental participant like the Campaign to Stop Killer Robots (GGE_CTSKR) or the ICRC, elaborating on the
concept in a more detailed manner (WP5.2-ICRC). When
used by states, MHC is framed to be important merely
by (potential target) states which are dedicated to a ban
of LAWS, like Pakistan or Brazil. In such cases, stronger
normative terms were used: " The task for the GGE now
should be to ascertain the scope and extent of human
control necessary to address the various concerns associated with LAWS to ensure that it is meaningful " (6bPAK); " [t]he proposal that humans retain 'meaningful
control' over LAWS seems to us the most promising avenue to explore " (6b-BR). Other nation-states' representatives do not seem similarly interested in the MHC
concept, disregarding the concept considering its level
of abstraction and necessity: " [I]t becomes difficult to
provide a technical statement of meaningful human
control. [...] To be meaningful, human control does not
necessarily have to be exercised contemporaneously
with the delivery of force. [...] " (WP2-ES/FI).
The U.S. also relativizes the conceptual impact of
MHC, as " an operator might be able to exercise meaningful control over every aspect of a weapon system, but

44

if the operator is only reflexively pressing a button to
approve strikes recommended by the weapon system
[...] " (WP4-US). Pointing out its lack of value because of
its conceptual unclarity functions as a devaluation
mechanism of the MHC concept across the CCW discourse: " 'Control' and 'judgment' are, however, flexible
terms, even when qualified by adjectives such as 'meaningful' [...], [...] used [...] to signify different things "
(6b-Estonia). Most decisively stated, MHC " or a similar
notion " is not considered " to reflect a new or emerging
norm of international law " (6b-Estonia), while one participant seems to regret " exploring the scope of meaningful human control in the delegation of decisions to
intelligent machines instead of what we must regulate
on LAWS " (GenEx-KO, own emp.).
Nevertheless, the widespread references to human
control stress the respective participants' underlying
interest in a hierarchical relationship, with human operators controlling LAWS to varying degrees. As laid out in
the following section, " Socio-Technological Values of
LAWS, " such control may be associated with various values, with the latter being regarded as defining characteristics of the concept of MHC. Thus, even though
actors show little support of the concept, they widely
refer to technological requirements defining the concept
of interest [75].
Third, while dominantly retaining human-focused
arguments within the CCW debate, there are still a considerable number of statements carrying a supportive
notion of technology, i.e., perceptions of technology as
a societal driving force or as essentially being. Phrases
that were coded as " tech_anthro " or " powerful_tech, "
respectively, show such instances: " The development of
artificial intelligence (AI) should be seen as a logical
process in computing science, " while demanding that
" the discussions on LAWS must reflect the undeniable
direction of technological development " (WP2-ES/FI).
Interest in the support of LAWS and related AI technology is further formulated, declaring " the system would be
capable of defining and thereby deciding the ultimate
goals of its functioning, very much like humans do "
(WP2-ES/FI), peaking in the following explicit statement
for implementation: " The level of abstraction of computing keeps getting higher and higher, leading towards
increasing possibilities for various levels of machine
autonomy. Past experience has shown that once new
technology proves to work, society quickly adopts it, and
later its use becomes the accepted norm " (WP2-ES/FI).
While remarks that were coded by " interaction " may
also imply characterizations of technology, they also
suggest a more reflective awareness of interactional
relationships between humans and LAWS as well as
related effects: " [T]he way humans use machines and
interact with them is changing [...] [because] [i]n

IEEE TECHNOLOGY AND SOCIETY MAGAZINE

∕

DECEMBER 2020
IEEE Technology and Society Magazine - December 2020

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - December 2020
