IEEE Technology and Society Magazine - September 2016 - 53

bring new arguments and proposals. While the breadth
of ideas should make the AWS regulation discussions
very rich, to date many of the proposed arguments
seem to be talking past one another. New proposals
should modify or build on current proposals, highlighting the conflicts and differences. The lack of a common
language with technical specificity makes it unclear
what conflicts need to be addressed and what consensus can be built upon in the future.

Operationalizing Regulations
Once conflicts have been addressed and a consensus
has been converged upon, there remains the issue of
operationalization: how do we write the international regulations to be both implementable and certifiable? A
summary of the progression of meaningful human control from ethical, to regulatory, to technological language
shows that more work is needed to operationalize proposals. Many have made the ethical argument that having an AWS make the decision to kill a human violates
the human right to life (only humans can make the necessary qualitative assessments). AWS decision-making, it
is argued, also prevents a remedy for unlawful action
(AWS cannot be held directly accountable for their
actions), and violates the principle of dignity (inanimate
machines cannot truly understand the value of life) [8].
These concerns were translated into regulatory language
by requiring a mandate that there should always be
meaningful human control over autonomous weapons
systems. This is necessary to ensure that a human will
make qualitative assessments, be held accountable for
actions taken, and value the human life [21]. Specifically,
the proposed regulations specify that the human operator must always have "a means for the rapid suspension
of attacks" and "sufficient time for deliberation" [22].
Only when the mandate gets to the technologist for
implementation is it realized that the proposed regulations outlaw numerous weapons systems that are already
used without controversy [19]. It is also seen that the proposed regulations are not operationalized enough to be
implemented [23], nor would they likely ensure meaningful human control if they were implemented.
If the proposals for meaningful human control - or
any other regulations - were implemented in their
current state, without being operationalized, the
result would actually undermine the intentions of the
initial ethical concerns that started these discussions
in the first case. The ambiguity would leave many of
the design decisions, such as how to interpret what
exactly is meant by "adequate contextual information" or "sufficient time for deliberation" in the hands
of each government or manufacturer. Not only would
this result in an inconsistent application of the regulation (as it is highly unlikely that any two designers,
september 2016

∕

much less two nations, would interpret them in the
same way), it also leaves the door open for an intentionally malicious interpretation that follows the letter
of the law, while ignoring its spirit. Furthermore, there
are no systematic ways to certify systems based on
these proposals.

Addressing the Four Issues of Autonomous
Weapons Systems through Cognitive
Systems Engineering
We propose that the language of cognitive systems engineering is uniquely poised to be the unifying, technical
language of the meeting of experts on autonomous
weapons systems. While the four key issues of definitions, human control, accountability, and certification
for human-autonomous systems may be relatively new
to the international weapons regulations community,
these issues have been topics of research for cognitive
systems engineers for over 30 years [24]. There are two
general synergies between cognitive systems engineering and the regulatory discussions of autonomous weapons systems. First, autonomous weapons systems fit
within the established domain of cognitive systems
engineering, which is focused on complex, sociotechnical, and safety-critical systems that are dependent on
human-technology interaction to perform successfully,3
including military test and evaluation [25], aviation [26],
and human space operations [27]. Second, whereas regulatory discussions are looking for methods of analysis,
design, and evaluation, cognitive systems engineering
already has standards and techniques for modeling and
measuring performance of complex, sociotechnical,
and safety-critical systems like autonomous weapons
systems [30], [32], [33].
Developing international regulation of autonomous
weapons is clearly a difficult problem. However, these
stalled discussions should be the source of motivation to
identify new perspectives that make it easier to identify
the sources of the conflict and solutions that have consensus, and to operationalize the solutions that are supported. We believe that to make progress on the four key
issues of definitions, human control, accountability, and
certification, discussants at the U.N. should utilize the
synergetic potential of cognitive systems engineering:
1) How do we define autonomy? (Should we?) Use the
requirements for effective function allocation to
develop standards for human-AWS interaction and
meaningful human control.
2) What amount or quality of human control is necessary for lawful use of AWS? Use function allocation's models and metrics to evaluate human-AWS
3
Definition adapted from the Cognitive Engineering Center at the Georgia
Institute of Technology (http://cec.gatech.edu) and [24].

IEEE Technology and Society Magazine

http://cec.gatech.edu

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - September 2016