IEEE Technology and Society Magazine - March 2018 - 59

Battlefield awareness may be computationally intractable, and there is no evidence that AWSs could operate
on the principle of distinction as well as human soldiers can.
IHL prohibits weapon systems that cannot be properly aimed, these weapons are unlawful per se in that they
fail to comply with the principle of discrimination. On
this basis, Sharkey [1] argues that the morally correct
course of action is to ban AWSs.
To counterargue, Schmitt and Thurnher [2] pose the
scenario of an autonomous system for an attack on a
tank formation in a remote area of a desert:
not all battlespaces contain civilians or civilian
objects. When they do not, a system devoid of an
ability to distinguish protected persons and
objects from lawful military targets can be used
without endangering the former.
The inability of the weapon systems to distinguish
bears on the legality of their use in particular circumstances, such as along a roadway on which military
and civilian traffic travels - but not on their lawfulness
per se.
Since increased mobility seems to be one of the
trends for AWSs, a killer robot will not necessarily be
active only in particular circumstances or specific battlespaces that can be cleared of civilians in advance.
Autonomy is a game-changer precisely because offensive
autonomous weapons have an autonomous capacity for
roaming in order to select and engage a target, which is
their intrinsic characteristic. The very notion of battlespaces that can be known to be totally safe beforehand becomes nonsensical: the location of the theatre of
warfare is dynamic and complex and can not be confined to "particular circumstances" in which an autonomous system would have permission to safely fail to
make a distinction between civilians and military targets.
Schmitt and Thurnher [2] also commend the present
pattern-recognition capabilities of military technology:

of a ballistic missile, but another very different thing
to determine whether a person is a civilian or a non-uniform-wearing combatant.
On the other hand, there are pattern-recognition
technologies available today that are indeed able to discriminate between a civilian and a uniform-wearing soldier based on images.4 It could be thus argued that
Sharkey's objections 1) and 2) above could be worked
out for uniform-wearing combatants. Objections 1) and
2) would still stand in situations where combatants are
not wearing uniforms. It doesn't require a leap of faith
to imagine that once effective uniform-seeking AWSs
are operational, the enemy would cease wearing uniforms altogether in order to avoid being detected. Since
an important discrimination variable - the uniform -
is eliminated, it's easy to see how this might in turn
heighten the risks to civilians.
According to Mike Schroepfer, Facebook's Chief Technology Officer, "in recent years, the state of AI research
has advanced very quickly - but there's still a long way
to go before computers can learn, plan, and reason like
humans" [17]. Thanks to state-of-the-art technology, a
leading company such as Facebook has the actual ability to segment visual objects and label them with information. The technology is already being used to
describe photos to visually impaired people, but it is far
from error-proof as the examples in [17] reveal. But even
if these recognition systems are rapidly improving, current AI systems such as Facebook's also fail at understanding the picture as a whole, including the context
surrounding the objects they recognize. Schroepfer
exemplifies this issue with a picture of a sausage pizza.
A human has no trouble visually discriminating whether
a sausage pizza is vegetarian or not.5 A computer, however, can't answer that question because it lacks the
necessary contextual understanding. Facebook's
researchers are currently trying to give computers contextual understanding, one case at the time. Schroepfer
[17] explains:

"[it] has advanced well beyond simply being able
to spot an individual or object. Modern sensors
can ... assess the shape and size of objects, determine their speed, identify the type of propulsion
being used ..."

we need to give them a model by which they can
understand the world - a set of facts and concepts they can draw from in order to answer questions like the one about the pizza. [...] Despite [...]
progress, there's a lot more work required to
make computer systems truly intelligent.

It is true that recognition technology is advancing
rapidly. Automated traffic-control systems equipped
with road sensors can detect if a car is speeding,
take a picture of the car, read its license plate number, and send the car owner a fine. However, it is one
thing to detect and recognize well-defined variables
such as the numbers on a license plate or the shape

4
In November 2016, Facebook published an example of their software for
understanding a visual scene that identifies the objects in it. A picture of
a woman and two policemen was correctly recognized and labeled by the
software. The picture is no longer on-line, but it can be accessed at the
Internet Archive: http://web.archive.org/web/20161205202820/http://
newsroom.fb.com/news/2016/11/accelerating-innovation-and-poweringnew-experiences-with-ai/.
5
Humans are able to do this thanks to representational structures often
knows as mental models, schemata, frames, etc., which enable humans to
interpret the world they see.

MARCH 2018

∕

IEEE Technology and Society Magazine

http://newsroom.fb.com/news/2016/11/accelerating-innovation-and-powering-new-experiences-with-ai/ http://web.archive.org/web/20161205202820 http://newsroom.fb.com/news/2016/11/accelerating-innovation-and-powering-new-experiences-with-ai/ http://newsroom.fb.com/news/2016/11/accelerating-innovation-and-powering-new-experiences-with-ai/

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - March 2018