IEEE Technology and Society Magazine - September 2016 - 55
could and should be developed to identify what types of
effective function allocations for human-AWS teams will
be capable of upholding the laws of war.
In designing human-AWS teams such that the operations are lawful, there are many potential problems that
must be accounted for in design. Table 1 relates the most
well-known problems with human-automation teams to
the design considerations that can attempt to alleviate or
even solve the problems. The problems and design solutions were adapted from Feigh and Pritchett's [30] review
of the function allocation literature, which has explored
human-automation interaction through ethnographies,
human-subjects studies, and computer simulations.
To the question of "what amount or quality of human
control is necessary for lawful use of AWS," the function allocation response is: the necessary amount or
quality of human control is that which enables each
agent within the team to be able to perform their task
or tasks with reasonable work within the dynamics of
the environment.
What Would an Accountability
Framework Look Like for AWS?
One of the main concerns with AWS is the apparent
ambiguity of who should be responsible for deaths
caused by AWS [1], [2], [8], an issue that has been
termed the responsibility gap [13]. There have been
claims that the use of AWS would inevitably result in
unacceptable responsibility gaps (e.g., operators and
commanders, programmers, and manufacturers escaping liability) [1]. Despite these claims, most nations at
the U.N. were confident in their ability to develop an
accountability framework for AWS [17]. Furthermore,
Müller [13] argues that we already live with responsibility
gaps in civilian life, and we should focus on narrowing
the responsibility gaps, not establishing an untenable
requirements of responsibility in all cases [13].
Narrowing responsibility gaps is already a focus of
cognitive systems engineers. Specifically, function allocation is used to model and measure responsibility gaps in
proposed and implemented human-autonomous systems
Table 1. Problems and Design Solutions for Human-automation Teams.
Potential Problems
Design Considerations
Brittle automation: where emergencies must be handled by humans
and there is little support for off-nominal conditions
High workload spikes during off-nominal situations [34] and excessively
low workload during normal operations in between the spikes leading
to humans becoming out of the loop [34], [35].
Leftover allocation (automate as many functions as technology will
permit, and assume the human will pick up whichever functions
are leftover) often results in a human assigned to monitor automation
or the environment for conditions beyond which the operate;
functions in which humans are ineffective [36]
Gap between authority and responsibility: automation is assigned
to execute a function is an operational sense but human will be held
accountable in an organizational and legal sense for its outcome. [37]
Design functions around both the automation and
the human's capabilities.
Consider authority-responsibility
Without being able to assess whether automation is correct,
humans often over trust or under trust the automation [38]
Automation and/or human unable to anticipate each other's information
needs and provide information at useful, non-interruptive times [39]
Clumsy automation which interrupts its team members because it
cannot implicitly sense information about whether other team
members would benefit from an interruption [40]
Consider automation as a team member.
In complex work environments where many functions interdependent
and coupled, some couplings may be hidden. Unaccounted-for couplings
can result in insufficient coordination, idling, and workload accumulation.
Dynamic analysis of the physical environment. Function
allocation should support how the work environment is
managed by the agents.
Adaptive function allocation (dynamically changing function allocation)
can aggravate environmental unpredictability.
Examine the tradeoff between maintaining predictability
for the human versus applying complex automated
capabilities and dynamically allocating functions [41].
september 2016
∕
IEEE Technology and Society Magazine
55
Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - September 2016
IEEE Technology and Society Magazine - September 2016 - Cover1
IEEE Technology and Society Magazine - September 2016 - Cover2
IEEE Technology and Society Magazine - September 2016 - 1
IEEE Technology and Society Magazine - September 2016 - 2
IEEE Technology and Society Magazine - September 2016 - 3
IEEE Technology and Society Magazine - September 2016 - 4
IEEE Technology and Society Magazine - September 2016 - 5
IEEE Technology and Society Magazine - September 2016 - 6
IEEE Technology and Society Magazine - September 2016 - 7
IEEE Technology and Society Magazine - September 2016 - 8
IEEE Technology and Society Magazine - September 2016 - 9
IEEE Technology and Society Magazine - September 2016 - 10
IEEE Technology and Society Magazine - September 2016 - 11
IEEE Technology and Society Magazine - September 2016 - 12
IEEE Technology and Society Magazine - September 2016 - 13
IEEE Technology and Society Magazine - September 2016 - 14
IEEE Technology and Society Magazine - September 2016 - 15
IEEE Technology and Society Magazine - September 2016 - 16
IEEE Technology and Society Magazine - September 2016 - 17
IEEE Technology and Society Magazine - September 2016 - 18
IEEE Technology and Society Magazine - September 2016 - 19
IEEE Technology and Society Magazine - September 2016 - 20
IEEE Technology and Society Magazine - September 2016 - 21
IEEE Technology and Society Magazine - September 2016 - 22
IEEE Technology and Society Magazine - September 2016 - 23
IEEE Technology and Society Magazine - September 2016 - 24
IEEE Technology and Society Magazine - September 2016 - 25
IEEE Technology and Society Magazine - September 2016 - 26
IEEE Technology and Society Magazine - September 2016 - 27
IEEE Technology and Society Magazine - September 2016 - 28
IEEE Technology and Society Magazine - September 2016 - 29
IEEE Technology and Society Magazine - September 2016 - 30
IEEE Technology and Society Magazine - September 2016 - 31
IEEE Technology and Society Magazine - September 2016 - 32
IEEE Technology and Society Magazine - September 2016 - 33
IEEE Technology and Society Magazine - September 2016 - 34
IEEE Technology and Society Magazine - September 2016 - 35
IEEE Technology and Society Magazine - September 2016 - 36
IEEE Technology and Society Magazine - September 2016 - 37
IEEE Technology and Society Magazine - September 2016 - 38
IEEE Technology and Society Magazine - September 2016 - 39
IEEE Technology and Society Magazine - September 2016 - 40
IEEE Technology and Society Magazine - September 2016 - 41
IEEE Technology and Society Magazine - September 2016 - 42
IEEE Technology and Society Magazine - September 2016 - 43
IEEE Technology and Society Magazine - September 2016 - 44
IEEE Technology and Society Magazine - September 2016 - 45
IEEE Technology and Society Magazine - September 2016 - 46
IEEE Technology and Society Magazine - September 2016 - 47
IEEE Technology and Society Magazine - September 2016 - 48
IEEE Technology and Society Magazine - September 2016 - 49
IEEE Technology and Society Magazine - September 2016 - 50
IEEE Technology and Society Magazine - September 2016 - 51
IEEE Technology and Society Magazine - September 2016 - 52
IEEE Technology and Society Magazine - September 2016 - 53
IEEE Technology and Society Magazine - September 2016 - 54
IEEE Technology and Society Magazine - September 2016 - 55
IEEE Technology and Society Magazine - September 2016 - 56
IEEE Technology and Society Magazine - September 2016 - 57
IEEE Technology and Society Magazine - September 2016 - 58
IEEE Technology and Society Magazine - September 2016 - 59
IEEE Technology and Society Magazine - September 2016 - 60
IEEE Technology and Society Magazine - September 2016 - 61
IEEE Technology and Society Magazine - September 2016 - 62
IEEE Technology and Society Magazine - September 2016 - 63
IEEE Technology and Society Magazine - September 2016 - 64
IEEE Technology and Society Magazine - September 2016 - 65
IEEE Technology and Society Magazine - September 2016 - 66
IEEE Technology and Society Magazine - September 2016 - 67
IEEE Technology and Society Magazine - September 2016 - 68
IEEE Technology and Society Magazine - September 2016 - 69
IEEE Technology and Society Magazine - September 2016 - 70
IEEE Technology and Society Magazine - September 2016 - 71
IEEE Technology and Society Magazine - September 2016 - 72
IEEE Technology and Society Magazine - September 2016 - 73
IEEE Technology and Society Magazine - September 2016 - 74
IEEE Technology and Society Magazine - September 2016 - 75
IEEE Technology and Society Magazine - September 2016 - 76
IEEE Technology and Society Magazine - September 2016 - 77
IEEE Technology and Society Magazine - September 2016 - 78
IEEE Technology and Society Magazine - September 2016 - 79
IEEE Technology and Society Magazine - September 2016 - 80
IEEE Technology and Society Magazine - September 2016 - 81
IEEE Technology and Society Magazine - September 2016 - 82
IEEE Technology and Society Magazine - September 2016 - 83
IEEE Technology and Society Magazine - September 2016 - 84
IEEE Technology and Society Magazine - September 2016 - 85
IEEE Technology and Society Magazine - September 2016 - 86
IEEE Technology and Society Magazine - September 2016 - 87
IEEE Technology and Society Magazine - September 2016 - 88
IEEE Technology and Society Magazine - September 2016 - Cover3
IEEE Technology and Society Magazine - September 2016 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2023
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2023
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2023
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2013
https://www.nxtbookmedia.com