IEEE Systems, Man and Cybernetics Magazine - April 2022 - 15

behavioral patterns analyzed to gauge intentions.
Thus, trust is developed through observable behaviors,
measurable actions, or analyzable patterns
through these types of computational interactions.
The computational results and implications may then
be judged as supportive or not.
◆ Performance: Trust should be built, earned, maintained,
and assured. The development of trust results
from the reliable, consistent, and predictable performance
of a partner over time [44]. In fact, performance
is identified as the primary contributor for establishing
trust for robots and significant for the establishment of
trust for automation [47]. Performance entails a variety
of attributes. For a machine to gain trust from its
human partner, it must demonstrate performance that
is valid (exhibiting as intended), reliable (being consistent
over time), dependable (having few errors), and
predictable (meeting human expectations).
◆ Adaptivity: AI entities need to be capable of learning
and understanding their human partners' intentions
and also the machine status and changes in the tasking
environment, monitoring the human cognitive workload
and performance, guarding the human resources
and time, and changing their course of action to work
with humans to achieve the team's common goals. An
AI entity that exhibits these adaptive and intelligent
characteristics may then be a trusted partner to build a
truly collaborative human-AI symbiotic partnership.
An effective IAS should enable machine adaptivity
through dynamic interactions between humans and
machines and, thus, foster the development, maintenance,
and assurance of mutual trust.
◆ Communication: Communication is a key to team success
and needs to be bidirectional for humans and
machines to learn and understand each other as partners.
An effective and trustworthy IAS should enable
AI entities and their human partners to clearly, fairly,
and directly exhibit and justify their intentions,
actions, and desired end states and how they may help
each other reach their common goals. These are critical
characteristics that machines must exhibit to
build up human confidence and trust. An IAS also
needs to be flexible and offer the types of feedback
that the human partner would like to receive and
accept such that effective communications occur at
the right time, in the right format, through the right
channel, and to the right recipient. If an IAS could
facilitate such effective communications, an appropriate
trust partnership would be enabled, maintained,
and assured constantly.
◆ Transparency: It has often been said that AI runs in a
black box because it works in a fashion that humans do
not fully understand and that they have no way of validating
regarding its intentions. The opacity of AI is
problematic for trust development, maintenance, and
assurance. Opacity has remained acceptable when the
machine is reliable and is designed for simple tasks.
Yet, when AI is expected to perform complex tasks,
involving multifaceted decisions in uncertain situations
alongside humans with potentially vital consequences,
transparency is crucial for humans to ascertain that AI
entities' goals and methods for achieving those goals
are aligned with shared intentions. Transparency can
be facilitated and optimized through an explainable
HMI that communicates real-time information to
humans and machines regarding their intentions, goals,
reasons, decisions, actions, and expected outcomes.
◆ Security: IAS capabilities are continuously enhanced
by the growth of AI-enabled functionalities; however, it
naturally increases the system complexity. In engineering,
complexity generally translates into uncertainty
and risk, and this generalization applies to the design of
IASs. The design should protect the system from accidents
or deliberate threats (e.g., a cyberattack) and,
thus, build human trust. A secured system must behave
as designed and implemented following rules or laws
even when it is under attack; otherwise, it cannot gain
or assure trust. This insight requires designers and
organizations to build confidence in IAS technologies
by providing goal-directed explanations of security
measures in place (i.e., at the right level of detail) to
protect and ensure system safety and performance.
According to the IMPACTS model, trust is a vertex and a
crucial ingredient for collaborative human-machine symbiotic
partnership. It is the careful balance on which healthy
relationships grow and are maintained and assured among
partners when considering physical, intellectual, emotional,
ethical, moral, relational, and even spiritual aspects of
human beings. For technologies to be truly trustworthy,
consistent, predictable, and reliable and to demonstrate
human-like integrity with shared intentions through their
adaptive behaviors and measurable performance, transparent
communications, and secured protection are indeed
" IMPACTS " that only humans can make with their AI partners.
Researchers and practitioners developing IASs must
carefully design them to inspire confidence and build trust.
The IMPACTS model is a conceptually practical tool to
guide the design and development of collaborative and
trusted human-AI symbiotic partnerships.
IMPACTS for Trustworthy IAS Development
and Acceptance
To address complex, lengthy, and error-prone target
engagement processes, the IMPACTS model has been
applied for the development of an IAS called the Authority
Pathway for Weapon Engagement (APWE) as an intelligent
decision support system [48]. The APWE automatically
streamlines engagement processes and enables
operators to visualize the dynamic engagement status
intuitively through its intelligent, adaptive HMI, thereby
reducing engagement times and errors while enhancing
operators' SA.
April 2022 IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE 15
IEEE Systems, Man and Cybernetics Magazine - April 2022

Table of Contents for the Digital Edition of IEEE Systems, Man and Cybernetics Magazine - April 2022
