IEEE Technology and Society Magazine - June 2021 - 86

Feature
management practices [38] that are attuned explicitly
to issues of responsible AI.
The abundance of tools
The final explanation for the principles-to-practices
gap that we discuss is, ironically, the potential
over-proliferation of solutions. Indeed, an increasing
number of researchers who have noticed the principles-to-practices
gap have begun proposing strategies
[39], often aimed at AI practices within industry.
These proposals include changes to software mechanisms
(such as audit trails), hardware mechanisms
(such as secure hardware enclaves), and institutional
mechanisms (such as red team exercises) [37]. This
work highlights that it is not only technical practices
that must adapt, but also organizational ones.
Among the most comprehensive work assessing
the principles-to-practices gap is the 2019 review by
Morley et al. [40], which systematically explores existing
responsible AI tools and methodologies mapped
against seven components of the AI development
lifecycle: 1) business and use-case development; 2)
design phase; 3) training and test data procurement;
4) building; 5) testing; 6) deployment; and 7) monitoring.
They identify 106 tools and methodologies.
Some such methods are relatively narrower in
scope, such as those surrounding explainable AI
[41], bias [42], or procurement (e.g., the AI-RFX
Procurement Framework). Other methodologies
adopt a broader scope of focus, including impact
assessments like the ISO 26000 Framework for
Social Responsibility [43] and the IEEE 7010 Recommended
Practice for Assessing the Well-Being
Implications of Autonomous and Intelligent Systems
[44]. Still more relevant methods and approaches
proposed for responsible AI also come from outside
of the AI domain and include privacy-by-design
[45], value-sensitive design [46], the Responsible
Research and Innovation (RRI) approach [47], and
numerous others.
Yet while creating more and better such tools and
methodologies is a worthy pursuit, an over-abundance
problem makes it difficult for individuals to
sort through and assess the utility of a given tool, or
to weigh it against the many other available tools.
As a result, individuals and organizations may fail to
take advantage of the useful tools and methodologies
that already exist. Even tools that do exist have
arguably not been tested sufficiently to demonstrate
which are most effective and in which contexts [11].
A further problem is that many tools and methodologies
do not contain sufficient instructions to apply,
customize, or troubleshoot, much less in particular
organizational contexts and use cases [40].
Closing the principles-to-practices gap
Criteria of an effective framework for
responsible AI
Given the proposed explanations above, how can
we begin to close the principles-to-practices gap? We
think an overarching framework for responsible AI
development can help to streamline practice and leverage
existing tools and methodologies. What would
be the desiderata of such a framework for responsible
AI?1 As a starting point and based on the identified
gaps, we suggest the following (see Figure 2):
* Broad: It should consider AI's impacts expansively,
across many different ethical issues and
aspects of social and economic life.
* Operationalizable: It should enable users to cast
conceptual principles and goals into specific
strategies that can be implemented in real-world
systems.
* Flexible: It should be able to adapt to a wide variety
of AI systems, use cases, implementation contexts,
and organizational settings.
* Iterative: It should be applied throughout the lifecycle
of an AI system and repeatedly.
* Guided: It should be easy to access and understand,
with sufficient documentation.
Figure 2. Criteria of a framework for responsible AI
development.
86
* Participatory: It should incorporate the perspectives
and input from stakeholders from a range of
disciplines as well as those that may be impacted
by the AI system, especially the public.
1Akin to what Dignum calls " Ethics in Design " [48].
IEEE Technology and Society Magazine
IEEE Technology and Society Magazine - June 2021

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - June 2021
