IEEE Technology and Society Magazine - June 2021 - 87

A framework that meets these criteria balances
the need for technical specificity with an equally
important need for conceiving of AI's impacts on
human well-being in their full breadth and complexity,
understanding we cannot fully predict all of AI's
possible ramifications. That is, while some prominent
strategies for responsible AI assume there are
only a small set of issues to address, such as bias,
transparency, and privacy, we have argued that AI's
impacts are more complex.
Impact assessments are one promising strategy
toward achieving many of these criteria [49]. Impact
assessments have been used historically in human
rights [50], in regulatory contexts [51], and more
recently to study the impact of AI or algorithms [52],
[53]. Measuring impact has several benefits: it enables
monitoring of risks, allows for a more secure environment
for investment, promotes accountability and
transparency, and overall enhances the prospects of
pro-social innovation [3].2 We focus on the recently
published IEEE 7010 standard as an exemplar [44],
[53] created specifically to assess AI's impacts on
human well-being.3 We argue below that impact
assessments like the well-being impact assessment
from the IEEE 7010 standard could be adopted by
companies pursuing responsible AI development as
well as incorporated into the institutions which train
future practitioners.
Impact assessments for responsible AI
According to the IEEE 7010 standard, a well-being
impact assessment is an iterative process that entails:
1) internal analysis; 2) user and stakeholder engagement;
and 3) data collection, among other activities
(see Figure 3). The internal analysis involves broadly
assessing the possible harms, risks, and intended and
unintended users and uses of an AI system. Here,
developers and managers of an AI system carefully
consider a wide range of an AI system's potential
impacts on human well-being, not limited to prominent
issues like privacy, bias, or transparency.
Critically, assessing impacts requires not just speculating
about impacts, but also measuring them. Therefore,
the user and stakeholder engagement stages of
the assessment include learning from users of AI systems
as well as others more indirectly impacted to
determine how the system impacts their well-being.
2 Also see [3] for a discussion of challenges with impact measurement.
3 While the authors of this article were involved in helping to develop IEEE 7010,
this article reflects the individual views of the authors and not an official position
of the IEEE.
Figure 3. Well-being impact assessment
process.
When developers have access to users, this may
include asking them about possible or actual psychological
impacts, economic impacts, changes to
relationships, work-life balance, or health. This is
again in contrast to strategies that focus solely on
technical fixes to issues like bias or privacy during
the design stage alone and fail to account for the
broader universe of well-being implications.
Beyond initial stakeholder engagement, data collection
based on the continuous assessment of the
identified possible impact areas is key. Data can be
collected through user surveys, focus, groups, publicly-available
data sources, or directly as system
outputs. In sum, we propose that adherence to-and
rigorous documentation of-impact assessments
can contribute to the continuous improvement of
AI systems by ensuring that organizations are better
able to understand and address AI's many impacts
on human well-being.
Importantly, not all tools entitled " impact assessment "
meet our definition. Many existing tools
consider only a small scope of possible impacts.
June 2021
87
IEEE Technology and Society Magazine - June 2021

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - June 2021
