IEEE Technology and Society Magazine - June 2021 - 59

discussed throughout the design, implementation, and
release phases of development. All these principles are
somewhat included or implied in the White Paper, but
we think that: 1) they should be specifically mentioned
and 2) specific references to auditability and impact of
human experimentation have to be included.
Need for a basic technical framework
To carry out a human rights by design approach
(which should also include a human-rights impact
assessment and the definition of potential risks or
hazards-or potential opportunities-for algorithmic
decision-making), it is important to achieve a common
understanding of how these systems are developed
and how they behave in contact with the public. A
descriptive model of the systems under scrutiny would
need to be specified, at least in a very high level fashion,
so all stakeholders can refer to the very specifics of
the AI domain. This model should not only include the
technological boundaries of the system, but also the
organizations in charge of decisions and human communities
affected by these decisions. These undoubtedly
introduce hybrid societal and technical feedback
loops that affect the broad dynamics of behavior of all
components. In this aspect, the EU White Paper falls
short. There is currently a very common gap between
the legal-ethics documents evaluated by policy-makers,
like the EU White Paper, and the technical and
system architectural documents used by engineers in
their practice. The reason for this gap can be derived
from the different background, education, and perspectives
of all involved in the regulatory effort: engineers,
ethicists, sociologists, policy-makers, and other
disciplines. If not tackled, this " disciplinary divide "
[16] leading to several abstraction errors as described
in [17] in software implementation and regulation can
be lost in translation when implemented by engineers.
This article advocates for this gap to be closed as early
as possible in the regulatory process through a set of
definitions and description of different device architectures,
if the regulation intends to have a real impact
in technical and practical terms. The idea is simple:
it is very hard to do a car emission inspection just by
stating that the car should be environmentally friendly
and without being very specific and formal about car
architecture, emission check points, what needs to be
inspected, and where, when, and how these inspections
need to take place. Likewise, just to say that an
AI system needs to be trustworthy, fair, or transparent
is not enough and leaves it very open to interpretation
June 2021
Figure 1. Simple ML schema with checkpoints.
and implementation, becoming useless in terms of
enforcement. We need a more formal definition and a
technical and architectural description of each type of
AI system. An image classifier, a voice or face recognition,
an ML that predicts human behavior in the future,
or a recommender system: all these groups of applications
have already a more or less stable architecture
and for each of them we could already start to identify
measures of performance and checkpoints that should
be tested before and during the algorithm operations.
In any technical regulation, it is essential to frame
the scope and the subject domain in sufficient technical
detail so that its contents are workable and
understandable, so that all stakeholders refer to a
common understanding of how an AI application is
working and how it is affecting its environment.
For instance, a system-level model of an ML system
used to classify citizens in any aspect or predict
behavior could be described like the one depicted
in Figure 1, making it easy to determine checkpoints
where, potentially, design options or system dynamics
could potentially have impact on human rights.
Once identified, the specific issues pertaining to
each checkpoint can be discussed. Apart from the
popular potential bias in predicted results or training
data, this view of the system in contact with its
environment enables us to see other risk sources and
checkpoints like the following:
* The success criteria of the learning algorithm are
linked to the original purpose of the algorithm. To
59
IEEE Technology and Society Magazine - June 2021

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - June 2021
