IEEE Technology and Society Magazine - March 2022 - 69

and not the result of deliberate value preferencing.
Second, such unintentionally embedded values are
often hard to observe. Engineers typically design systems
for technical or usability reasons and when that
arrangement happens to promote some values over
others as a byproduct, that effect can be difficult
to detect. Finally, in such cases, there is the legitimate
argument that the design choice occurred for
reasons completely unrelated to value preferences
undermining the claim that any value embedding
exists at all.
The point is that there does not have to be a
nefarious intent, or any intent at all, for a technology
to promote certain values or social groups at the
expense of others. Rather any system can come to
embody values or biases for completely innocuous
reasons. Nonetheless, whether values are intentionally
or unintentionally embedded in a technology,
the impact on society can be equally impactful
when it is broadly deployed.
Aura of technological objectivity
An additional point is that technological systems
often have the misleading semblance of objectivity,
as compared to human decision makers [1]. This
contrast may be particularly evident when such systems
are used within the law. When people-such as
judges-make decisions, it is obvious that as members
of society, they may be preferencing some values
over others or be unduly influenced by emotion
or bias. By contrast, legal technological systems, particularly
those that rely upon data, may superficially
appear have an aura of objectivity and neutrality.
These systems are nonhuman artifacts, and as such,
are not subject to emotion, and they may appear to
render objective decisions who outcome is the necessary
result of neutral data analysis. For this reason,
the value-laden aspects of such technological systems
may be overlooked, dismissed, or deferred to,
even as they have real-world effects.
For example, consider the systems that judges
rely upon to assist in their criminal sentencing decisions.
These systems make recommendations based
upon analysis of data. However, the output of the
system-a formal recommendation report-actually
masks an underlying series of subjective judgments
on the part of the system designers. These subjective
choices include: what data sources to use to build
the predictive model, which parts of the chosen data
to include or exclude, how to weight that data, what
March 2022
techniques to use to analyze the data, how to validate
the system, and what information to emphasize
or deemphasize when presenting the analysis to the
judge [1]. However, because the recommendation
is generated by an automated system using some
mechanistic process and appears in stark, computational
form, the outcome can have the misleading
appearance of near-mathematical objectivity
within law.
Because of this aura of mechanistic objectivity,
there may be an inclination on the part of judges and
others to give more deference to computer-based
recommendations, in contrast to comparable
human-based assessments. This human tendency
to unduly ascribe value neutrality to technological
decision making (as compared to similarly situated
humans), and to defer to the seeming precision
of mathematical and data-based analysis, should
be queried closely in the context of technological
systems that influence substantive legal outcomes
affecting the lives of people.
Building upon the foundational points of embedded
values just discussed, the next part will examine
areas of concern particularly related to artificial
intelligence within law.
Artificial intelligence, law, and
embedded values
Recently, systems used within the legal context
have begun to incorporate techniques from artificial
intelligence including machine learning and formal
rule representation. Artificial intelligence raises some
novel issues of embedded values than those encountered
with previous technologies. This part will survey
some common issues of embedded values in
legal systems that incorporate artificial intelligence.
Designers of legal technological systems should take
special care with an eye to the embedded patterns
discussed. This is because the law has a unique place
in ordering society and in safeguarding the rights and
liberties of societal members. When technologies
used within the legal system contain subtle value
preferences, this can have an outsized impact on
society at large.
Biases in data and models
Legal technological systems that incorporate
machine learning approaches raise unique values
issues. Machine learning refers to a body of
techniques from artificial intelligence involving
69
IEEE Technology and Society Magazine - March 2022

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - March 2022
