Instrumentation & Measurement Magazine 24-1 - 7









where:
◗◗ P represents probability
◗◗ Hp represents the prosecutor's hypothesis
◗◗ Hd represents the defense hypothesis
◗◗ E refers to the evidence (i.e., observing a correspondence
between the profile of the crime scene trace and the reference profile of a suspect)
◗◗ the vertical bar | characterizes the conditional probability one is interested in (the probability of an event given
knowledge of the occurrence of other events), for example
the probability to observe an item of evidence, E, given
that the prosecutor's hypothesis is considered true, Hp,
and given some background knowledge about the case, B.
This equation represents the odds form of Bayes' theorem.
In this framework, DNA results are rendered in the form of a
Bayes factor, P E|H p , B P  E|Hd , B  , that expresses the probability of observing the evidence if the suspect, rather than
somebody else in the suspect population, is the source of the
trace. Bayes factor can thus be conceptualized as a metric that
provides a balanced measure of the degree to which particular
evidence, independent of its nature, is capable of discriminating among competing hypotheses put forward by opposing
parties at trial. Such reasoning is considered normative in the
sense that it prescribes a standard that, if followed, allows reasoners to avoid logical fallacies.
Consider, for example, a Bayes factor of 1 million. This
means that the expert's observations are 1 million times more
likely if the suspect is the source of the trace than if somebody
else in the population of interest is the source. The Bayes factor
has a value comprised between 0 and infinity, with 1 being the
neutral value that expresses the fact that the evidence does not
support either of the hypotheses, Hp or Hd, over the other. The
evidence can be said to be irrelevant because it does not allow
one to discriminate between the hypotheses. The Bayes factor
can be combined with the other elements in the case-quantified through the so-called prior odds, P H p|B P  Hd|B  -to
assess the probability that the hypotheses of interest are true,
given all the evidence considered; this is expressed by the posterior odds, P H p|E, B P  H d|E, B .
The advantage of the Bayesian approach is that it is transparent, in that it informs the decision maker of the amount
of uncertainty inherent in the measurement of the probative
value of the evidence considered; moreover, it allows one to
coherently update one's uncertainty on the hypotheses of interest. It also makes it clear what the respective roles of the fact
finder (opining as to hypotheses) and the expert (assessing
the evidence under competing hypotheses) are. One limitation, however, is that the practical application of this approach
is still widely neglected in real casework due to a lack of understanding as to its merits. And one of the main criticisms
towards the Bayesian approach is that it is too difficult for lay
people such as judges and jurors to understand (which, we
would argue, is not a sufficient reason for not using it).







February 2021



Unknown Error Rates
The Bayesian approach theoretically allows one a scientifically coherent assessment of the evidence and of the case as
a whole. But in practice, it still falls short of the mandate. The
reason is that there is still very little data related to error rates
in the forensic field. For the most part of the 20th century, forensic practitioners managed to convince judges and courts that
(contrary to all human beings) they never made any mistakes.
In the 1990s, this position became untenable, though: the Innocence Project, initiated by two law professors in New York,
Barry Scheck and Peter Neufeld, started reopening cases of
people who claimed to have been wrongfully convicted and
reinvestigating them with the help of DNA evidence. It quickly
became clear that forensic science evidence was a major cause
of wrongful convictions in the United States: the criminal
justice system was regularly misled because forensic practitioners mislabeled or exchanged specimens, contaminated crime
scene or reference samples, made computational errors, used
a faulty paradigm, and repeatedly claimed that their results
were certain instead of applying a model that dealt explicitly
with uncertainty [5].
In the mid-2000s, the reputation of forensic science had been
sufficiently damaged by the number of so-called DNA exonerations that the U.S. National Research Council (NRC) decided
to conduct a systemic evaluation of the field. The report, published in 2009, revealed that many forensic disciplines were
insufficiently validated and produced results that were not
scientifically robust [6]. One infamous example was forensic
odontology, specifically the comparison of bite marks. For a
long time, self-proclaimed bite mark experts would match bite
marks found on victims of assault, rape or murder with the dentition of a suspect. Without any data related to the validity and
accuracy of their work, they would declare that the suspect had
in fact bitten the victim to " a reasonable degree of scientific certainty " or " indeed and without doubt. " As it later turned out,
many bite mark experts were not able to distinguish a bite from
a bruise, or a human bite from an animal bite, let alone identify
the individual who was the source of a given bite mark [7].
The publication of the 2009 NRC report had several consequences for the forensic field. Some disciplines, such as bite
mark comparisons, were abandoned in certain jurisdictions
and are not used in criminal prosecutions anymore. Other
forensic fields have heeded the warnings of the NRC and begun large-scale validation studies. Researchers in fingerprint
comparisons, for example, have endeavored to develop false
positive and false negative error rates, mainly through black
box studies [8]. In forensic DNA analysis, some laboratories
have started conducting their own studies into errors, such
as contamination of crime scene samples [9]. There remain
some problems, though [10]. First, efforts to assess error rates
are fragmentary and depend entirely on the goodwill of individuals and laboratory managers. Second, the results of such
studies are only published if it is convenient to do so, and
some incidents suggest that unsatisfactory results are simply
hidden away. Third, expert reports do not usually contain an
assessment of the probability of a false positive result, i.e., the

IEEE Instrumentation & Measurement Magazine 7

Instrumentation & Measurement Magazine 24-1

Table of Contents for the Digital Edition of Instrumentation & Measurement Magazine 24-1