IEEE Systems, Man and Cybernetics Magazine - July 2022 - 34

uncertainty. However, there is not yet a comprehensive
study to support this claim. Recent studies have clearly
shown the inherent uncertainty in handcrafted conventional
ML- and DL-based predictions and its close relationship
with public trust in science [57]. It relies on the
fact that decision making can gain our trust when accompanied
by the least uncertainty. For example, one of the
fundamental criticisms related to the accuracy of handcrafted
conventional ML and DL models presented for
COVID-19 detection is that these models ignore considering
their uncertainty. In other words, the weakness of AIbased
models emerges as they cannot clearly show the
uncertainties or propose a solution to deal with them
neither in the early stage of model making nor in the
stage of making predictions. Therefore, treating and
quantifying uncertainty can be considered a form of
transparency [58].
As displayed in Figure 2, there is a small boundary
between trust and mistrust related to uncertainty.
Accordingly, this can be generalized to ML and DL methods.
If various proposed ML and DL methods cannot pay
attention to these uncertainties in their model and its
predictions, it can lead to distrust. But perhaps it can be
argued that this issue is becoming much more prominent
in the medical world. Trust in the predictions and
recommendations of ML and DL models becomes more
questionable unless the uncertainty is taken into
account. This is important because of the close relationship
of these predictions and recommendations with
patients' lives. This makes perfect sense as physicians
prefer not to take risks when people's lives are in danger.
This is the main point at which the uncertainty measurement
and quantification techniques help increase
the level of trust in how these methods are developed
and the obtained results by considering the lack of
knowledge when developing any ML and DL model or
reporting the results (predictions). Therefore, UQ and
estimates can help people make better decisions
because individuals may not correctly understand
uncertainty or may not have advanced skills [58]. Therefore,
in ML and DL, we use the term uncertainty to refer
to humans' lack of knowledge regarding some important
results of interest [58].
The Need to Quantify Uncertainties
in ML/DL for Medical Data Analysis
When discussing trust in medicine, the importance of
trust in the decision-making process becomes more serious.
Trust-based decisions are critical for medical data
analysis and for different applications. For this reason,
decisions must be made with certainty or with the least
uncertainties. An accurate clinical diagnosis is a critical
task that can refrain from an overconfident, incorrect
quantification. This allows clinicians and medical
experts to execute subsequent revisions in cases with
high uncertainties [59].
Dealing with high uncertainty (or low confidence) in
Model Training
Malignant
OR
Model Testing
Begin
Unseen
Input LayerHidden LayerOutput Layer
P is the Prediction of the Model
DL
P (Malignant)
Input LayerHidden LayerOutput Layer
P (Benign)
DL
P (Malignant) = 0.2
P (Benign) = 0.8
Figure 4. A normal prediction procedure for seen and unseen
samples. This figure shows a DL model used for skin cancer binary
classification.
34 IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE July 2022
handcrafted conventional ML and DL methods regarding
their medical data predictions plays a significant role in
having more reliable machine-based models. As an ideal
goal, we prepare models for expressions " I do not know "
or " I am not sure. " This seems to be a fundamental necessity
for the safe deployment of such AI-based medical systems.
This gives an important capacity to refrain from
providing a diagnosis or prediction during uncertainty.
And this results in having a second opinion to avoid having
unpredictable outcomes.
As depicted in Figure 4, the model needs to make a
prediction anyway, even when it is not
sure of the results. This problem appears
in most classic handcrafted conventional
ML/DL approaches. In other words, many
AI methods show outstanding performance
and have significant weaknesses
in measuring uncertainties. One solution
to deal with such problems is to apply UQ
methods. The models encompass both the
objective and subjective information
derived from clinical data and clinical
experiences, respectively [60]. It can be
noted that exclusively data-driven models
(i.e., ML and DL medical data analysis
models) have extremely limited applicability
as they overlooked the clinical
experience. The various models used in
medicine continually approximate different
realities, but most of them do not consider
reality itself.
These models are often dependent on
the state of current knowledge about a set
IEEE Systems, Man and Cybernetics Magazine - July 2022

Table of Contents for the Digital Edition of IEEE Systems, Man and Cybernetics Magazine - July 2022
