Instrumentation & Measurement Magazine 24-2 - 89

Table 2 - Comparing I&M and ML terminologies
ML regression
Term

True value

True model

Reference
value

Error

meaning in I&M

meaning

the true quantity value of
the measurand; true value is
unknowable [11, 2.11 Note 1]

the true value of the output for
the given input; if the input
exists in the dataset, the true
value is also in the dataset and
therefore known, otherwise it
is unknown

not an official term, and
not used in any standard or
guidelines.

the real f in Fig.1, not the fˆ
estimated by the ML model
[12, p.12]; it might be possible
to determine the true model by
analytical modeling

in the context of a
measurement instrument:
value serving as agreed upon
reference for comparison [15,
3.5]

not widely used; sometimes
used as another name for true
value

in the context of measurement
results: the true value [11, 5.18]

not widely used; sometimes
used as another name for true
value

measured value minus the
reference value [11, 1.26]; can
be expressed as systematic error
+ random error

As training error a.k.a. insample error: the value of the
cost function for the best fit
model [12, p.7]; commonly
measured in RMSE, MSE, R2,
Absolute Error, Mean Absolute
Error, etc.
As testing or prediction error
a.k.a out-of-sample error: can
be expressed as bias2 + variance
+ noise [12, eq. 9]

ML classification
relation with I&M

both refer to what the true
outcome should be, whether
known or unknown.

same as ML regression

✔

✔

same as ML
regression

N/A

relation with
I&M

meaning

N/A

False friends!
Agreed-upon reference is not
the true value.

✘

same as ML regression

both refer to the true value.

✔
As single input error: when
considered for a single
input/output pair in ML
(whether training, testing, or
predicting), the error indicates
the difference between the
estimated output and the
reference value; so this is
similar to measurement error.

✔
As model error: False friends;
when considered for the whole
ML dataset (whether training,
testing, or predicting), the error
is not an indication of what
happens to a single specific
input.

rejection
of a true or
non-rejection
of a false
hypothesis,
known a
Type I or
Type II error,
respectively

False friends!

✘

✘

Systematic
error

the part of error that
remains constant or varies
in a predictable way
when repeating the same
measurement [11, 2.17]

For a training dataset: the
systematic error in a training
dataset has the same I&M
meaning.
For a prediction result: N/A;
not commonly used.

Fixed-output ML: for a single
input at run time, the ML error
between the estimation and
the true value contributes to
the measurement system's
systematic error.

✔
Variable-output ML: for
a single input at run time,
the average value of the
ML error contributes to
the measurement system's
systematic error.

✔

Random error

the part of error that varies
in an unpredictable way
when repeating the same
measurement [11, 2.19]

For a training dataset: the
random error in a training
dataset has the same I&M
meaning; a.k.a. noise in ML
(see definition of ML noise
below).
For a prediction result: N/A;
not commonly used.

N/A

Fixed-output ML: does not
contribute to the measurement
system's random error

✘
Variable-output ML:
for a single input at run
time, the variance of the
outputs contributes to the
measurement system's random
error.

✔

April 2021	

IEEE Instrumentation & Measurement Magazine	89
Instrumentation & Measurement Magazine 24-2

Table of Contents for the Digital Edition of Instrumentation & Measurement Magazine 24-2
