Instrumentation & Measurement Magazine 23-3 - 33

Size of data is celebrated while only their usefulness in decision-making is of value.
Principles, methodologies and techniques of metrology are crucial to ensure proper and effective use of data in decision-making.
Spreading of dataism (i.e., unconditioned belief on data) is boosted by lack of metrology culture.

suitable knowledge about the underlying problem are needed.
Speaking the sharp language of the Star Wars movie saga, considering only the amount of data lead us to fall into the dark
side, that is, a seemingly easy and seductive way to reach the
desired objective, but also an extremely dangerous and potentially very expensive choice.
Making important and complex decisions based on data
that are not critically analyzed may lead to catastrophic consequences. Let us consider, for instance, the catastrophic
effect of an improper evaluation of in-flight sensor data reliability in the recent Boeing 737 Max disaster [6], [7]. Also, the
assessment of research quality based only on bibliometric indicators is not devoid of possible serious consequences, even
if less tragic and dreadful. Interested readers can find many
emblematic situations in which the uncritical use of indicators lead to absurd and detrimental decisions in references
[8] and [9].

Flaws of Uncritical Data Usage
Even if motivated by the best of intentions, a bad use of data
might induce many flaws and dysfunctions that often leads
to deceptive conclusions, especially when complex problems
have to be addressed. Among the many flaws due to uncritical use of data, some of the most common are the following
ones [9]:
Measurability Bias: due to a tendency to prefer the most easily
measurable factors rather than the most relevant ones for the
problem considered;
Goal Displacement: people's attention and efforts tend to be
focused on measured factors, often at the expense of other, possibly more important, unmeasured factors;
Promotion of Short Termism: long-term effects are difficult, or
even impossible, to be fostered from data because they depend
on something that is still unknown. Consequently, data might
promote short-term goals, while stifling radical innovation
and creativity, valuable qualities in most settings; risk taking
and long-term investments are also discouraged, possibly resulting in stagnation and work gratification decline;
Manipulation of Indicators: through a variety of improper practices, such as: avoiding situations that might have negative
impact on performance (practice known as creaming), failing
to report negative instances (omission), lowering standards
in order to improve scores, and even altering data to fabricate
false evidence (cheating);
May 2020	

Degradation of Cooperation and Common Purpose: data tend
to promote competition rather than cooperation and common
purpose, which are both based on unmeasurable-or hardly
measurable-motivations.
It is also worth noticing the potential highly negative impact that wrongly used measurements might have in sensible
sectors, especially when socially relevant purposes are involved, such as education or healthcare [8], [9].

Proper and Effective Data Usage
To ensure proper and effective use of data we need to master the principles, methodologies and techniques that enable:
◗◗ the discrimination between what is relevant for the
addressed problem from what can be neglected, remembering that not everything that is important is measurable,
and conversely, what can be measured is not always what
is worth measuring (or, using the words of a familiar
saying, "not everything that can be counted counts, and
not everything that counts can be counted");
◗◗ the critical interpretation of information acquired with
measurement, by recognizing the limits of the models to
which data (explicitly or implicitly) refer; indeed, models
always provide a partial description of a portion of reality
that depends on the intended purposes, prejudices and
knowledge limits of model designers;
◗◗ the identification of all uncertainty sources that affect
measurements significantly, in order to evaluate trustworthiness of available information, its impact on the
derived conclusions and, eventually, of the risk of wrong
decision-making.
It is thus clear that, except in the simple case of single
aimed, fully structured, and fully informed decision-making activities [10], generally measurements should not replace
expert judgment. Indeed, serious problems might arise when
data are uncritically used in automatic decision criteria. Conversely, measurements should support expert judgment,
which needs to properly take into account the relevance of
data with respect to what has not been measured. Moreover,
measurement demands judgment: judgment about whether to
measure, what and how to measure, how to assess the significance of the obtained results, and how to properly use
them [9].

Principles of Core Metrology
The principles, methodologies and techniques mentioned
above are at the core of metrology, the science of measurement
and its application [10]-[12]. Unlike the dataist, the metrologist,
leverages on the metrology body of knowledge, and:

IEEE Instrumentation & Measurement Magazine	33
Instrumentation & Measurement Magazine 23-3

Table of Contents for the Digital Edition of Instrumentation & Measurement Magazine 23-3
