Instrumentation & Measurement Magazine 24-6 - 14

progressive neurodegenerative dementia and those with functional
memory disorders with 90% accuracy [4].
Perhaps even more promising is the use of automated
speech analysis to detect mild cognitive impairment (MCI),
a period of subtle decline which can occur prior to dementia.
Roark et al. distinguished between patients with MCI and
healthy older controls with an area under the receiver operating
characteristic curve (AUC) of 0.86 based on measures
relating to speech rate and pause rate, as well as syntactic measures
[5]; similar results have subsequently been obtained by
other research groups [6], [7].
We now turn to the practical details involved in speech
analysis. Note that the term speech analysis actually encompasses
a wide range of lower-level features when we delve
into the details of the processing. Fig. 1 offers a non-exhaustive
summary of the features that can be extracted from speech and
linked to cognition. Such features can include low-level acoustic
metrics of voice quality and regularity, prosodic features
that relate to pausing and speech rate, as well as summary features
to assess how much time is spent speaking. For example,
OpenSMILE (https://www.audeering.com/opensmile/) is
an open-source tool that can be used to extract 1582 acoustic
features from the Interspeech 2010 Paralinguistic Challenge
feature set. While a complete description of the features in Fig.
1 is outside the scope of this article, detailed feature descriptions
are available in the literature, e.g., [3]-[6], [8], and [10].
Audio files can then be automatically transcribed using
automatic speech recognition (ASR) software, e.g., Kaldi
(https://kaldi-asr.org/), which is an open-source speech
recognition toolkit. By analyzing the transcript, a wealth of
additional information becomes available: features relating
to the complexity and grammaticality of utterances (syntactic
features), word repetitions, the variety of different words
used and their linguistic properties (semantic features), and
even the speaker's ability to carry on a conversation and
handle misunderstandings (pragmatic features) are all indicative
of cognitive health. The interested reader is referred to
[8] for a more complete description of the commonly used features
listed in Fig. 1. COVFEFE is an open-source package for
extracting approximately two thousand acoustic, prosodic,
syntactic, semantic and pragmatic features [8]. This package
can be applied to any studies that include linguistic data.
While individual features could be linked directly to cognitive
status, in most current research the features are combined
in a machine learning framework. The machine learning models
used in the literature vary from simple logistic regression
and decision tree classifiers, to state-of-the-art deep learning
models. Such models vary in terms of complexity and interpretability,
and the choice depends on a number of factors
including how much training data is available, the data representation
or number of features, and whether the output
should be categorical (classification) or continuous (regression).
The Python package scikit-learn is one widely-used
option for machine learning (https://scikit-learn.org/).
Thus, a typical analysis pipeline would involve recording
the speech, generating text from the speech, extracting features
from the audio and/or text streams, and then feeding the
features to a machine learning model to output an estimate of
cognitive status. Evidently, there are numerous sources of error
and uncertainty along that pipeline that may propagate
through the system.
Current State-of-the-Art
In their discussion on uncertainty in medical measurements,
Parvis and Vallan enumerate three distinct categories of medical
measurement instruments: those that directly measure
some physical quantity of clinical interest; those which convert
the extracted quantities into more interpretable features;
and those which combine several quantities into a single indicator
[9]. Clearly, the measurement of cognition from speech
falls into the last and most
complex category.
Accordingly, there are
a number of concerns that
arise when we consider
the current state-of-theart
in using speech as a
measurement instrument
for cognition including
the reference standard, repeatability,
resolution and
sensitivity of the system.
Fig. 1. Example features extracted from speech data for the measurement of cognitive status.
14
IEEE Instrumentation & Measurement Magazine
Reference Standard
In many cases, researchers
consider a brief cognitive
screen, such as the MiniMental
Status Exam
(MMSE), to be the reference
standard. In addition
to the general problems of
September 2021

https://scikit-learn.org/ https://www.audeering.com/opensmile/ https://kaldi-asr.org/

Instrumentation & Measurement Magazine 24-6

Table of Contents for the Digital Edition of Instrumentation & Measurement Magazine 24-6