Instrumentation & Measurement Magazine 24-7 - 10

Audio Information Retrieval
and Musical Acoustics
Marco Olivieri, Raffaele Malvermi, Mirco Pezzoli, Massimiliano Zanoni,
Sebastian Gonzalez, Fabio Antonacci, and Augusto Sarti
I
nformation retrieval and machine learning have grown to
the point of playing a leading role in all aspects of sound
analysis. There are some research areas, however, in
which the potential of information retrieval techniques is only
now beginning to have an impact on the research community.
One of these areas is musical acoustics. In this area, in fact, machine
intelligence and information retrieval have been widely
used only for timbral analysis of the tone produced by musical
instruments, thus effectively objectifying an analysis that
is traditionally considered subjective. The role of information
retrieval and machine learning in the analysis of vibrational
and acoustic properties of musical instruments is a more recent
development. In this manuscript, we offer an overview
on methodologies for vibrational, acoustic and timbral analysis,
based on machine intelligence. We discuss some challenges
that are emerging today and will have to be faced in the near
future in order to make information retrieval and machine
learning become an integral part of the analysis process in musical
acoustics.
From Vibrational Analysis to Timbral
Evaluation
Musical acoustics is, by its very nature, a highly interdisciplinary
field. In fact, not only does it deal with the physics of
sounds, but it also considers how we perceive sounds; what we
experience as being musical; and how this definition is socially
construed. For example, the question " what makes a Stradivarius
violin so desirable an instrument? " clearly comes with
multiple facets. On the one hand, how does the physical object
produce sounds? What particular vibratory phenomena in
the wooden body of the violin do we associate with a " good "
sound? How is this sound structured in space and frequency?
On the other hand, it is equally relevant to wonder how such
descriptors influence the way we experience the sound beyond
its mere pitch and intensity; that is, how they determine
its timbre. The field is naturally divided into two main sub-areas:
vibroacoustics, which deals with the physical aspects of
sound; and timbral analysis, which studies the phenomenological
experience of sound and how this is related to the above
10
physical aspects. In this review we focus on both areas from the
common ground of data-driven methodologies and how these
can help with the analysis-an area in which our laboratory is
on the forefront of research.
Traditionally, vibroacoustics has always relied on a balanced
mix between experiments and simulations. Vibrations
in musical instruments can be measured through laser interferometry
[1] or, more commonly, through vibrational sensors
(piezoelectric or accelerometers) placed on key locations of the
vibrating surface [2]. In the case of violins, particularly those
of relevant historical value, it is customary to proceed in a noninvasive
fashion by minimizing physical contact, for example
by measuring their bridge admittance. Simulations usually
are based on finite element modeling, from shell modeling [3]
to fully-coupled air-body simulations [4]. All such models ultimately
share an abundance of unknown parameters and a
sheer quantity of data produced via simulation. Yet, until recently
[5], no statistical studies had been conducted on such
data and results were limited to just a few observables (for the
case of the violin the " body modes " ) [3], [6]. Today, feature extraction
algorithms combined with data-driven methods, from
Principal Component Analysis (PCA) to autoencoders, are beginning
to show their strength when compared with more
traditional methods. The literature devoted to experiments
in the context of musical acoustics has always taken inspiration
from the wider field of structural analysis, where the need
for robust solutions to be used in noisy and industrial scenarios
made room for methodologies that are still considered as
standard measurement procedures [7], [8]. Indicators based
on averaged cross-correlations over a set of noisy acquisitions
can offer a reliable technique for acquiring vibrational data
with low-cost equipment and lay the foundations for fair comparisons
between instruments and style classification starting
from vibration. Nonetheless, the introduction of feature representations
inspired by machine learning can improve the
interpretability in this family of applications and is presented
in the section on Feature-based Analysis on Vibrational Data
The primary goal of a musical instrument is to radiate
acoustic energy. Therefore, relevant effort has been put in the
IEEE Instrumentation & Measurement Magazine
1094-6969/21/$25.00©2021IEEE
October 2021
Instrumentation & Measurement Magazine 24-7

Table of Contents for the Digital Edition of Instrumentation & Measurement Magazine 24-7
