Instrumentation & Measurement Magazine 23-6 - 12

among researchers and practitioners, when DL-based visual
object recognition systems such as IDSIA and AlexNet beat
out their competition by huge margins in terms of improved
error rate [9], thanks partly to GPUs providing the significant
speed up needed by DL. Another reason for their popularity is
that DL algorithms automate the process of extracting discriminative features in a dataset, which usually requires domain
knowledge and significant human efforts [10]. This is why DL
has revolutionized Applied AI in recent years. To understand
how DL can achieve such great results, and how it is being
used in I&M, let us now take a look at the basics of DL.

Deep Learning Basics

that GPUs can assist tremendously. Finally, the output layer
produces the final output of the neural network.
How a DL algorithm learns about features and manages
to extract them depends on the availability of the training dataset. Like any machine learning technique, the DL approach
of learning can be done as either supervised learning or unsupervised learning. In supervised learning, the dataset is
comprised of labeled data. That is, we deal with input data for
which the respective output is known and defined. In unsupervised learning, however, we deal with unstructured data
for which the output is unknown.
Training the DL network with the training dataset works as
follows. As data flows from the input layer to the first hidden
layer, the neurons in this layer use their activation functions to

A basic understanding of neural networks, as described in [3],
can be helpful to the readers here. A DL algorithm is based
on a layered architecture of data representation in which the
high-level features are extracted from the last layers of the neural network while the low-level features are extracted from the
lower layers [11]. The true inspiration behind such an architecture is to mimic how the biological brain works-as brains
extract data representation from an input, e.g., scene information from the eyes, the generated output is a classified object [11].
One main capability of DL is to extract complex features
from a huge amount of data and discover hidden patterns and
trends in them. This is achieved by the utilization of a neural
network that is constituted by a set of interconnected neurons
(computational or processing units). A single neuron receives
data from inputs or other neurons, multiplies them with
weights and then feeds them through an activation function to
produce an output as depicted in Fig. 2.
In a neural network, neurons are organized into an input Fig. 2. A depiction of the operation of a neuron: it receives inputs (like x1 and
layer, one or more hidden layers, and an output layer as shown x2), multiplies them with weights (like w1 and w2), sums up the multiplication
results, and finally passes the sum to an activation function that produces the
in Fig. 3. The input layer is composed of input neurons that feed output f (x,w).
in the input data. A hidden layer receives its input
data either from the input
layer neurons or the neurons of a preceding hidden
layer. Hidden layers perform intensive processing
of the data that were originally supplied by the input
layer. Apparently, the more
hidden layers the neural
network has, the deeper and
more intense processing
the data go through. In case
the neural network architecture is designed with
two or more hidden layers,
we end up with a DL architecture. Definitely, the use
of more hidden layers entails the need for higher
computational and proFig. 3. Architecture of a neural network. The W1, W2, ... , W14 parameters are the weights of the connections.
cessing capability, an area
12	

IEEE Instrumentation & Measurement Magazine	

September 2020
Instrumentation & Measurement Magazine 23-6

Table of Contents for the Digital Edition of Instrumentation & Measurement Magazine 23-6
