Instrumentation & Measurement Magazine 23-4 - 58

obtained, and the patch
with maximal energy was
selected as the final patch.
Next, a raw pulse signal,
over time, was generated
using the average on all the
points in the final patch.
The feature set was performed by windowing the
raw pulse signal. For this, a
window of 10-sec size was
shifted each second, and
the DFT was computed in
each window. In this way,
we have a DFT feature vector to characterize the beats
per minute (bpm) in each
second of the signal.

HR Estimation

Fig. 2. Block diagram of the video magnification method using the HT.

using the HT over the chrominance channel of the image sequence. It is carried out by a convolution, in each spatial
direction, of each frame of the video with the Hermite filters,
that are defined by the 1D Hermite polynomials multiplied by
a Gaussian window, obtaining filters of Gaussian derivatives
until an k-th order [12]. Second, an ideal band-pass temporal
filter is applied in DFT domain to isolate the motion components of interest (e.g., eliminating the tremor movements).
Third, the isolated motion components are multiplied by the
magnification factor α. Fourth, using the inverse HT, which
uses similar filters to those used in the decomposition, a reconstruction of the image sequence is carried out. Finally, the
magnified sequence is added to the original sequence.

ROI Detection and Feature Extraction
The next steps of the proposed method consist of extracting
the raw pulse signal from the video magnification result and
performing feature extraction. To extract the raw pulse signal, first, the resulting magnified sequence was divided into
patches of 10-by-10 pixels in each time instant. Then, in each
patch in a given time, its average chrominance was calculated.
Later, a temporal average of chrominance for each patch was
58

A feed-forward NN is employed for estimating the
HR. In that sense, the NN
receives an input feature
vector of N elements representing the windowed
DFT of the extracted signal, and it estimates a real
value corresponding to the
HR estimation. The topology of the NN considers: an
input layer of N neurons,
then two hidden layers of
10 neurons each, and lastly
an output layer of one neuron. The activation function used in
the neurons at the hidden layers is the hyperbolic tangent sigmoid, and the one for the output layer is the linear function.
For training, we employed a custom dataset. In a nutshell,
it consists of a set of five video recordings (attempts) with 2
min length focusing on the wrist of five subjects, and the corresponding HR measurement in bpm. Using this dataset, we
extracted the DFT feature vectors. Then, for each subject, the
training set contains DFT feature vector samples from four
attempts as inputs and their corresponding HR samples as
outputs. To this end, we train the NN with the backpropagation method minimizing the error between the target and
estimated outputs. For testing, we employed the remaining
attempt of the dataset for each subject. Finally, the time-series
of the HR estimation is post-processed using a 3-size average
filter.

Description of the Dataset
We have used a personal contact pulse monitoring device to
measure the reference pulse signal in bpm while recording a
video with standard RGB camera at 30 frames per second (fps).
We have recorded five healthy subjects and five sessions per

IEEE Instrumentation & Measurement Magazine

June 2020

Instrumentation & Measurement Magazine 23-4

Table of Contents for the Digital Edition of Instrumentation & Measurement Magazine 23-4