Instrumentation & Measurement Magazine 25-1 - 21

Fig. 1. High level block diagram of VM and rPPG.
faces many challenges including
its sensitivity to
motion, light conditions,
and its bias to skin tone [3],
[8]. Additionally, a limitation
of the VM algorithm is
that it amplifies noise as the
amplification factor is inthe
frame and then undergoes temporal processing to assess
variations within the ROI over time for detection of heart rate
or respiratory rate.
VM/rPPG Algorithm Overview
Although the assessment of heart rate through VM [2] or rPPG
[3] methods is relatively recent, with the first papers being
published within the last decade, research on remote assessment
of heart rate has been underway for nearly 20 years [1].
Despite considerable progress and recent advances, there remain
several hurdles that must be overcome before VM/rPPG
techniques can be widely adopted for vital sign measurements
and health assessment. Key challenges include signal extraction
from sensor data that has poor signal-to-noise ratio and
limited dynamic range, or that contains subject motion, ambient
light variability and interference, and reflection.
Video Magnification enables the magnification of subtle
color and motion changes in videos that are imperceptible to
the human eye [2] and shown in Fig. 2. This technique works
by first selecting an ROI (such as facial skin) and spatially
processing the video into different spatial bands using a Laplacian
Pyramid and then temporally filtering the different
pixel color values using a custom-fit band-pass filter to focus
on a range expected to include the signal of interest. The filtered
signal is then amplified using an amplification factor to
reveal the minuscule color and/or motion changes. The original
work included reconstruction of the video with amplified
signal features and was called Eulerian Video Magnification.
In many applications, the VM algorithm is used without reconstruction
to non-invasively and unobtrusively detect heart
rate in humans from their videos. However, this technique
creased, and the amplified noise could be falsely interpreted
as a biological signal.
The rPPG algorithm and method [3] is similar in structure
to VM and focuses on the chrominance changes in the skin during
the pulse wave associated with the variation in the color of
the reflected light caused by the variation in the skin color during
the pulse wave (Fig. 2). This initial work included some
of the foundational theoretical models for dealing with variance
in the lighting over time, including both luminance level
and light source color, and proposed a method to rebalance the
RGB levels. The rPPG and VM methods both focus on assessment
of the subtle changes in skin color that occur during the
pulse wave, and researchers are exploring ways to address the
many challenges with this assessment.
Several recent review papers [9], [10] have summarized the
work on VM and/or rPPG methods, and actual implementation
methods for rPPG [11] and VM [12] provide technical
guides on how these methods can be implemented. The balance
of this paper will explore each of the stages shown in
Fig. 1 and review the emerging methods for each to improve
performance.
Video Camera Selection
The rPPG and VM methods have been applied to a diverse set
of video cameras ranging from high performance and highquality
cameras down to low-cost web cams; representative
citations are provided in Table 1. In addition to conventional
red/green/blue (RGB) color cameras, researchers have explored
color video camera variants including cameras that
support depth vision [5] and cameras that have wavelength
specific filters leading to monochrome images focused on specific
red and infrared color bands (red: 675 nm; IR: 800 nm and
842 nm) [6]. Infrared thermal cameras have been evaluated as
an alternative to RGB cameras to overcome issues related to
Table 1 - Example papers for different video camera
technologies evaluated for VM/rPPG applications
Category
Full color RGB video cameras
RGB video cameras augmented with
depth vision
Monochrome video cameras focused on
specific wavelength bands
Fig. 2. Visible or Infrared light experiences reflection and diffusion when it
interacts with the skin. The result is some scattering of the light by the skin.
The skin color will affect the color of the reflected light while small motions
will also affect the reflection captured by the camera.
February 2022
Infrared thermal cameras including near
and far-infrared bands
Night vision cameras
IEEE Instrumentation & Measurement Magazine
Reference
[2], [3], [13]-[15]
[5]
[6]
[5]-[7]
[7]
21
Instrumentation & Measurement Magazine 25-1

Table of Contents for the Digital Edition of Instrumentation & Measurement Magazine 25-1
