Instrumentation & Measurement Magazine 25-1 - 23

rate detection or around the nose and mouth for respiration
rate detection.
In another method, the image was divided into a grid fixed
to the image frame [20], and each of these candidate ROIs was
independently processed for the respiration rate. The algorithm
then adaptively selected ROIs based on an SNR estimate
of the respiration signal in the candidate ROI for a final respiration
rate estimate. An example of this is shown in Fig. 5, where
an image has been divided into a grid of potential ROIs. The
VM method is applied to each of the regions, and the resulting
signal was analyzed to determine if it contains a potential heart
rate or respiration rate. The second part of the image shows
some of the regions that could have a potential heart rate signal
in blue and those that have a potential respiration rate in green.
The use of fusion of two different camera types has also
been proposed, such as the use of a visible light camera in conjunction
with a thermal camera, where the ROI is derived from
face detection and tracking using the visible light image, and
the landmarks are then used to identify the nose and mouth
areas in thermal video for respiration rate analysis [21]. The
impact of small and larger motion on heart rate assessment
was analyzed for a VM application [17], and the work showed
that a Gaussian Pyramid improved performance over the Laplacian
Pyramid for small motions but not for large motions.
The computational load associated with VM and rPPG algorithms
can be significant and can be highly varied, as the use
Fig. 6. Temporal processing block diagram.
of larger ROIs leads to the processing of larger numbers of pixels
while the inclusion of methods for motion tracking such as
face detection also increases the computational load [22].
Temporal Processing
The results of spatial processing are then passed into temporal
processing where the frame to frame (image to image) variation
of the ROI is analyzed within the time/frequency domain
to determine the relevant heart or respiration rate. Temporal
processing follows the flow model shown in Fig. 6, and recent
advancements are summarized in Table 3.
For heart rate assessment, small color changes in the skin
associated with the presence or absence of capillary blood
within the pulse wave provide a signal that can be assessed to
determine heart rate, and an example set of results is shown in
Fig. 7. Through spatial selection of a region of skin such as the
forehead, the signal from all of the pixels within a given frame
is combined, leading to a feature set for each frame such as average
red, green, or blue pixel color. The choice of color and
wavelength is important as the performance of rPPG and VM
methods within green wavelengths for heart rate is better than
Table 3 - Temporal Processing Algorithmic
options that extend beyond the base rPPG / VM
proposed methods
Temporal Processing Extension
Two stage processing using wide bands
followed by narrow more focused bands.
Introduction of a skin reflection model to
manage skin/light variation
Analysis of background image to provide
estimate and compensate for lighting shift
Application of auto-regressive model as the
analysis method
Assessment of the effects of noise
Fig. 5. Example location of the Regions of Interest based on a grid-based
method locked to the image. (a) A set of fixed regions and the pixels from each
region are processed; (b) ROIs with potential heart rate (blue) and potential
respiration rate (green).
February 2022
Dynamic selection of a subset of ROI where
higher quality (better SNR) regions for
signal of interest uses
Application of HRV to provide respiration rate
estimate
IEEE Instrumentation & Measurement Magazine
Reference
[27]
[8]
[24]
[25]
[26]
[28]
[15]
23
Instrumentation & Measurement Magazine 25-1

Table of Contents for the Digital Edition of Instrumentation & Measurement Magazine 25-1
