Instrumentation & Measurement Magazine 24-3 - 81

Angular Softmax Loss: An alternative is the Angular Softmax [5] for which the weights and biases terms wTi x  bi are
rewritten as wiT x cos  i  with the removal of bias term and
 j i 0   j ,i   being the angle between vectors wj and xi. The
weights are normalized such that w j  1, and a marginal term
m with m>2 added. When m is increased, the angular margin
increases. The Angular Softmax Loss is:



Lang 



1
N






exp xi cos m yi ,i


log


 exp xi cos m yi ,i   j y exp xi cos  j ,i
i








 

 (2)

.



Experiments
We first investigate how the object detectors perform for multiface detection before looking at the complete tracking pipeline.
For multi-face detection, we compare MTCNN to two common object detectors, SSD and R-FCN, in combination with
three feature extractors: SSD Inception V2, SSD MobileNet V1
and R-FCN ResNet 101. We use the two face datasets WIDER
FACE [28] and FDDB [29] to train these detectors. R-FCN
ResNet 101 Inception V2 and SSD Inception V2 are trained on
WIDER FACE; SSD Inception V2 and SSD MobileNet V1 are
trained on FDDB. We initialize these networks with weights
based on training on the COCO dataset [10].
We train MTCNN on WIDER FACE [28] dataset with the
Tensorflow Framework to have comparable results to the general purpose object detectors. All three cascaded networks
of MTCNN (P-Net, R-Net and O-Net) are trained on WIDER
FACE for face detection and CelebA [30] for facial landmark
detection.

Multi-Face Tracking Evaluation
Our main work is the evaluation of the actual multi-face tracking phase. We first compare our Deep SORT adaption for faces
to two state-of-the-art offline multi-face trackers [1], [2] and
two online trackers [31], [32]. Here, we compare two different
classifier loss functions with each other. We also evaluate our
Deep SORT adaption on a new video benchmark and compare
it to the IOU-Tracker [32].
Face tracking in unconstrained environments is not well
handled by distance only based trackers, so we only use the
face feature distance in our Deep SORT adaption for the association cost term and the spatial distance is only used for
gating. First, the feature classifiers in the multi-face tracking system need to be obtained by training the deep feature
CNN with a classifier loss function. We randomly select 700
different celebrities' faces from YouTube Faces [33] as the
training set and randomly select another 700 celebrities'
faces from the rest as the testing set. We split off 10% of the
training data for validation. We crop out the faces from the
given face ground-truth bounding boxes and label the face
ID from 0 to 699 for both, so the training and testing sets are
based on the celebrities' names in alphabetical order. Finally,
we train with the different feature classifier loss functions
that we have previously introduced. The deep feature CNN
May 2021

consists of 15 layers which is the same as in the original Deep
SORT [3] method.
The Music Video dataset [1] and our benchmark House
of Common dataset are used to evaluate the overall multiface detection and tracking system. While our tracker is not
specialized on music videos, it will nevertheless give a good
indication of its general performance because music videos
take place in unconstrained scenes. The details of the datasets
are presented as an Appendix at the end of the paper.
The hardware used in all of these experiments is an Intel
Core i7-7770k CPU with 16 GB system memory and a NVIDIA
GeForce GTX1060 GPU. The experiments are conducted with
Tensorflow1.12.

Results and Analysis
We present the multi-face detection results prior to proceeding
to the complete multi-face tracking pipeline since the detection
results have a large influence on the tracking.

Multi-Face Detection Results and Analysis
The training results are presented for MTCNN, SSD and RFCN in Table 1 when trained on WIDER FACE. Table 2 shows
results for two SSD variants trained on FDDB. All results are
shown at an average precision at 0.50 Intersection over Union
(IOU) and 0.75 IOU.
For the detectors trained on WIDER FACE, we observe that
MTCNN achieved the highest average precision (AP) at both
0.50 IOU and 0.75 IOU, and R-FCN ResNet 101 achieved the
second highest AP and was the highest among three common
object detectors. R-FCN has lower losses in both classification
and localization than SSD. The results of detector trained on
FDDB reveals that SSD Inception V2 achieved higher AP at
0.75 IOU, and SSD Inception V2 also has lower bounding box
localization and classification losses than SSD MobileNet V1.
Therefore, we use the following detectors to assess their performance in face tracking: (1) MTCNN trained on WIDER

Table 1 - Average precision results of detectors
trained on WIDER FACE
Detectors

Average
Precision at 50%
IOU

Average
Precision at 75%
IOU

MTCNN

96.50

73.04

R-FCN ResNet 101

90.31

67.10

SSD Inception V2

84.76

55.55

Table 2 - Average precision results
of SSD trained on FDDB
Average
Precision at 50%
IOU

Average
Precision at 75%
IOU

SSD Inception V2

97.23

76.22

SSD MobileNet V1

97.70

66.33

Detectors

IEEE Instrumentation & Measurement Magazine 81

Instrumentation & Measurement Magazine 24-3

Table of Contents for the Digital Edition of Instrumentation & Measurement Magazine 24-3