IEEE Systems, Man and Cybernetics Magazine - October 2021 - 28

data sets, only the images from the dog-versus-cat image
set are natural and straightforward enough to use for
observation and evaluation.
Therefore, the designed network is first trained with
the dog-versus-cat images to determine the hyperparameters.
After the training, the test images are fed
into the network for prediction and to simultaneously
obtain the heatmaps. For heatmap generation, the
weighted summation of all the feature maps from the
last step of the recur-convnet is used to form the index
matrix, which is used to select the values of a chosen
color map. The weights involved in the summation are
the ones accompanying the post-GAP layer. The representatives
of these achieved intuitive heatmaps, which
prove the effectiveness of the designed network, are
provided in Figure 4.
As the next step, the prepared EEG topographical data
are processed by the network. A leave-one-subject-out test
paradigm is considered here, which means that each time
the data of one subject are fixed, the data of all the remaining
subjects are used to train the network. The hyperparameters
are directly migrated from the case of the
training on the dog-versus-cat image set. Because, based
on these hyperparameters, the implicit attention induced
by the network can be intuitively observed, it is conjectured
that the consequent training with the same network
architecture on the EEG topographical data set could
unveil the implicit attention as well.
Image
Heatmap for Cat Heatmap for Dog
We compare the test accuracies achieved via the
network in this article with the state-of-the-art (SOTA)
results obtained by a recursive convolutional neural
network in Table 2. Because the network here might
not be an optimized one to target the mind workload
classification, and GAP, which collapses the whole
feature map into one point is still too coarse, the
obtained test accuracy in this article is not as good as
the SOTA results. However, because the purpose of
this article is to demonstrate that the attention mechanism
is an intrinsic property of a rather-general neural
network, in this regard, a fairly comparable result is
also acceptable.
For heatmap generation, to minimize the incorrect
predictions that can interfere with the following investigation,
only samples those with correct predictions
are considered. The aforementioned procedure is
repeated for all the subjects, with correctly predicted
samples and the corresponding heatmaps totaling 2,174
for analysis.
To investigate the properties of these heatmaps, for
example, whether heatmaps corresponding to samples
under a different mind workload are separated or not,
t-distributed stochastic neighbor embedding (t-SNE) is utilized
to observe the distribution of these heatmaps in lower-dimensional
spaces, such as in 2D [28]. Because the
heatmaps are of high-dimensional data, potentially spreading
along certain manifolds instead of using Euclidean distance,
the structural similarity index (SSIM) is considered
here [29]. As inferred from [29], the SSIM measures the perceptual
difference between two similar images and cannot
judge which of the two is better. This is permissible with
this research, because whether a specific heatmap is good
or not is not confirmative, and only the collective heatmaps
can propose certain conclusions. Therefore, as in
[28], for heatmaps hi
and h ,j
calculated as in
p =j ; i
exp^b^1- ssim^hi
|exp^b^1- ssim^
jk
!
The perplexity of t-SNE is set at 20 to map the heatFigure
4. The implicit attention capability of the
designed network verified by the dog-versus-cat image
set. (Source: https://www.kaggle.com/biaiscience/
dogs-vs-cats)
maps from a high-dimensional space of 4,096 dimensions
onto a 2D plane. The algorithm is iterated for 1,000 times
to obtain the distribution of heatmaps in low dimension
for examination.
Table 2. Accuracies of the memory workload classification for each subject.
Test Subject
S1
RCNN*
GAP
88.9
57.3
S2
76.5
64.6
S3
93.3
84.9
S4
99
87.6
S5
100
96.4
S6
98
94
S7
100
96.4
*The statistics are taken directly from [2]. RCNN: recursive convolutional neural network.
S8
98.5
97.5
S9
99
91.9
S10
96.8
88.4
S11
96.5
82.9
hh
h
,
i
,
jhh2
k
hh
h
2h
.
(5)
the conditional probability is
S12
91
67
S13
46.8
47.3
Mean
91.1
81.3
28 IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE October 2021
https://www.kaggle.com/biaiscience/dogs-vs-cats https://www.kaggle.com/biaiscience/dogs-vs-cats

IEEE Systems, Man and Cybernetics Magazine - October 2021

Table of Contents for the Digital Edition of IEEE Systems, Man and Cybernetics Magazine - October 2021

Contents
IEEE Systems, Man and Cybernetics Magazine - October 2021 - Cover1
IEEE Systems, Man and Cybernetics Magazine - October 2021 - Cover2
IEEE Systems, Man and Cybernetics Magazine - October 2021 - Contents
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 2
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 3
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 4
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 5
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 6
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 7
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 8
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 9
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 10
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 11
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 12
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 13
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 14
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 15
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 16
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 17
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 18
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 19
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 20
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 21
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 22
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 23
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 24
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 25
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 26
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 27
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 28
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 29
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 30
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 31
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 32
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 33
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 34
IEEE Systems, Man and Cybernetics Magazine - October 2021 - Cover3
IEEE Systems, Man and Cybernetics Magazine - October 2021 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/smc_202110
https://www.nxtbook.com/nxtbooks/ieee/smc_202107
https://www.nxtbook.com/nxtbooks/ieee/smc_202104
https://www.nxtbook.com/nxtbooks/ieee/smc_202101
https://www.nxtbook.com/nxtbooks/ieee/smc_202010
https://www.nxtbook.com/nxtbooks/ieee/smc_202007
https://www.nxtbook.com/nxtbooks/ieee/smc_202004
https://www.nxtbook.com/nxtbooks/ieee/smc_202001
https://www.nxtbook.com/nxtbooks/ieee/smc_201910
https://www.nxtbook.com/nxtbooks/ieee/smc_201907
https://www.nxtbook.com/nxtbooks/ieee/smc_201904
https://www.nxtbook.com/nxtbooks/ieee/smc_201901
https://www.nxtbook.com/nxtbooks/ieee/smc_201810
https://www.nxtbook.com/nxtbooks/ieee/smc_201807
https://www.nxtbook.com/nxtbooks/ieee/smc_201804
https://www.nxtbook.com/nxtbooks/ieee/smc_201801
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1017
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0717
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0417
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0117
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1016
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0716
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0416
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0116
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1015
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0715
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0415
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0115
https://www.nxtbookmedia.com