IEEE Systems, Man and Cybernetics Magazine - October 2021 - 27

Figure 3. First, a convolutional network
(convnet) is applied to the
nonoverlapping segmented topographical
data at each time step.
Note that the weights of the convnet
are shared when processing
each segment. Then the processed
data are fed into a recurrent network
for further computation. To
better explore the spatial information
of the topographies, convolution,
instead of a conventional
linear transformation plus nonlinear
activation, is used inside the
recurrent cell. Hence, the subnetwork,
that is, the recurrent
convnet, is known as the recurconvnet.
In detail, assuming at
time step t, the input to the recur-convnet is
the hidden state is h ,t
state is c ,t
The complexity of
the cognitive and
data-acquisition
processes brings
nonintuition into
the understanding
of EEG data, which
consequently hinders
interpretation of the
network outcome.
x ,t
input gate, forget gate, and output gate are i ,t
the cell
and the values of the
f ,t and o ,t
respectively. The computation is governed by the following
formulas:
| = concat ih ,^h
tt 1t
-
,, @= ^hconv | ,
tt 1tt
vv |ti
hh ,
))tan conv
,
6ifowtt tt g
=+(1)
(2)
cc
i^^ ^^hh hhfw (3)
ho tan cttt =v )^^
(4)
Table 1. The network architecture.
Block
Layer
ConvNet
Conv 2D
Conv 2D
Conv 2D
Conv 2D
Average pool
Conv 2D
Conv 2D
Average pool
Conv 2D
Average pool
Recur- ConvNet*
GAP
FC
Conv 2D
Average pool
Linear
*The number of recurrences is seven. ReLU: rectified linear unit.
where w represents the weights of
different network circuits in (1)-(4).
After recurrence, only the feature
maps of the last step are considered.
GAP is then applied to these feature
maps to obtain the resulting weights
before the final classification. These
feature maps and weights are used
to construct the heatmaps, as in [17].
The detai led configuration of
the network structure is provided
in Table 1.
Heatmap Generation
and Investigation
The complexity of the cognitive
and data-acquisition processes
br ings nonintuit ion into the
understanding of EEG data, which consequently hinders
interpretation of the network outcome. To explicitly
demonstrate the implicit attention capability of the network,
a natural dog-versus-cat image set [27]-compiled
for categorizing dogs and cats-is used to verify the
ROIs of the network when performing classification in
the first place. Because the designed network has a
recurrent part, for each dog or cat image, it repeats
seven times to form a sequential sample to resemble the
EEG topographical data. This means that the layouts of
the respective input data are identical between the two
Filters
Eight
Eight
Eight
Eight
Size
-
16
16
-
32
-
128
-
-
(3, 3)
(3, 3)
(3, 3)
(3, 3)
(2, 2)
(3, 3)
(3, 3)
(2, 2)
(3, 3)
(2, 2)
(3, 3)
(8, 8)
Number of classes
Activation
ReLU
ReLU
ReLU
ReLU
-
ReLU
ReLU
-
ReLU
-
-
-
Softmax
Padding
Same
Same
Same
Same
-
Same
Same
-
Same
-
Same
Valid
-
October 2021 IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE 27

IEEE Systems, Man and Cybernetics Magazine - October 2021

Table of Contents for the Digital Edition of IEEE Systems, Man and Cybernetics Magazine - October 2021

Contents
IEEE Systems, Man and Cybernetics Magazine - October 2021 - Cover1
IEEE Systems, Man and Cybernetics Magazine - October 2021 - Cover2
IEEE Systems, Man and Cybernetics Magazine - October 2021 - Contents
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 2
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 3
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 4
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 5
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 6
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 7
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 8
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 9
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 10
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 11
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 12
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 13
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 14
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 15
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 16
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 17
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 18
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 19
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 20
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 21
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 22
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 23
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 24
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 25
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 26
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 27
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 28
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 29
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 30
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 31
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 32
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 33
IEEE Systems, Man and Cybernetics Magazine - October 2021 - 34
IEEE Systems, Man and Cybernetics Magazine - October 2021 - Cover3
IEEE Systems, Man and Cybernetics Magazine - October 2021 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/smc_202110
https://www.nxtbook.com/nxtbooks/ieee/smc_202107
https://www.nxtbook.com/nxtbooks/ieee/smc_202104
https://www.nxtbook.com/nxtbooks/ieee/smc_202101
https://www.nxtbook.com/nxtbooks/ieee/smc_202010
https://www.nxtbook.com/nxtbooks/ieee/smc_202007
https://www.nxtbook.com/nxtbooks/ieee/smc_202004
https://www.nxtbook.com/nxtbooks/ieee/smc_202001
https://www.nxtbook.com/nxtbooks/ieee/smc_201910
https://www.nxtbook.com/nxtbooks/ieee/smc_201907
https://www.nxtbook.com/nxtbooks/ieee/smc_201904
https://www.nxtbook.com/nxtbooks/ieee/smc_201901
https://www.nxtbook.com/nxtbooks/ieee/smc_201810
https://www.nxtbook.com/nxtbooks/ieee/smc_201807
https://www.nxtbook.com/nxtbooks/ieee/smc_201804
https://www.nxtbook.com/nxtbooks/ieee/smc_201801
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1017
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0717
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0417
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0117
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1016
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0716
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0416
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0116
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1015
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0715
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0415
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0115
https://www.nxtbookmedia.com