IEEE Systems, Man and Cybernetics Magazine - July 2020 - 32

of our knowledge, such methods have never been tested
supplementary material for this article which can be found
and compared on workload, emotions data sets, or with
in IEEE Xplore.
subject-independent calibration. Thus, we propose this
As introduced previously, we studied both subject-speevaluation in this article.
cific and subject-independent calibrations. For the latter,
We also suggest some new variants of these algothe first half of each user's trials was used for training and
rithms. Altogether, we studied seven algorithms. First,
the second half for testing. For the subject-independent calCSP and LDA were used as a baseline, since they are wideibration, the training set comprised all trials of all users
ly used by the BCI community [14].
except the current user, i.e., around
We then explored the FFBCSP and
21 × 1,440 = 30,240 training trials.
LDA [13], a CNN [33], and four difTo allow comparisons between calPassive BCIs are
ferent methods based on Riemanibration types, the testing set of
nian geometry: two existing ones,
each user was the same with subnot used directly for
namely the Fisher geodesic miniject-specific calibration, i.e., the
control, but to monitor
mum distance to the mean classifier
second half of the trials (720 test(FgMDM) and the TSC [32] and two
ing trials) from this user.
users' mental states
new extensions we propose here to
in
real
time
and
better exploit the spectral informaEmotion EEG Data Set
adapt
an
application
tion, namely the filter bank FgMDM
The data set used for studying
and the filter bank TSC. For the
emotions was the Database for
accordingly.
workload data set, we assess perforEmotion Analysis using Physiologimances using classification accuracal Signals (DEAP) [28]. It used
cy, i.e., the percentage of test trials
music-video clips to influence two
classified correctly. For the emotion data set, we used baltypes of emotion dimensions, valence and arousal, accordanced accuracy, i.e., the average of recall obtained on each
ing to Russell's circumplex model [25]. The data set conclass, since the classes were unbalanced.
tains 40 trials, corresponding to 40 music-video clips,
recorded on 32 participants. EEGs were recorded using
32 electrodes (placed according to the international 10-20
CSPs
system). Valence and arousal levels were measured using
CSP is a widely used algorithm for binary EEG classificaRussell's valence-arousal scale directly after each video by
tion for oscillatory activity-based BCI. Changes in both
clicking on a 1-9 continuous scale. This self-assessment
workload [16] and emotions [5] have been shown to induce
system on a continuous scale makes the classes definition
changes in EEG oscillatory activity. The CSP algorithm
more complex: in DEAP [28] as well as in our study, 5 was
optimizes spatial filters, i.e., a linear combination of the
kept as a threshold to split trials into two classes, low and
original EEG signals, so that the variance of a spatially
high, for both emotion-arousal and emotion-valence data
filtered signal, i.e., the band power of this signal, is maxisets, making classes unbalanced. All classifiers were able
mized for one class and minimized for the other. Formalto deal with unbalanced classes except the CNN, for which
ly, CSP optimizes spatial filter w by either maximizing
we upsampled the minority class by randomly duplicating
or minimizing
trials from this class to obtain balanced classes.
wX 1 X T1 w T wC 1 w T
For the subject-specific study, given the low number of
	
J CSP ^w h =
=
, (1)
wX 2 X T2 w T wC 2 w T
trials, we performed a leave-one-out cross-validation.
Thus, we used 40 models for each subject, with each model
where T denotes transpose, X i is the bandpass filtered
trained on 39 trials and tested on one trial. For the subjecttraining signal matrix for class i (with the samples as colindependent study, we kept all trials of all subjects to comumns and the channels as rows), and C i is the spatial
pose the training set, except the current subject used for
covariance matrix from class i.
testing (i.e., 31 × 40 = 1,240 trials for the training set). The
In practice, the covariance matrix C i is defined as the
testing set of each subject was composed of all trials of
average covariance matrix of each trial from class i [14].
this subject, i.e., 40 trials.
The spatial filters w that maximize or minimize J CSP ^w h
are the eigenvectors corresponding to the largest and lowest eigenvalues, respectively, of the generalized eigenvalue
Machine-Learning Algorithms Explored
decomposition of matrices C 1 and C 2 . In this study, we
The existing algorithms we evaluate here were all studied
on EEG-based motor imagery classification, a widely used
used six filters, corresponding to the three largest and three
BCI design, and obtained impressive results. Since motor
lowest eigenvalues, as recommended in [14]. Once these filimagery, workload, and emotions all lead to change in
ters are obtained, we use f = log (wXX T w T ) as CSP feaEEG oscillatory activity, it is likely that methods that
tures, i.e., the band power of the spatially filtered signals.
proved effective for motor imagery are effective for workWe used these features as input to an LDA classifier. The
load or emotion classification as well. However, to the best
CSP requires EEG signals to be bandpass filtered in a
32	

IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Ju ly 2020



IEEE Systems, Man and Cybernetics Magazine - July 2020

Table of Contents for the Digital Edition of IEEE Systems, Man and Cybernetics Magazine - July 2020

Contents
IEEE Systems, Man and Cybernetics Magazine - July 2020 - Cover1
IEEE Systems, Man and Cybernetics Magazine - July 2020 - Cover2
IEEE Systems, Man and Cybernetics Magazine - July 2020 - Contents
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 2
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 3
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 4
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 5
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 6
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 7
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 8
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 9
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 10
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 11
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 12
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 13
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 14
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 15
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 16
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 17
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 18
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 19
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 20
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 21
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 22
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 23
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 24
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 25
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 26
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 27
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 28
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 29
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 30
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 31
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 32
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 33
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 34
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 35
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 36
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 37
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 38
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 39
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 40
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 41
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 42
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 43
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 44
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 45
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 46
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 47
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 48
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 49
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 50
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 51
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 52
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 53
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 54
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 55
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 56
IEEE Systems, Man and Cybernetics Magazine - July 2020 - Cover3
IEEE Systems, Man and Cybernetics Magazine - July 2020 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/smc_202110
https://www.nxtbook.com/nxtbooks/ieee/smc_202107
https://www.nxtbook.com/nxtbooks/ieee/smc_202104
https://www.nxtbook.com/nxtbooks/ieee/smc_202101
https://www.nxtbook.com/nxtbooks/ieee/smc_202010
https://www.nxtbook.com/nxtbooks/ieee/smc_202007
https://www.nxtbook.com/nxtbooks/ieee/smc_202004
https://www.nxtbook.com/nxtbooks/ieee/smc_202001
https://www.nxtbook.com/nxtbooks/ieee/smc_201910
https://www.nxtbook.com/nxtbooks/ieee/smc_201907
https://www.nxtbook.com/nxtbooks/ieee/smc_201904
https://www.nxtbook.com/nxtbooks/ieee/smc_201901
https://www.nxtbook.com/nxtbooks/ieee/smc_201810
https://www.nxtbook.com/nxtbooks/ieee/smc_201807
https://www.nxtbook.com/nxtbooks/ieee/smc_201804
https://www.nxtbook.com/nxtbooks/ieee/smc_201801
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1017
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0717
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0417
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0117
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1016
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0716
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0416
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0116
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1015
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0715
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0415
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0115
https://www.nxtbookmedia.com