IEEE Systems, Man and Cybernetics Magazine - July 2020 - 31

workload, arousal, and valence classification from EEG
obtained 77% and 74% classification accuracy, respectively,
signals. Preliminary results on mental workload data only,
by using a combination of wavelet entropy and average
and with a few existing algorithms, were published as an
wavelet coefficient coupled with an SVM.
extended conference abstract in [35].
Al-Nafjan et al. [27] used SVMs, deep neural networks,
We also propose guidelines about which algorithm to use
or random forest to classify data from the DEAP data set
in which context. As a baseline, we use two standard meth[28] (presented in the "Methods" section of this article)
ods to study workload levels/affective states classification:
using a two-class problem with 60-s-long epochs. They
common spatial pattern spatial filters with an LDA classifier
obtained 79%, 49%, and 56% classification accuracy,
and the FBCSP [13], which is a CSP extension that won
respectively. However, the deep learning method was not
numerous active BCI competitions.
described in this article, which
Then, we studied two Riemannian
makes it difficult to assess the
approaches, manipulating and clasvalidity and superiority of this
BCI applications
sifying EEG signals as covariance
approach. The logistic regression
matrices: minimum distance to the
(LR) method to discr iminate
enable their users
mean with Fisher geodesic filtering
valence levels (low versus high)
to interact with
classifier (FgMDM) and TSC. Such
was used [29] and it obtained 71%
methods have recently won six interaccuracy on 6-s ranges.
computers by using
national brain-signal competitions
In [28], the authors introduce the
brain activity only,
[32]. We then propose to improve
DEAP data set we use here; they
usually measured
these Riemannian approaches by
applied a naive Bayes classifier for
working on a bank of bandpass filters
two-class discrimination (low verwith EEGs.
such as the ones used for FBCSP,
sus high) for both valence and
instead of using a unique bandpass
arousal dimensions (60-s epochs).
filter. We name these new approaches
They obtained 57% and 62% of accufilter back FgMDM (FBFgMDM) and FBTSC. Finally, we used a
racy for the valence and arousal dimensions, respectively.
CNN, i.e., a deep learning algorithm, which recently obtained
Most studies proved that classifying affective states from
promising results for many machine-learning problems [34].
EEG remains very challenging since results hardly go over
We studied the CNN developed in [33], since it obtained promischance-level accuracy. Some studies were even unable to
obtain better than chance results when reproducing previing results for motor imagery-based BCIs.
ous works with statistically rigorous evaluation methods
In this article, we first present the workload and emo[30]. Finally, confounding factors due to electromyography
tion EEG data sets used, before describing each machine(e.g., facial muscles activity during emotion expression) have
learning algorithm. We perform two evaluation studies.
likely played a role in the performances obtained in many
The first is a subject-specific study with each algorithm
studies. However, other studies have obtained better accuratrained on data specific to each subject and then tested on
cies when using different EEG patterns. For instance, Frantother data from the same subject. This is the standard way
zidis et al. [31] obtained 81% classification accuracy using
current BCIs are designed, given the large between-subject
evoked potentials for a four-class valence/arousal classificavariability [14]. The second is a subject-independent study,
tion. Note that this article focuses on oscillatory activity, as
with each algorithm trained on all data recorded from all
this can classify cognitive and affective states from continusubjects except that of the target subject, on which algoous EEG without the need for a stimulus-locked response,
rithms are tested. This is much more challenging, but if it
which evoked potentials require.
is successful, it would enable BCI-based monitoring without requiring any calibration for new subjects.
Article Objectives
Methods
The classification accuracies obtained so far, mostly
around 70% for workload and approximately 60%-65% for
emotions in oscillatory-based studies, revealed the need
Mental Workload EEG Data Set
for more robust and accurate EEG classification algoThe data set used comes from [4]. Signals from 28 EEG
rithms. Therefore, we propose studying algorithms that
electrodes (active electrodes in a 10/20 system without T7,
proved efficient either in recent active BCI classification
T8, Fp1, and Fp2) were recorded from 22 users [4]. To
competitions [13], [32], notably RGCs, or in other fields of
induce mental workload variations, N-back tasks were
artificial intelligence such as deep learning [33], [34]. Note
used: the user had to indicate whether a letter displayed on
that such algorithms have been explored mostly for EEG
screen was the same as the one displayed N letters before,
classification of motor tasks but were not systematically
in a stream of successively displayed letters. Here, 2-s trials
studied and compared for workload/affective states estifrom a 0-back task were labeled low workload while those
mation. Here, we formally study and compare these various
from a 2-back were labeled high workload. In total, 720 trialgorithms as well as two new variants we proposed, for
als were available for each workload level and user. See the
	

Ju ly 2020

IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE	

31



IEEE Systems, Man and Cybernetics Magazine - July 2020

Table of Contents for the Digital Edition of IEEE Systems, Man and Cybernetics Magazine - July 2020

Contents
IEEE Systems, Man and Cybernetics Magazine - July 2020 - Cover1
IEEE Systems, Man and Cybernetics Magazine - July 2020 - Cover2
IEEE Systems, Man and Cybernetics Magazine - July 2020 - Contents
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 2
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 3
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 4
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 5
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 6
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 7
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 8
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 9
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 10
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 11
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 12
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 13
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 14
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 15
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 16
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 17
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 18
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 19
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 20
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 21
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 22
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 23
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 24
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 25
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 26
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 27
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 28
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 29
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 30
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 31
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 32
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 33
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 34
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 35
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 36
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 37
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 38
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 39
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 40
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 41
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 42
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 43
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 44
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 45
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 46
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 47
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 48
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 49
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 50
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 51
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 52
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 53
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 54
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 55
IEEE Systems, Man and Cybernetics Magazine - July 2020 - 56
IEEE Systems, Man and Cybernetics Magazine - July 2020 - Cover3
IEEE Systems, Man and Cybernetics Magazine - July 2020 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/smc_202110
https://www.nxtbook.com/nxtbooks/ieee/smc_202107
https://www.nxtbook.com/nxtbooks/ieee/smc_202104
https://www.nxtbook.com/nxtbooks/ieee/smc_202101
https://www.nxtbook.com/nxtbooks/ieee/smc_202010
https://www.nxtbook.com/nxtbooks/ieee/smc_202007
https://www.nxtbook.com/nxtbooks/ieee/smc_202004
https://www.nxtbook.com/nxtbooks/ieee/smc_202001
https://www.nxtbook.com/nxtbooks/ieee/smc_201910
https://www.nxtbook.com/nxtbooks/ieee/smc_201907
https://www.nxtbook.com/nxtbooks/ieee/smc_201904
https://www.nxtbook.com/nxtbooks/ieee/smc_201901
https://www.nxtbook.com/nxtbooks/ieee/smc_201810
https://www.nxtbook.com/nxtbooks/ieee/smc_201807
https://www.nxtbook.com/nxtbooks/ieee/smc_201804
https://www.nxtbook.com/nxtbooks/ieee/smc_201801
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1017
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0717
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0417
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0117
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1016
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0716
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0416
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0116
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1015
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0715
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0415
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0115
https://www.nxtbookmedia.com