IEEE Systems, Man and Cybernetics Magazine - July 2020 - 27

and software and providing all of the necessary scripts to
run different tests, along with the details of how to put
together the system, the work herein presented results useful for researchers across many different fields Similar to all
other mobile EEG applications, signal processing steps are
still needed to compensate for power-line interference and
muscle and movement artifacts. The latter can be particularly troublesome in ambulant settings, but open-source algorithms [55] are available to enable accurate EEG analysis
under such conditions [45].

[4] M. Alaker, G. R. Wynn, and T. Arulampalam, "Virtual reality training in laparoscopic
surgery: A systematic review & meta-analysis," Int. J. Surg., vol. 29, pp. 85-94, May
2016. doi: 10.1016/j.ijsu.2016.03.034.
[5] L. Freina and M. Ott, "A literature review on immersive virtual reality in education:
State of the art and perspectives," in Proc. Conf.: eLearning and Software for Education (eLSE), Bucharest, Romania, 2015, pp. 133-141.
[6] N. Gavish et al., "Evaluating virtual reality and augmented reality training for
industrial maintenance and assembly tasks," Interact. Learn. Environ., vol. 23, no. 6,
pp. 778-798, Nov. 2015. doi: 10.1080/10494820.2013.815221.
[7] M. Maskey et al., "A randomised controlled feasibility trial of immersive virtual reality treatment with cognitive behaviour therapy for specific phobias in young people with

Conclusion
In this article, we described the development and evaluation of a wireless, integrated BMI-HMD that allows for the
acquisition of multimodal physiological signals, such as
EEG, ECG, EOG, and facial EMG during mobile VR/AR
applications. The developed system allows real-time feedback, measurement of relevant experience factors (e.g.,
sense of presence, cybersickness, and attention, among
others), and the creation of objective outcome measures
for different VR-based treatments. Future steps and
numerous potential applications for the developed BMI-
HMD were described and discussed.

autism spectrum disorder," J. Autism Dev. Disord., vol. 49, no. 5, pp. 1912-1927, 2019. doi:
10.1007/s10803-018-3861-x.
[8] S. Weech, S. Kenny, and M. Barnett-Cowan, "Presence and cybersickness in
virtual reality are negatively related: A review," Front. Psychol. vol. 10, p. 158,
Feb. 2019. doi: 10.3389/fpsyg.2019.00158.
[9] J. G. Apostolopoulos, P. A. Chou, B. Culbertson, T. Kalker, M. D. Trott, and S. Wee,
"The road to immersive communication," Proc. IEEE, vol. 100, no. 4, pp. 974-990, 2012.
doi: 10.1109/JPROC.2011.2182069.
[10] K. Brunnström et al., "Qualinet white paper on definitions of quality of experience," Qualinet, Lausanne, Switzerland, White Paper, 2013.
[11] H. K. Kim, J. Park, Y. Choi, and M. Choe, "Virtual reality sickness questionnaire
(VRSQ): Motion sickness measurement index in a virtual reality environment," Appl.
Ergon., vol. 69, pp. 66-73, May 2018. doi: 10.1016/j.apergo.2017.12.016.

Acknowledgment
We wish to thank Andreea Iva
˘nescu for her earlier work on
EEG-based cybersickness detection, which served as
motivation for the development of the BMI-HMD.

[12] B. G. Witmer and M. J. Singer, "Measuring presence in virtual environments: A presence questionnaire," Presence, vol. 7, no. 3, pp. 225-240, 1998. doi:
10.1162/105474698565686.
[13] R. Cassani, M.-A. Moinnereau, and T. H. Falk, "A neurophysiological sensorequipped head-mounted display for instrumental QoE assessment of immersive multi-

About the Authors
Raymundo Cassani (raymundo.cassani@emt.inrs.ca) is
with the Institut National de la Recherche Scientifique, University of Quebec, Montreal, Canada.
Marc-Antoine Moinnereau (marc-antoine.moinnereau
@emt.inrs.ca) is with the Institut National de la Recherche
Scientifique, University of Quebec, Montreal, Canada.
Liviu Iva
˘ nescu (liviu@emt.inrs.ca) is with the Institut
National de la Recherche Scientifique, University of Quebec,
Montreal, Canada.
Olivier Rosanne (olivier.rosanne@emt.inrs.ca) is with
the Institut National de la Recherche Scientifique, University of Quebec, Montreal, Canada.
Tiago H. Falk (falk@emt.inrs.ca) is with the Institut
National de la Recherche Scientifique, University of Quebec,
Montreal, Canada.

media," in Proc. 10th Int. Conf. Quality of Multimedia Experience (QoMEX), pp. 1-6,
May 2018. doi: 10.1109/QoMEX.2018.8463422.
[14] F. Lotte et al., "Combining BCI with virtual reality: Towards new applications and
improved BCI," in Towards Practical Brain-Computer Interfaces (Biological and
Medical Physics, Biomedical Engineering). Berlin, Heidelberg: Springer-Verlag, 2012,
pp. 197-220.
[15] U. Engelke et al., "Psychophysiology-based QoE assessment: A survey," IEEE
J. Sel. Topics Signal Process., vol. 11, no. 1, pp. 6-21, Feb. 2017. doi: 10.1109/
JSTSP.2016.2609843.
[16] T. O. Zander and C. Kothe, "Towards passive brain-computer interfaces: Applying
brain-computer interface technology to human-machine systems in general," J. Neural Eng., vol. 8, no. 2, p. 025005, 2011. doi: 10.1088/1741-2560/8/2/025005.
[17] A. Martínez-Rodrigo, B. García-Martínez, R. Alcaraz, P. González, and A. Fernández-Caballero, "Multiscale entropy analysis for recognition of visually elicited negative
stress from eeg recordings," Int. J. Neural Syst., vol. 29, no. 2, p. 1,850,038, 2018. doi:
10.1142/S0129065718500387
[18] F. Dehais et al., "Monitoring pilot's cognitive fatigue with engagement features

References

in simulated and actual flight conditions using an hybrid fNIRS-EEG passive BCI," in

[1] C. Ashley, J. Stoeber, B. D. Gilbert, and T. Long, "Why now is the best time to invest

Proc. 2018 IEEE Int. Conf. Systems, Man, and Cybernetics (SMC), pp. 544-549. doi:

in VR," Polygon, Feb. 13, 2019. Accessed on: July 18, 2019. [Online]. Available: https://

10.1109/SMC.2018.00102.

www.youtube.com/watch?v=-2ApDD37YkE

[19] A. Clerico, A. Tiwari, R. Gupta, S. Jayaraman, and T. H. Falk, "Electroencepha-

[2] F. D. Rose, B. M. Brooks, and A. A. Rizzo, "Virtual reality in brain damage reha-

lography amplitude modulation analysis for automated affective tagging of music

bilitation: review," CyberPsychol. Behav., vol. 8, no. 3, pp. 241-262, June 2005. doi:

video clips," Front. Comput. Neurosci., vol. 11, p. 115, 2018. doi: 10.3389/fncom.2017

10.1089/cpb.2005.8.241.

.00115.

[3] J. N. Silva, M. Southworth, C. Raptis, and J. Silva, "Emerging applications of virtual

[20] M. Meehan, B. Insko, M. Whitton, and F. P. Brooks Jr., "Physiological measures

reality in cardiovascular medicine," JACC: Basic Transl. Sci., vol. 3, no. 3, pp. 420-430,

of presence in stressful virtual environments," ACM Trans. Graph., vol. 21, no. 3,

June 2018. doi: 10.1016/j.jacbts.2017.11.009.

pp. 645-652, July 2002. doi: 10.1145/566654.566630.

	

Ju ly 2020

IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE	

27
https://www.youtube.com/watch?v=-2ApDD37YkE https://www.youtube.com/watch?v=-2ApDD37YkE
IEEE Systems, Man and Cybernetics Magazine - July 2020

Table of Contents for the Digital Edition of IEEE Systems, Man and Cybernetics Magazine - July 2020
