IEEE Systems, Man and Cybernetics Magazine - January 2021 - 21

Feedback

Action

The playing times of
the games were not
uniform.

The SD of time decreased
from 331 s (M = 388) in the first
experiment to 62 s (M = 477) in the
second experiment.

The scores of the games
were not uniform.

The SD of scores decreased from
393 (M = 525) in the first experiment
to 347 (M = 626) in the second
experiment.

The Thanksgiving game
(AWSR task) was difficult
to play.

- Multiple immediate visual and
auditory feedbacks were added
to the interactions.
- The speed of the game was
decreased.
- A replay function was added for
listening to the pronunciation of
the words again.

The original task in the
Easter game was not
effective.

This task was completely removed
and replaced by a task based on
the Reading Accelerated Program
[25].

Some of the reading
materials were similar to
the prior sessions.

It was intentionally designed in this
way as blocked practice at the first
stage of automaticity acquisition,
particularly for young children, leads
to better results [45].

The game (AWMR task)
was difficult for English
as a second language
children.

This problem was not reported when
testing with native English speakers
in the second experiment.

Speech recognition in
the Christmas game
(AWD task) was not
precise.

The speech recognition system was
improved by limiting the recognition
possibilities to only active word in
the game scene.

	

12
Presentation Rate

Table 2. The collected feedback from
the first study.

with minor tweaks, the data of the three games were combined to obtain more statistically significant data.
It was expected that the system would start by selecting the words or word lists having the highest frequency
level. However, because the numbers of words in each frequency level were not equal, the number of presentations
should be moderated with this disparity to reach meaningful data. For this objective, the number of presentations for
each frequency level was divided by the number of words
(or word lists) existing at that level. The results are displayed in Figure 2, which looks obvious since users often
played the game only once or twice.
Additionally, it was expected that the model would
more often select the words having the highest frequency
levels and less often the words with the next neighboring
frequency levels. Similarly, the model prioritized the content with lower difficulty levels, which was marked by
word length. Figure 3 illustrates this tendency. However,
the number of presentations for each word length level had
to be moderated by the number of words in that level. This
is important because, for example, the only single-letter
words in the English Language are " a " and " I. " Therefore,
the number of presentations in each word length level was
divided by the number of words (or word lists) at that level.
Word length is not the only attribute that can be used as
an indicator of word difficulty. For the investigation of this

10
8
6
4
2
0

1

6

11

16

21 26 31 36
Word Frequency

41

46

51

Figure 2. The presentation rate of the word frequency

levels.

10
Presentation Rate

the most important feedback received from the first
usability study.
The second usability study was carried out with the
second iteration of the games that were improved based on
the first feedbacks. Both subjective and objective data
were collected, either automatically from the system and
in the cloud or directly by the experimenter. The first interesting analysis was made on the behavior of the system
and its optimization model to determine if they produce
the expected content. The number of the presentation for
each piece of reading material (here called " presentation " )
and each wrong answer of the users (here called " error " )
were recorded for each item of reading material.
The data of the presented words (the number of presentations) was analyzed relative to the frequency of the words
in the language. As explained previously, frequency was
chosen as the value parameter of the words in the optimization model. These frequencies were classified into 53 levels.
Since the games were using the same optimization model

8
6
4
2
0

1

2

3
4
5
Number of Letters

6

7

Figure 3. The presentation rate of the word length

levels.

Ja nu a r y 2021

IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE	

21



IEEE Systems, Man and Cybernetics Magazine - January 2021

Table of Contents for the Digital Edition of IEEE Systems, Man and Cybernetics Magazine - January 2021

CONTENTS
IEEE Systems, Man and Cybernetics Magazine - January 2021 - Cover1
IEEE Systems, Man and Cybernetics Magazine - January 2021 - Cover2
IEEE Systems, Man and Cybernetics Magazine - January 2021 - CONTENTS
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 2
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 3
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 4
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 5
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 6
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 7
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 8
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 9
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 10
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 11
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 12
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 13
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 14
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 15
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 16
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 17
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 18
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 19
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 20
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 21
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 22
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 23
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 24
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 25
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 26
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 27
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 28
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 29
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 30
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 31
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 32
IEEE Systems, Man and Cybernetics Magazine - January 2021 - 33
https://www.nxtbook.com/nxtbooks/ieee/smc_202110
https://www.nxtbook.com/nxtbooks/ieee/smc_202107
https://www.nxtbook.com/nxtbooks/ieee/smc_202104
https://www.nxtbook.com/nxtbooks/ieee/smc_202101
https://www.nxtbook.com/nxtbooks/ieee/smc_202010
https://www.nxtbook.com/nxtbooks/ieee/smc_202007
https://www.nxtbook.com/nxtbooks/ieee/smc_202004
https://www.nxtbook.com/nxtbooks/ieee/smc_202001
https://www.nxtbook.com/nxtbooks/ieee/smc_201910
https://www.nxtbook.com/nxtbooks/ieee/smc_201907
https://www.nxtbook.com/nxtbooks/ieee/smc_201904
https://www.nxtbook.com/nxtbooks/ieee/smc_201901
https://www.nxtbook.com/nxtbooks/ieee/smc_201810
https://www.nxtbook.com/nxtbooks/ieee/smc_201807
https://www.nxtbook.com/nxtbooks/ieee/smc_201804
https://www.nxtbook.com/nxtbooks/ieee/smc_201801
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1017
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0717
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0417
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0117
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1016
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0716
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0416
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0116
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1015
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0715
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0415
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0115
https://www.nxtbookmedia.com