IEEE Systems, Man and Cybernetics Magazine - January 2021 - 20

API [42]). Since the frequencies were based on the occurrence count of the words in bodies of texts, the numbers
could be greatly different. For controlling this, the frequencies were classified into 53 categories, and numbers ranging
from 1 to 53 were used for marking the value of each word.
As stated previously, these values were only the starting
point, and after each session, they will be updated. If a word
has already been shown to the user who provided a correct
answer, its value will be decreased by a certain amount so
that the probability of presenting the same word again will
be decreased. Similarly, if a word has been presented and the
user could not provide a correct answer, its value will be
increased so that the probability of presenting the same
word again will be increased. The amount of these adjustments can also depend on the response time of the user.
The difficulty level of each learning material is presented by the weight parameter in the optimization model. Various attributes, such as word length, frequency, or the
consistency of the phoneme-grapheme mapping, can be
used for the difficulty level of a word. Here, it was decided
to choose the word length (number of letters) for the
weight of each word. Then, each time, depending on the
performance of the user in the previous training session,
the maximum and the minimum weight limits (UW and
LW) will be adjusted for the next training session. If the
session was too difficult for the user, the UW and LW can
be lowered, and, conversely, if the session was too easy,
those parameters can be increased to make the next session more difficult. Even the length of sessions can be
increased and decreased depending on the level of the user
and by modifying the LN and UN values.
In addition to the optimization algorithm solved by the
system each time to determine the training content, various adaptive components were created into the games.
Notably, the speed of the games and the rewards were
adapted based on the answers of the users and their
response time, respectively. The optimization model that
was integrated into the games can be p
- resented by:

Table 1. The variables and parameters of
the optimization model.
xi

Item i with a value of zero or one

The value of the item i

The weight of the item i

The number of presentations of the item i

The minimum allowed weight

The maximum allowed weight

The lower limit on the number of selected item

The upper limit on the number of selected items

The upper limit on the presentation count for each item

IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Janu ar y 2021

The objective of maximizing the value of each training
session includes:

Maximize |v i x i, (1)
i=1

which is subject to the constraint for the difficulty level of
each training session:

LW 1 |w i x i # UW, (2)
i=1

the constraint for the number of items in each training
session:

LN # |x i # UN, (3)
i=1

and the constraint for the number of presentations for
each piece of reading material:

6 i p i x i # P, (4)

where

x i e {0, 1} . (5)

Table 1 presents the variables and parameters of the
optimization model, where x i is the variable and the other
elements are the parameters of the model.
Experimental Procedure
Two usability studies were carried out to test the functionality and usability of the games. The two studies were separated by a period of four months to allow for conducting
an iterative design of the system. For the first study, 15
participants were selected from a group of English as second language learners who volunteered to take part in the
functionality testing study. Sixteen other participants who
were native English speakers were involved in the second
study. The two samples comprise 18 males (58%), with
ages ranging between 7 and 50 [mean (M) = 19.4, standard
deviation (SD) = 8.9]. Four of them were left-handed (13%),
and eight wore glasses (26%). Each of the volunteers
played the games individually, and they were asked to play
each game at least once.
A combination of subjective data (questionnaires and
written feedback) and objective data (player performance
and engagement) was used to evaluate various aspects
regarding the behavior of the users and the functionality
and interactivity of the games. A system usability scale
(SUS) [43] was used to evaluate the usability of the gamified reading system, and a short version of the situational
motivation scale (SIMS) [44] was administered to evaluate
the perceived motivation of the users.
Data Analysis and Results
The first usability study was carried out mainly to test
the functionality and interactivity of the games. The
received written feedback was valuable in building the
second design iteration of the system. Table 2 contains

IEEE Systems, Man and Cybernetics Magazine - January 2021

Table of Contents for the Digital Edition of IEEE Systems, Man and Cybernetics Magazine - January 2021