IEEE Consumer Electronics Magazine - July 2018 - 34

USER SATISFACTION

The decision accuracy was
quantitatively measured by
the percentage of participants
who switched to a different, better
option when, once having finished
the recommendation session,
they browsed all alternatives
in the data set.
bought the same product (i.e., a higher decision accuracy),
the majority of these purchases were done so collaboratively
(i.e., for 65% of the cycles), whereas only 35% of these purchases were done so individually. By contrast, when users
bought a different product, they were doing so individually
(i.e., for 75% of cycles); meanwhile, the rest of the interactions were done collaboratively (i.e., for 25% of cycles). These
results show the effectiveness of the CCR approach, which
assists users in finding their desired products.

ACG Selection (%)

70
60
50
40
30

To collect user satisfaction measurements, a posttest questionnaire that follows the methodology proposed in [27] was
administered. A careful examination of the posttest results
suggests focusing on the following dimensions (Figure 8):
learnability, collaboration, perceived accuracy, perceived
usefulness, and user satisfaction. The learnability dimension
analyzes how easy it is for the user to learn to use the recommender. Second, collaboration expresses the perceived user
experience when a user collaborates with others. A user's
perceived accuracy is the degree to which the user feels the
recommendations match their interests and preferences. The
user's perceived usefulness or utility of the recommender is
observed by comparing it with an environment that lacks a
recommender. Finally, a user's satisfaction refers to the global impression users have about their experiences working
within the framework.
Participants perceived a high level of recommender accuracy with an average of 5.9 (as indicated by the perceived
accuracy graph in Figure 8). Figure 8 suggests that users perceive that the recommender assists them during the recommendation process, and, overall, participants found it easy to
learn how to interact with the recommender and with other
users (as indicated by the 5.78 score in the learnability graph
in Figure 8). Moreover, the user satisfaction of participants is
high with an average satisfaction score of 5.68. The favorable
results in these three dimensions encourage the further development of this 3-D VW.
In general, participants found the recommender to be useful when searching for a specific product, with an average
score of 5.58 in the usefulness dimension in the graph. However, a lower score in the dimension of collaboration (4.98)
may reflect the belief that, although collaboration helps, it is
not an essential part of the recommendation process.

CONCLUSION

10
0
The IC Without
a Target (Task 5)

The CCR Without
a Target (Task 4)
(a)

This article, based on a previously proposed CCR, defines
a state-based model of user-recommender interaction and
presents a specific implementation of the CCR framework
in a 3-D VW. A 3-D interface may have advantages over

120

FIGURE 7. The (a) decision accuracy and (b) decision analysis in
the user evaluation.

34 IEEE Consumer Electronics Magazine

July 2018

io
fa
ct
Sa
tis

tio
n

ln
es
fu
se
U

Collaborative Ratio
(b)

P
Ac erc
cu eiv
ra ed
cy

Individual Ratio

Selected the Same Selected a Different Product

3
2

la
bo

35%

75%

5
4

bi
lit

65%

Postquestionnaire Results

7
6

Average Score Value

25%

Le
ar

Cycles Ratio (%)

100

Dimensions of Evaluation

FIGURE 8. The average dimensions score value and standard deviation for the posttest questionnaire in a seven-point Likert scale.

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - July 2018