Systems, Man & Cybernetics - April 2015 - 12

Modeling

Psychological Level

the range of social signals that
there are now 3 billion Internet
people can display, e.g., phones
users worldwide (roughly 40% of
SSP is the computing
that only allow the use of vocal
the population) and over 2 billion
domain aimed at the
behavior. The second is the use
active social media accounts (29%
of t ech nolog ie s t h at re qu i re
of the world population).
modeling, analysis,
people to use social signals fully
The initial reaction to the the
and synthesis of
alternative to those available in
social media phenomenon was an
social signals in
face-to-face interactions. This
attempt to transfer the social sigis the case of social networking
nals typical of face-to-face interhuman-human and
platforms, where people interact
actions to the virtual world. For
human-machine
extensively but cannot use any of
example, emoticons are a popular
the social signals that have been
way to communicate emotional
interactions.
studied so far in SSP.
states, and Second Life, an interaction platform where people act
State of the Art
through avatars, is an attempt to reproduce human behavSo far, SSP has focused on settings like the one depicted in
ior. However, the lack of similar functionalities does not
the upper part of Figure 1: one individual displays observappear to be an obstacle toward the success of social
able behavior, particularly nonverbal cues, to perform all
media and mobile apps allowing people to interact. Therefunctions necessary to interact with others [11], [12].
fore, the question is not only how to transfer face-to-face
The repertoire of available nonverbal cues is wide,
social signals to the online world but also what the social
and, according to a taxonomy commonly applied in [13], it
signals are in these new, technology-mediated settings and
includes five major classes (see cues in Figure 1):
how they are exchanged.
From this point of view, there are two main situ◆ face and head behavior (e.g., facial expressions, gaze
ations. The first is the use of technologies that limit
contact management, and head nods and shakes)

Cues:

Functions:

Face and Head
Behavior

Impression
Management

Vocal Behavior

Emotion
Expression

Gestures and
Posture

Sending Relational
Messages

Appearance

Deceiving and
Deception Detection

Analysis

Social Verticality
Expression

Automatic
Detection of Cues

Automatic
Interpretation of Cues

Synthesis

Computational Level

Space and
Environment

Automatic
Generation of Cues

Interaction
with Users

Figure 1. The process of SSP. The upper part of the figure (psychological level) shows the two main elements

of face-to-face interactions, nonverbal cues, and functions that these contribute to perform (the subject of
modeling in SSP). The lower part (computational level) shows the main technological components of analysis
and synthesis approaches.

IEEE Systems, Man, & Cybernetics Magazine A pr il 2015

Table of Contents for the Digital Edition of Systems, Man & Cybernetics - April 2015