Systems, Man & Cybernetics - April 2015 - 10

New Social
Signals in a New
Interaction World
The Next Frontier for Social Signal Processing

by Alessandro Vinciarelli and Alex "Sandy" Pentland

S

ocial signal processing (SSP) is the domain
aimed at the modeling, analysis, and
synthesis of social behavior, particularly the nonverbal aspects. So far, the field
has focused on face-to-face interactions,
where it is possible to use the whole range of nonverbal
cues that people have at their disposal to communicate
(i.e., gestures, facial expressions, vocalizations, and so
on). However, increasingly more interactions take place
through communication technologies that limit the use
of nonverbal cues (e.g., phone calls do not allow one to
display facial expressions) or that require the adoption of
artificial cues (e.g., "likes" on social media). This opens
a new frontier for SSP, where the main questions are: Do
people still exchange social signals? And if so, how?

Digital Object Identifier 10.1109/MSMC.2015.2441992
Date of publication: 16 September 2015

10

IEEE Systems, Man, & Cybernetics Magazine A pr il 2015

Social Signals
There is no social interaction without social signals, i.e.,
without observable cues allowing people to understand
and reasonably predict the behavior of others. The literature defines social signals in different ways:
◆ acts or structures that influence the behavior or internal state of other individuals [1]
◆ actions whose function is to bring about some reaction
or to engage in some process [2]
◆ communicative or informative signals that provide
information about social facts [3].
While emphasizing different aspects of social interaction, all of the definitions agree on one point, i.e., social
signals are observable behaviors that produce, intentionally or not, tangible changes in others, whether this means
modifying their inner state (e.g., to stimulate the emotions
they experience), modifying their observable behavior (e.g.,
making them laugh in response to a joke), or changing their
2333-942X/15©2015IEEE



Table of Contents for the Digital Edition of Systems, Man & Cybernetics - April 2015

Systems, Man & Cybernetics - April 2015 - Cover1
Systems, Man & Cybernetics - April 2015 - Cover2
Systems, Man & Cybernetics - April 2015 - 1
Systems, Man & Cybernetics - April 2015 - 2
Systems, Man & Cybernetics - April 2015 - 3
Systems, Man & Cybernetics - April 2015 - 4
Systems, Man & Cybernetics - April 2015 - 5
Systems, Man & Cybernetics - April 2015 - 6
Systems, Man & Cybernetics - April 2015 - 7
Systems, Man & Cybernetics - April 2015 - 8
Systems, Man & Cybernetics - April 2015 - 9
Systems, Man & Cybernetics - April 2015 - 10
Systems, Man & Cybernetics - April 2015 - 11
Systems, Man & Cybernetics - April 2015 - 12
Systems, Man & Cybernetics - April 2015 - 13
Systems, Man & Cybernetics - April 2015 - 14
Systems, Man & Cybernetics - April 2015 - 15
Systems, Man & Cybernetics - April 2015 - 16
Systems, Man & Cybernetics - April 2015 - 17
Systems, Man & Cybernetics - April 2015 - 18
Systems, Man & Cybernetics - April 2015 - 19
Systems, Man & Cybernetics - April 2015 - 20
Systems, Man & Cybernetics - April 2015 - 21
Systems, Man & Cybernetics - April 2015 - 22
Systems, Man & Cybernetics - April 2015 - 23
Systems, Man & Cybernetics - April 2015 - 24
Systems, Man & Cybernetics - April 2015 - 25
Systems, Man & Cybernetics - April 2015 - 26
Systems, Man & Cybernetics - April 2015 - 27
Systems, Man & Cybernetics - April 2015 - 28
Systems, Man & Cybernetics - April 2015 - 29
Systems, Man & Cybernetics - April 2015 - 30
Systems, Man & Cybernetics - April 2015 - 31
Systems, Man & Cybernetics - April 2015 - 32
Systems, Man & Cybernetics - April 2015 - 33
Systems, Man & Cybernetics - April 2015 - 34
Systems, Man & Cybernetics - April 2015 - 35
Systems, Man & Cybernetics - April 2015 - 36
Systems, Man & Cybernetics - April 2015 - 37
Systems, Man & Cybernetics - April 2015 - 38
Systems, Man & Cybernetics - April 2015 - 39
Systems, Man & Cybernetics - April 2015 - 40
Systems, Man & Cybernetics - April 2015 - 41
Systems, Man & Cybernetics - April 2015 - 42
Systems, Man & Cybernetics - April 2015 - 43
Systems, Man & Cybernetics - April 2015 - 44
Systems, Man & Cybernetics - April 2015 - 45
Systems, Man & Cybernetics - April 2015 - 46
Systems, Man & Cybernetics - April 2015 - 47
Systems, Man & Cybernetics - April 2015 - 48
Systems, Man & Cybernetics - April 2015 - Cover3
Systems, Man & Cybernetics - April 2015 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/smc_202110
https://www.nxtbook.com/nxtbooks/ieee/smc_202107
https://www.nxtbook.com/nxtbooks/ieee/smc_202104
https://www.nxtbook.com/nxtbooks/ieee/smc_202101
https://www.nxtbook.com/nxtbooks/ieee/smc_202010
https://www.nxtbook.com/nxtbooks/ieee/smc_202007
https://www.nxtbook.com/nxtbooks/ieee/smc_202004
https://www.nxtbook.com/nxtbooks/ieee/smc_202001
https://www.nxtbook.com/nxtbooks/ieee/smc_201910
https://www.nxtbook.com/nxtbooks/ieee/smc_201907
https://www.nxtbook.com/nxtbooks/ieee/smc_201904
https://www.nxtbook.com/nxtbooks/ieee/smc_201901
https://www.nxtbook.com/nxtbooks/ieee/smc_201810
https://www.nxtbook.com/nxtbooks/ieee/smc_201807
https://www.nxtbook.com/nxtbooks/ieee/smc_201804
https://www.nxtbook.com/nxtbooks/ieee/smc_201801
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1017
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0717
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0417
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0117
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1016
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0716
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0416
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0116
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1015
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0715
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0415
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0115
https://www.nxtbookmedia.com