IEEE Robotics & Automation Magazine - June 2019 - 16

1) Movement alone: The seated robot extends its left leg and
taps the suspended ball.
2) Movement and lights: The seated robot extends its left leg
and taps the suspended ball while all of the robot's LEDs
rapidly and continuously change color using the NAOqi
library's rasta() function.
3) Movement and sound: The seated robot extends its left leg
and taps the suspended ball while performing prerecorded
infant babbling noises.
Procedure
Each infant was brought to the lab for one hour-long session.
Before scheduling the visit and again when the parent and
infant first entered the
experiment space, the parent or legal guardian
The robot's reward
(heretofore referred to as
parent) received a written
behavior was activated
overview and verbal
explanation of the procewhen the infant's right leg
dures. The parent signed
an informed consent form
moved above a resultant
prior to the infant's participation. We affixed one
acceleration threshold.
Opal inertial movement
sensor to each infant limb
using custom-made leg
warmers with pockets. We also attached a head-mounted eye
tracker to the infant.
The infant was then seated in a chair across from and facing the NAO robot, as shown in Figure 2. The parent sat adjacent to the infant. We measured the infant's baseline
movement level while the infant sat in the pictured setup

facing a motionless robot for 2 min. During this phase, the
parent occasionally engaged with the infant in an effort to
maintain interest and prevent fussiness; otherwise, the parent
was asked not to interact with the infant. The infant then
entered the 8-min contingent reward portion of the experiment, during which the reward conditions ran for 2 min and
40 s each. To help maintain infant attention and satisfaction,
there was no break between sequential reward phases. Next, a
so-called extinction phase of 2 min occurred. In the contingent learning literature, the extinction phase is the interval
when the reward is removed. In our study, the robot stopped
moving and reacting during the extinction phase.
At the start of the first contingency condition, the robot
demonstrated three sequential knee-extension ball kicks with
the left leg. After that, the robot moved only when the infant
moved beyond the previously set acceleration threshold.
A video with clips of different procedure phases is available in
the supplemental material included with this article [21].
Following the infant-robot interaction, the parent completed a survey on his or her perceptions of the infant's experiences. A research team member administered the Alberta
Infant Motor Scale assessment [19] to quantify the infant's
motor development status. Infant weight, length, and head
circumference were also measured. Each participating family
received compensation of US$40 for completing the study.
System Architecture and Behavior Control
We designed a system architecture that allowed for real-time
use of sensor readings to trigger robot responses, as shown in
Figure 3. Throughout the session, the Opal sensors provided
data to the robot. Using synchronized streaming, the raw
inertial sensor data were sent to an off-robot computer.
There, the system calculated the instantaneous acceleration

IArch GUI
NAOComp
(Python)

Session
Configuration

SensorComp
(Java)
Wearable Sensor

Robot Instructions
NAO Robot

ExecutiveComp
(Python)

Processed Data From Sensors

Controls and Coordinates the
Execution of the Use Case
Figure 3. The SAR intervention system architecture, designed to integrate inertial sensor data into the robot's contingent behavior
response. GUI: graphical user interface.

IEEE ROBOTICS & AUTOMATION MAGAZINE

JUNE 2019

IEEE Robotics & Automation Magazine - June 2019

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2019