IEEE Robotics & Automation Magazine - December 2012 - 35

switching rule. In this article, we focused on learning the
switching rule only. However, for more complex interaction
scenarios it might be important to adapt all of these parameters. Another limitation of the proposed learning algorithm
is the use of binary evaluation information. As a result, optimization of the parameters in a gradient descent manner is
not possible. Another drawback of binary evaluation information is that only positive feedback examples are retained
for use in the learning set while negative feedback examples
are removed from the learning set. With respect to the first
limitation, the desired postures and feedback gains can be
regarded as attractors and velocities in a low-dimensional
space. Amor et al. [7] have shown that such attractors can be
efficiently learned in a low-dimensional space while incorporating kinesthetic assistance provided by the user. In the
future, we therefore hope to integrate such a method into
the proposed PHRI algorithm. As for the second limitation,
we are considering the use of pressure sensors on the body
of the robot. The amount of pressure issued by the caregiver
can then be used as an approximate evaluation information.
This allows for a finer grained reward value and, consequently, the use of modern optimization algorithms. Pressure sensors are also helpful to distinguish whether the
human is currently in contact with the robot.
In summary, this study provided interesting insights into
the dynamics of PHRIs. The combination of a softbody
robot and an efficient learning scheme is an important step
toward responsive robots that share a common living space
with humans.
References
[1] S. Ikemoto, T. Minato, and H. Ishiguro, "Analysis of physical human-robot
interaction for motor learning with physical help," Appl. Bionics Biomech.
(Special Issue on Humanoid Robots), vol. 5, no. 4, pp. 213-223, 2008.
[2] S. Ikemoto, H. B. Amor, T. Minato, H. Ishiguro, and B. Jung, "Physical
interaction learning: Behavior adaptation in cooperative human-robot tasks
involving physical contact," in Proc. IEEE Int. Symp. Robot and Human Interactive Communication (Ro-Man), Sept. 2009, pp. 504-509.
[3] A. De Santis, B. Siciliano, A. De Luca, and A. Bicchi, "An atlas of physical
human-robot interaction," Mechanism Mach. Theory, vol. 43, no. 3,
pp. 253-270, Mar. 2008.
[4] O. Khatib, K. Yokoi, O. Brock, K. Chang, and A. Casal, "Robots in human
environments," in Proc. 1st Workshop Robot Motion and Control, 1999,
pp. 213-221.
[5] S. Yohanan and K. E. MacLean, "The haptic creature project: Social
human-robot interaction through affective touch," in Proc. AISB Symp. Reign
of Catz & Dogs: The 2nd AISB Symp. Role of Virtual Creatures in a Computerised Society, 2008, vol. 1, pp. 7-11.
[6] K. Kosuge, T. Hayashi, Y. Hirata, and R. Tobiyama, "Dance partner robot-
MS-danSer," in Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems, 2003,
vol. 3, pp. 3459-3464.
[7] H. B. Amor, E. Berger, D. Vogt, and B. Jung, "Kinesthetic bootstrapping:
Teaching motor skills to humanoid robots through physical interaction," KI
2009: Advances in Artificial Intelligence (ser. Lecture Notes in Artificial Intelligence), B. Mertsching, Ed. Berlin, Germany: Springer-Verlag, 2009,
pp. 492-499.

[8] J. Tani, R. Nishimoto, J. Namikawa, and M. Ito, "Codevelopmental learning
between human and humanoid robot using dynamic neural network model,"
IEEE Trans. Syst., Man, Cybern., vol. 38, no. 1, pp. 43-59, 2008.
[9] S. Calinon and A. Billard, "What is the teacher's role in robot programming by demonstration?-Toward benchmarks for improved learning" Interaction Studies (Special Issue on Psychological Benchmarks in Human-Robot
Interaction), vol. 8, no. 3, pp. 441-464, 2007.
[10] T. Odashima, M. Onishi, K. Tahara, K. Takagi, F. Asano, Y. Kato,
H. Nakashima, Y. Kobayashi, T. Mukai, Z. W. Luo, and S. Hosoe, "A soft
human-interactive robot ri-man," in Proc. IEEE/RSJ Int. Conf. Intelligent
Robotics and Systems (IROS, v018), 2006, p. 1-1 (video session).
[11] P. Evrard, E. Gribovskaya, S. Calinon, A. Billard, and A. Kheddar, "Teaching physical collaborative tasks: Object-lifting case study with a humanoid," in
Proc. IEEE-RAS Int. Conf. Humanoid Robots (Humanoids), Dec. 2009,
pp. 399-404.
[12] D. Lee, C. Ott, and Y. Nakamura, "Mimetic communication with impedance control for physical human-robot interaction," in Proc. IEEE Int. Conf.
Robotics and Automation (ICRA), 2009, pp. 1535-1542.
[13] B. D. Argall and A. G. Billard, "A survey of tactile human-robot interactions," Robot. Autonom. Syst., vol. 58, no. 10, pp. 1159-1176, 2010.
[14] G. Schwarz, "Estimating the dimension of a model," Ann. Statist, vol. 6,
no. 2, pp. 461-464, 1978.
[15] T. Minato, Y. Yoshikawa, T. Noda, S. Ikemoto, H. Ishiguro, and M. Asada,
"Cb2: A child robot with biomimetic body for cognitive developmental robotics," in Proc. IEEE-RAS/RSJ Int. Conf. Humanoid Robots (Humanoids), 2007,
pp. 557-562.
[16] A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihood from
incomplete data via the EM algorithm," J. R. Statist. Soc. Series B (Methodological), vol. 39, no. 1, pp. 1-38, 1977.
[17] L. R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol. 77, no. 2, pp. 257-285, Feb. 1989.
[18] K. Fukunaga and D. Olsen, "An algorithm for finding intrinsic dimensionality of data," IEEE Trans. Comput., vol. 20, no. 2, pp. 176-183, 1971.
[19] Y. Kuniyoshi, Y. Ohmura, K. Terada, A. Nagakubo, S. Eitoku, and
T. Yamamoto, "Embodied basis of invariant features in execution and perception of whole body dynamic actions-Knacks and focuses of rolland-rise
motion," Robot. Autonom. Syst., vol. 48, no. 4, pp. 189-201, 2004.

Shuhei Ikemoto, Department of Multimedia Engineering,
Osaka University, Japan. E-mail: ikemoto@ist.osaka-u.ac.jp.
Heni Ben Amor, Intelligent Autonomous Systems Group, Technische Universitaet Darmstadt, Germany. E-mail: amor@ias.
tu-darmstadt.de.
Takashi Minato, ATR Hiroshi Ishiguro Laboratory, Kyoto,
Japan. E-mail: minato@atr.jp.
Bernhard Jung, Virtual Reality and Multimedia Group, Technische Universität Bergakademie Freiberg, Germany. E-mail:
jung@informatik.tu-freiberg.de.
Hiroshi Ishiguro, Department of Systems Innovation, Osaka
University, Japan. E-mail: ishiguro@is.sys.es.osaka-u.ac.jp.

DECEMBER 2012

*

IEEE ROBOTICS & AUTOMATION MAGAZINE

*

35
http://www.tu-darmstadt.de
Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2012
