IEEE Robotics & Automation Magazine - September 2019 - 25

the overloading of joint torques for subphase (A) among 10 subjects was 30.24% ± 2.38% (mean ± standard error of mean) as
follows: 32.65% ± 16.12% in the hip, 33.50% ± 14.16% in the
knee, 32.54% ± 12.44% in the ankle, 37.21% ± 15.23% in the
shoulder, and 15.30% ± 17.25% in the elbow. Statistical significance tests were conducted at the 0.05 significance level using
the Bonferroni post hoc t-test. The optimized configuration
showed a significantly lower overloading effect in all joints
compared with the initial configurations. The difference values of joint torque overloading in each joint were 8.24 ± 0.97 Nm
(p #0.001) in the shoulder, 2.13 ± 0.65 Nm (p #0.001) in the
elbow, 9.29 ± 1.31 Nm (p #0.001) in the hip, 10.37 ± 1.26 Nm
(p #0.001) in the knee, and 10.62 ± 1.18 Nm (p #0.001) in
the ankle. These results provide solid evidence of the cobot's
ability to make online adaptations to human factors and so
contribute to better ergonomics.
Conclusions and Future Work
This article presented the development of a unified HRC
framework that aims to improve human ergonomics and the
reconfigurability of the production/assembly processes in
industrial environments. The proposed framework enabled a
cobot to simultaneously adapt to user states, such as pose,
overloading torques, manipulating hand, positional variations
in the workspace, and task condition, by detecting the tools
and parts in the workspace.
A human-robot comanipulation task was further considered in this study, with 10 subjects participating in drilling/
polishing experiments. The framework was intensely evaluated during the KUKA Innovation Award at Hannover Messe,
where we ran 8 h of live daily demonstrations for one entire
week. The accuracy of the algorithm in tracking multiple subjects and tools and its robustness to the detection of multiple
visitors at the booth (so that robot behavior is adapted to the
subject and not a visitor in the crowd) were validated, demonstrating its high potential in realistic industrial environments.
One of the key extensions of this article with respect to
our previous approach in [14] was the introduction of an
external vision system for real-time tracking of the human's
pose and estimation of the human's intention (by recognizing
handheld tools). This consideration was intended to improve
the cost effectiveness and applicability of the proposed HRC
framework in cross-domain industrial scenarios. The accuracy of the human pose tracking using the visual system was
evaluated by comparing 10 joints of the human body to the
results of a commercial, wearable suit during arbitrary yet
extensive whole-body movements. Although the joints of the
lower body demonstrated low differences, the upper body
joints had slightly higher values: the upper body joints can be
covered by human hands or tools during the action, which
causes high errors in the vision system's reconstruction step
from 2D to 3D (see the section "Human Whole-Body Kinematic Tracking"). Nevertheless, the overall accuracy of the
visual tracking system was acceptable for our experiments.
Future work will focus on the introduction of a face recognition component in the vision module so that the SESC

parameters of the workers and other subject-specific information can be updated autonomously. This will enable cobots to
assist multiple workers, with robot responses optimized to
each individual. Next, robot mobility will be added to the
framework so that a cobot can follow the subject in the production line and provide assistance. This concept is similar to
wearable force-augmenting exoskeletons, with the major difference being that the cobots do not add extra weight to users
or affect user comfort.
Acknowledgments
The Human-Robot Interfaces and Physical Interaction Laboratory contributed to the development of human dynamic
modeling and optimization, robot interaction control, and
feedback interfaces. The iCub Facility and Humanoid Sensing
and Perception Laboratory contributed to robotic vision, i.e.,
object recognition and human kinematic tracking. The work
presented here was the winner of the KUKA Innovation
competition 2018. Phuong D.H. Nguyen was supported by a
Marie Curie Early Stage Researcher Fellowship (H2020MSCA-ITA, SECURE 642667).
References
[1] A. A. Shikdar and N. M. Sawaqed, "Worker productivity, and occupational health and safety issues in selected industries," Comput. Ind.
Eng., vol. 45, no. 4, pp. 563-572, 2003.
[2] M. G. Mehrabi, A. G. Ulsoy, and Y. Koren, "Reconfigurable manufacturing systems and their enabling technologies," Int. J. Manuf. Technol.
Manage., vol. 1, no. 1, pp. 114-131, 2000.
[3] A. Ajoudani, A. M. Zanchettin, S. Ivaldi, A. Albu-Schäffer, K.
Kosuge, and O. Khatib, "Progress and prospects of the human-robot
collaboration," Auton. Robots, vol. 42, no. 5, pp. 957-975, 2018.
[4] I. El Makrini, K. Merckaert, D. Lefeber, and B. Vanderborght,
"Design of a collaborative architecture for human-robot assembly
tasks," in Proc. 2017 IEEE/RSJ Int. Conf. Intelligent Robots and Systems
(IROS), 2017, pp. 1624-1629.
[5] B. Matthias and T. Reisinger, "Example application of ISO/TS 15066
to a collaborative assembly scenario," in Proc. ISR 2016: 47th Int. Symp.
Robotics, 2016, pp. 1-5.
[6] K. Kosuge and N. Kazamura, "Control of a robot handling an object
in cooperation with a human," in Proc. 6th IEEE Int. Workshop Robot
and Human Communication, 1997, pp. 142-147.
[7] D. J. Agravante, A. Cherubini, A. Bussy, P. Gergondet, and A. Kheddar, "Collaborative human-humanoid carrying using vision and haptic
sensing," in Proc. 2014 IEEE Int. Conf. Robotics and Automation, 2014,
pp. 607-612.
[8] L. Peternel, N. Tsagarakis, and A. Ajoudani, "A human-robot comanipulation approach based on human sensorimotor information,"
IEEE Trans. Neural Syst. Rehabil. Eng., vol. 25, no. 7, pp. 811-822, July
2017.
[9] F. Ficuciello, A. Romano, L. Villani, and B. Siciliano, "Cartesian
impedance control of redundant manipulators for human-robot comanipulation," in Proc. 2014 IEEE/RSJ Int. Conf. Intelligent Robots and
Systems, 2014, pp. 2120-2125.
[10] L. Roveda, "A user-intention based adaptive manual guidance with
force-tracking capabilities applied to walk-through programming for

SEPTEMBER 2019

IEEE ROBOTICS & AUTOMATION MAGAZINE

IEEE Robotics & Automation Magazine - September 2019

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2019