IEEE Robotics & Automation Magazine - September 2019 - 21

Figure 5(a), the human is operating without any tool and,
thus, without the effect of any external forces. Therefore, the
overloading on the joints is always low.
Finite-State Machine
The integration of various components described previously
into a unified HRC framework was achieved using a global
finite-state machine (FSM), depicted in Figure 4. This was
due to the requirement for continuous transitions from one
state to another in response to the external inputs and to
coordinate the data exchange among the three modules.
Communication among the different modules of the framework, such as ergonomics, control, and vision, was implemented via UDP messages, while submodules of the vision
system utilized YARP protocols (that is, a combination of
UDP, Transmission Control Protocol, and so on).
The FSM's initial state is identified by the homing primitive in which the robot end effector is in the center of the
workspace waiting for an input from the vision module.
When the vision system recognizes that the subject is holding a tool, the object-picking state is triggered, and the robot
arm picks the corresponding part, bringing it close to the
tool to start the comanipulation stage. In this phase, the
stiffness and damping matrices are modulated to avoid deviations from the desired trajectories caused by the part's
weight. After receiving the optimization pose from the ergonomics module, the robot moves toward the desired configuration, where the comanipulation can be executed more
comfortably. At this stage, three possible actions can be triggered as shown in Figure 4: hand change, human pose
change, and external force detection. If the subject moves in
the workspace or he or she switches the tool hand, the vision
module communicates the relative changes to the control
module, and a new path is generated from the trajectory
replanning and executed by the move primitive. On the
other hand, if external forces are detected on the y axis, the
object is returned to its original position, and the robotic
arm goes back to the homing configuration.
Experimental Setup
Figure 2 illustrates the experimental setup in our study. As
noted previously, we used the KUKA LBR iiwa robotic
manipulator equipped with the Pisa/IIT softhand. We
chose to use a synergy-driven, underactuated robotic
hand because of the mechanical adaptability of the hand
to the shapes of the parts, which simplifies grasp planning
and control.
All software components were implemented in a C++
environment. The robot was controlled by joint torque, and
the torque commands were sent to KUKA using Fast
Research Interface (FRI) at 500 Hz. The Cartesian impedance
controller in the section "Robot Interaction Controller" was
implemented by setting the stiffness and damping gains of the
FRI's default joint impedance controller to zeros. This allowed
us to directly set the torque references related to the impedance behavior of the robot; in our case, this was the desired

stiffness and damping as we targeted quasi-static interactions
between humans and the iiwa robot.
The hand was controlled using a custom control board
that implements an outer position loop and inner current
regulator (that is, impedance controlled) at 1 kHz. The
hand control gains were tuned to have a firm grasp of the
parts and kept constant during the experiments. The reliability of each module's performance and the communication between modules was important, because a continuous
demonstration of the proposed ergonomic and reconfigurable HRC framework for the entire duration of the KUKA
Innovation Award (one week, 8 h/day) was a critical measure for success.
Ten healthy subjects (eight males and two females; age,
30.2 ± 3.7 years; weight, 79.6 ± 10.9 kg; and height, 178.4 ±
5.5 cm) participated in the overall experiments. An electric
drill/screwdriver (4 kg) and a polisher (3 kg) were placed
next to the subjects. Each tool was associated with a part to
be manipulated (see also Figure 6). Two components of a
robot actuator (outer shell and inner part) were chosen for
this purpose. The subjects were asked to pick a tool by
choice [Figure 6(a) and (g)], in a random order, perform the
manipulation, and push the robot in the y axis [Figure 6(f)]
to end the task. While manipulating, the subjects could
change hands [Figure 6(c) and (d)] and move in the workspace arbitrarily and repeatedly [Figure 6(e)]. Nevertheless,
they were instructed to follow the robot end-effector movements in the sagittal plane (xz) and possibly align the wholebody pose to the one illustrated by the graphical interface
[Figure 6(b) and (c)] to achieve ergonomic postures. The
current pose and the optimal one were illustrated to the user
in black and blue, respectively.

High

Medium

Low

(a)

(b)
Figure 5. Examples of the information provided by the
graphical interface. (a) Example postures without a tool.
(b) Example postures with a tool. The levels of joint torque
overloading are color coded to denote a high (red), medium
(orange), or low (green) value and are illustrated in the main
joints of the human body.

SEPTEMBER 2019

IEEE ROBOTICS & AUTOMATION MAGAZINE

IEEE Robotics & Automation Magazine - September 2019

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2019