IEEE Robotics & Automation Magazine - June 2021 - 64

with the given target pose pt and required constraints ct
than
it would be without the additional degree of freedom provided
by the virtual joint. Figure 4 sketches a typical situation.
The reader is referred to [17] and [20] for a more comprehensive
discussion of this feature.
Planning Phase
Since we assume that the relative position of the robot with
respect to the shelf is repeatable with good accuracy and
that the uncertainty regarding the locations of items to pick
from the desk is handled at the control level, the whole planning
phase can be carried out offline for each coarse orientation
of the object, assuming an initial pose located in the
center of the pick desk. To present the planning pipeline, it is
convenient to introduce the following reference frames
defined through the corresponding homogeneous transformation
matrices:
●
Tj
b
●
the base frame.
Tg
j
●
Tv
j
is the pose of the frame fixed to object j with respect to
i is the pose of the ith grasp frame withiG1= f j
l is the pose of the lth pregrasp frame withlR1= f j
,,
with respect to the frame fixed to object j; these frames are
the poses where the robot end effector should go before
approaching the object (see Figure 6).
Figure 4. A typical situation where the virtual joint is essential
to find a solution to the pick-and-place task: the initial grasp
configuration is not compatible with the place location on
the shelf.
● Tl
which is a translation of Tg
b
frame upward.
● Ttj
RGB-D Camera
●
Tactile Sensors
b is the pose of the target frame where the object j has to
be placed; the robot will place the product in the outermost
location of that facing, and then it will push it inward, in
impedance-control mode, for a rough amount to leave
enough space for another product.
Tpt
b
j is the pose of the pretarget frame of object j, defined as
a simple translation along the negative direction of the zand
y-axes of the grasp frame of a certain offset from the
target frame.
Note that a pregrasp frame Tv
j
frame Tg
j
l is associated to each grasp
i but not vice versa, and a desired object j image is
associated with each grasp frame, which is the reference
image for the image-based visual-servoing (IBVS) controller
described in the " Visual-Servoing Controller " section.
The sequence of planning requests to MoveIt! is the folFigure
5. The robot end effector equipped with an Intel D435i
RGB-D camera and WSG50 gripper with SUNtouch tactile
sensors installed. The grasp frame is depicted with the RGB
labeling convention.
lowing, where q0
qTTTTT TTt
14 b
j
b
v
j
23 5
l
j
b
g
j
b
pt
b
RR RR R
li jj
is an arbitrary initial robot configuration:
.
(1)
The output of the generic request Ri
is twofold: a motion trajectory in the
joint space ()
qti
that satisfies all
constraints (mainly the absence of collisions)
and a control modality activation
flag for the lower reactive control
layer. It is important to recall that this
sequence is simply a plan that will not
be executed as it is. The actual execution
phase is described in the " Execution
Phase " section.
Planning request R1
generates the
Figure 6. A sample of two pregrasp frames for object P5.
64 * IEEE ROBOTICS & AUTOMATION MAGAZINE * JUNE 2021
motion trajectory to bring the robot
from the initial configuration to the
b is the pose of the end effector after the lifting phase,
j along the z-axis of the base
,, , i.e.,
the frame placed in the grasp center of the gripper (see Figure
5), with respect to the frame fixed to object j.
IEEE Robotics & Automation Magazine - June 2021

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2021
