IEEE Robotics & Automation Magazine - June 2021 - 66

assuming that no collision can occur because the robot is far
enough from the shelf and the pick desk.
The second motion segment is executed by activating the
visual-servoing controller, which tries to achieve the grasp
pose based on the RGB-D camera images only, as detailed in
the " Visual-Servoing Controller " section. As soon as the visual
tracking error is below a threshold, the gripper is commanded
to grasp the object with a slipping-avoidance control
mode, and the third motion segment is executed by lifting the
object far from the pick desk.
At the end of this motion, the configuration of the robot ql
is not the same as at the end of planning request R3
the visual-servoing action; therefore, the trajectories ()
due to
qt4i
cannot be executed exactly as planned. An exponential connection
to the planned trajectories is implemented to asymptotically
bring the actual robot desired motion ()
planned one:
qt4
qt qq te tt
l
44 4
() (()) --a +
=-tt(),
() qt
to the
(2)
where α is a parameter that establishes the convergence rate,
and
qt4 () = *h
where Ii
qt
qt
41
4N
()
()
t
t I
!
!
I1
,
N
last motion segment computed by request R5
as it is.
represents adjacent time intervals from t0 to 3. The
is executed
Reactive Control
The reactive control layer, apart from the two modules
described in the following sections, also includes force- and
impedance-control modes of the arm that are useful during
the product placing and pushing maneuvers cited in the definition
of the target pose Tt
b
j
.
Visual-Servoing Controller
The object pose given by the object-detection module is a
rough estimation of the real one. We need an algorithm able to
adjust this pose. Based on extensive experience, we estimate
that the real and nominal grasping points should be as close as
2 mm for an effective pivoting maneuver. This required accuracy
cannot be ensured by the recent vision-based, open-loop
approaches to 6D pose estimation, e.g., [11], that can handle a
large number of objects even in a partial occlusion.
Unfortunately, they yield a successful pose localization if
the value of the average closest point distance (ADD-S) metric
is below 2 cm, a threshold not suitable for our application.
For this reason, we chose a closed-loop visual-servoing
approach. The only drawback is that such an approach is suitable
only for texture-rich objects. Nevertheless, the objects
considered in the REFILLS project have many labels that
make them texture rich.
The visual-servoing module, based on the Visual Servoing
Platform Library [21], uses data acquired from the depth camera
66 * IEEE ROBOTICS & AUTOMATION MAGAZINE * JUNE 2021
(3)
to control the movement of the robot in real time, with the aim
of adjusting the robot pose with respect to the object to correctly
grasp it. The RealSense D435i depth camera has been arranged
in an eye-in-hand configuration. The IBVS controls the robot
motion by minimizing the error between a set of previously
learned features s)^h and those identified in the actual image, s:
et sIts t
rr
ie
() (( ), ,) (),=- )
(4)
where I(t) is the actual image, and ir and er are the intrinsic
and extrinsic parameters of the camera, respectively. In a
common IBVS, the image features are vectors of 2D matching
points, which the algorithm aligns in the camera image plane.
The proposed approach, instead, uses 3D feature points
obtained from the depth image. The module acquires in real
time the actual RGB-D image and tries to align s to s*
moving the camera with the velocity
()
c
vt =- mLe t
(),
(5)
where L is the interaction matrix and λ is the control gain that
yields the exponential convergence of the error [21]. The
object-tracking algorithm adopted in this work is the keypoint
tracker, which recognizes useful points and tracks them
in subsequent images. Using a constant s*
by
corresponds to
applying a step reference to the control algorithm.
For stability reasons, the high initial error imposes the use
of low gains; typically, an adaptive gain is used that is low
when the error is high, and it increases as the error declines.
This approach implies a slower motion in the initial phase. To
speed up the robot motion during the visual-servoing phase
by using higher gains while ensuring stability, we generate a
time-varying (),st*
and the target ones.
interpolating the initial features (( ))sI 0
Grasp Controller
The manipulation abilities exploited and automatically selected
by the planner can be achieved by controlling the grasp
force fn used by the parallel gripper adopted in REFILLS for
grasping the products. The model of the soft contact proposed
in this article is based on classical results in the literature
on distributed contacts [22].
The fingertip, embedding a force/tactile sensor, is hemispherical,
with a curvature radius lower than those of the manipulated
objects so that the contact surface can be approximated with a
circle and the pressure distribution can be assumed axisymmetric.
Another assumption is that the object is much stiffer than the
fingertip so that it can be considered rigid.
Under these assumptions, the maximum force ft and torsional
moment τn that the contact surface can withstand
with dry friction belong to a geometrical locus in the plane
(, )ftnx called the limit surface (LS) [22]. Such a curve can be
used to predict the sliding motion given the measured force
and torque. Figure 8 shows the LS normalized with respect to
the maximum possible values of the friction force and
torque, i.e.,
IEEE Robotics & Automation Magazine - June 2021

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2021
