IEEE Robotics & Automation Magazine - September 2021 - 24

t = 0:10
t = 0:40
t = 1:20
t = 1:50
(a)
(b)
(c)
(d)
Figure 10. A personal assistant: a fetch-and-carry service with a medicine dispenser. (a) Opening the drawer. (b) Grasping the
dispenser. (c) Closing the drawer. (d) Carrying the dispenser. The execution time for this task is fewer than 2:00 min. The insets show
a bird's eye view.
In this scenario, the medicine is located in a kitchen
drawer. Using the GUI on the tablet, one can command the
robot to move to the kitchen and localize the drawer of
interest. The robot recognizes objects and their locations in
the environment autonomously. Once objects are localized,
possible actions involving the respective target objects are
proposed in the GUI. This way, the user can choose to open
the drawer, upon which the robot plans and executes a collision-free
motion to fulfill the requested task [Figure 10(a)].
When the drawer is open, the medicine dispenser can be
localized, and the robot grasps and places it on the kitchen
counter at the user's request [Figure 10(b) and (c)]. Afterward,
the robot is commanded to close the drawer and
transfer the dispenser to the person in need [Figure 10(d)].
In other fetch-and-carry scenarios, it is also conceivable
that the user does not know where the object is located, for
example, when searching for a glasses case. Given that the
personal assistance robot has a database of objects belonging
to the user, the task is one of finding a known object in
a domestic environment, moving there, grasping the object,
and bringing it to a placement area close to the user. Within
the ecosystem, this task is demonstrated using 3D modeling
and object-recognition capabilities, as described in the
" Autonomous Manipulation " section.
Once the request to retrieve the glasses case is issued by
the user, e.g., via voice command, the robot creates a 3D
model of the room and starts scanning for the requested
object [Figure 11(a) and (b)]. Upon successful localization
of the glasses case in the RGB images, the 3D room model
is used to autonomously navigate close to the object [Figure
11(c)] so that its pose can be estimated [Figure 11(d)]
and it can be autonomously grasped [Figure 11(e)]; then,
the object is brought to the user or placed somewhere safe
[Figure 11(f)].
t = 0:10
t = 0:40
Telepresence Support
Haptic teleoperation can serve as an alternative solution in
emergency situations and for tasks that cannot be autonomously
achieved. In the EDAN use case, one task identified
during the interviews was changing the position of a blanket
at night, for instance, when the
person feels too warm or cold. Potential
users of the system reported that
they sometimes feel uncomfortable
waking caregivers to adjust a blanket
by a few centimeters.
On the other hand, the ability to
t = 1:10
(a)
t = 1:40
(b)
t = 1:50
(c)
t = 2:20
(d)
(e)
(f)
Figure 11. A personal assistant: retrieving a lost glasses case. (a) Creating a 3D model
and searching for the glasses case (marked in red). (b) The resulting 3D model of the
environment. (c) Planning collision-free motion and approaching the case. (d) The
recognized pose of the glasses case used for grasping. (e) Grasping the glasses case in a
confined environment. (f) Placing the glasses case on the kitchen counter. The execution
time for this task is 2:20 min.
24 * IEEE ROBOTICS & AUTOMATION MAGAZINE * SEPTEMBER 2021
safely and robustly handle a blanket
in autonomous mode is currently
beyond the state of the art in robotics.
However, haptic teleoperation
offers a practical solution for such a
case. Moreover, direct teleoperation
can also be used if a task has not yet
been implemented on the system, for
example, when the user has purchased
a new kitchen appliance not
yet known to the robot.
Using direct teleoperation of
EDAN, we have demonstrated the
scenario of operating a microwave,
as depicted in Figure 12. The teleoperator
opens the microwave by pushing
the correct button [Figure 12(a)
IEEE Robotics & Automation Magazine - September 2021

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2021
