IEEE Robotics & Automation Magazine - September 2021 - 23

the mug, the shared-control skill of pouring is activated,
which results in the rotation of the bottle about its vertical
axis to approach a natural pouring orientation.
Moving closer to the mug, the actual pouring motion is
automatically superimposed onto the user's commands.
First, the bottle is rotated around its center to move the lid
above the mug [Figure 8(b) and (c)]. Once close enough to
the mug, the rotation happens around the tip of the bottle,
as depicted in Figure 8(d) and (e), to allow the actual pouring.
The user can determine the amount of liquid to pour
by specifying the degree of rotation of the bottle by means
of moving toward or away from the mug. Once the liquid
has been poured, the user can retract the bottle from the
mug and place it on the table surface again. Afterward, the
user can intentionally pick up the mug, again with support
of the shared-control system [Figure 8(f)], and, finally,
drink from it.
Opening a Door and Passing Through
The task of opening a door and passing through it is composed
of three parts. First, the wheelchair needs to be
aligned normally to the door plane. Since system transparency
plays a major role when providing assistive robots with
shared-control capabilities, the task of aligning the wheelchair
to the door has to be manually activated by the user.
Therefore, upon detection and localization of the door handle
from the RGB-D data, the subtask of aligning becomes
available on the user's graphical interface and can be activated
by use of the head switch and continuous interface.
Once selected, the wheelchair automatically aligns to the
door such that only pure forward motion is needed to pass
through it [Figure 9(a)]. After alignment, the shared-control
task to open the door is automatically activated. At this
point, the user can command the end effector of the robot,
while the shared-control constraints provide guidance to
t = 0:00
t = 0:12
reach for the handle, automatically enforcing the correct
orientation of the robotic hand [Figure 9(b)-(d)].
Once at the handle, the user commands the manipulator
to press down and finally open the door, while the sharedcontrol
module forces the robotic hand to stay on the path
described by the handle [Figure 9(e)-(g)]. As a result, the
user does not need to take care of the coordinated motion
of the arm and the wheelchair but, instead, has only to provide
a forward command to progress in the task.
During all of these activities, the wheelchair motion is
coordinated by a reactive whole-body controller [22], [33]
to ensure reachability of the robotic arm and, at the same
time, maneuver the wheelchair through the door. Once the
door is fully open, the user can release the handle by commanding
the end effector upward and fully drive through
the door, controlling the wheelchair directly. Afterward, the
door can be pushed until it is closed, making use of the
whole-body control behavior again while freely commanding
the end effector [Figure 9(h)].
Personal Assistance
In contrast to individual independence, as presented in the
previous section, the scenario of personal assistance is
meant for people who still have some level of mobility and
independence but require support in everyday activities. A
typical use case for a robotic personal assistant involves
fetch-and-carry services for household objects.
As an example, a person may request the robot to bring
a remote control or a medicine. Of course, the medicine
could also be served in a time-dependent manner without
the need of an additional user request. In our prototypical
implementation of the ecosystem, the humanoid robot Justin
serves as a personal assistant. It is commanded in supervisory
control via a tablet interface, either directly by the
person in need or remotely by a relative.
t = 0:22
t = 0:29
(a)
(b)
t = 0:31
t = 0:40
(c)
t = 0:50
(d)
t = 1:30
(e)
(f)
(g)
(h)
Figure 9. Individual independence: opening a door and passing through. (a) Approaching the door. (b). (c) A cone-shaped constraint
guides the end effector to the door handle and aligns to the handle accordingly. (d) Downward motion makes the robot press the
latch. (e)-(g) Opening the door and driving through while the end effector follows the motion of the door handle. (h) Closing the
door by pushing on it. The task is executed by means of shared control and whole-body control on EDAN. Performing this task takes
about 2:00 min. Note that the demonstration shown here is with a nondisabled user.
SEPTEMBER 2021 * IEEE ROBOTICS & AUTOMATION MAGAZINE *
23
IEEE Robotics & Automation Magazine - September 2021

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2021
