IEEE Robotics & Automation Magazine - September 2021 - 17

By selecting an object overlay on the tablet screen,
like one of the mugs in the scene shown in Figure 4(c),
the operator receives the object-specific command set
that displays only the current commands of interest [Figure
4(c)]. After selecting one of the commands, e.g., to
clean a mug or to stack them, the robot performs the
task autonomously. This technology has already been
successfully validated in space missions as a remote-control
tool. In particular, several astronauts located on the
International Space Station (ISS) have used this intuitive
interface to control the humanoid robot Justin remotely
from orbit [20].
Using the mission control system developed for the ISS
experiments, the user interface can be adapted to the user's
level of experience:
●
●
It can provide novice users with an intuitive user in -
terface that displays only the most important abstract
commands.
Experienced users can access more complex robot functions
and manually parameterize the autonomous task
execution.
●
In the future, trained health-care professionals may be
able to perform patient-specific medical diagnostic or
therapeutic procedures.
Natural Language
Ultimately, the use of natural language combined with an
audio interface promises to be the most intuitive manner
of interaction with a robotic assistant. Thus, the cognitive
load can be further reduced, and the
ease of interaction is improved. For
the current generation of elderly
people who may not yet be familiar
with smartphones or tablets, a natural-language
interface may be most
appropriate. So far, the automated
semantic analysis of natural-language
commands does not cover
a wide variety of tasks. As such,
speech recognition can be used to
trigger the autonomous functions of
a robot that are known to the user,
but full speech-based commanding
is not yet achievable.
All of the aforementioned interfaces
are designed to provide designated
users of the assistive systems
with control capabilities. Additionally,
the ecosystem allows for direct
haptic teleoperation of the systems.
This additional interface is intended
for professionals, i.e., trained teleoperators,
to access and command the
system in case of an emergency or
when user-level functionalities do
not suffice to fulfill a task.
Levels of Autonomy
In addition to user-specific interfaces, the level of autonomy
of the robot also needs to be tailored to the particular requirements
given by the user. Currently, the ecosystem is composed
of four user-level control modes offering different levels
of robot autonomy (Figure 5). Analogously to the interfaces,
these control modes are complemented by a telepresence
option in which an expert takes control over the system.
Direct Control
The most basic control scheme is direct control, in which the
user instantly commands the pose of the end effector of the
robot via an appropriate user interface. Actually, for wheelchair-mounted
robot arms, similar to EDAN's, direct control
based on joystick commands is the most commonly used mode
of operation. Typically, a joystick provides two or three continuous
velocity commands, which are mapped to subsets of the
task-space DoF of the end effector. An additional trigger signal
makes it possible to switch among different sets of input mappings
(IMs), including the tool or gripper, to provide the user
with full control over the system. Many activities of daily living
can be performed with the approach of direct control, essentially
transferring the task of motion planning to the human operator.
In EDAN, direct control is achieved with a 3-DoF input
device, based either on a joystick or on measuring residual
muscular activity. In combination with the trigger signal, this
enables the user to switch among purely translation control,
rotational control, or opening and closing of a specific grasp
type of the robotic hand. Using this cycle of control modes
(a)
(b)
(c)
Figure 4. The tablet interface app for Justin. (a) A user with the tablet app in map view
mode. (b) A detailed view of the map view mode, which allows the user to move the
robot in the room. (c) The camera view mode, which allows the user to manipulate
detected objects, including close ups of the user menu listing available views and
actions. The sliders on the bottom and the left of the map view mode allow the user to
change the robot's field of view.
SEPTEMBER 2021 * IEEE ROBOTICS & AUTOMATION MAGAZINE *
17

IEEE Robotics & Automation Magazine - September 2021

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2021

Contents
IEEE Robotics & Automation Magazine - September 2021 - Cover1
IEEE Robotics & Automation Magazine - September 2021 - Cover2
IEEE Robotics & Automation Magazine - September 2021 - Contents
IEEE Robotics & Automation Magazine - September 2021 - 2
IEEE Robotics & Automation Magazine - September 2021 - 3
IEEE Robotics & Automation Magazine - September 2021 - 4
IEEE Robotics & Automation Magazine - September 2021 - 5
IEEE Robotics & Automation Magazine - September 2021 - 6
IEEE Robotics & Automation Magazine - September 2021 - 7
IEEE Robotics & Automation Magazine - September 2021 - 8
IEEE Robotics & Automation Magazine - September 2021 - 9
IEEE Robotics & Automation Magazine - September 2021 - 10
IEEE Robotics & Automation Magazine - September 2021 - 11
IEEE Robotics & Automation Magazine - September 2021 - 12
IEEE Robotics & Automation Magazine - September 2021 - 13
IEEE Robotics & Automation Magazine - September 2021 - 14
IEEE Robotics & Automation Magazine - September 2021 - 15
IEEE Robotics & Automation Magazine - September 2021 - 16
IEEE Robotics & Automation Magazine - September 2021 - 17
IEEE Robotics & Automation Magazine - September 2021 - 18
IEEE Robotics & Automation Magazine - September 2021 - 19
IEEE Robotics & Automation Magazine - September 2021 - 20
IEEE Robotics & Automation Magazine - September 2021 - 21
IEEE Robotics & Automation Magazine - September 2021 - 22
IEEE Robotics & Automation Magazine - September 2021 - 23
IEEE Robotics & Automation Magazine - September 2021 - 24
IEEE Robotics & Automation Magazine - September 2021 - 25
IEEE Robotics & Automation Magazine - September 2021 - 26
IEEE Robotics & Automation Magazine - September 2021 - 27
IEEE Robotics & Automation Magazine - September 2021 - 28
IEEE Robotics & Automation Magazine - September 2021 - 29
IEEE Robotics & Automation Magazine - September 2021 - 30
IEEE Robotics & Automation Magazine - September 2021 - 31
IEEE Robotics & Automation Magazine - September 2021 - 32
IEEE Robotics & Automation Magazine - September 2021 - 33
IEEE Robotics & Automation Magazine - September 2021 - 34
IEEE Robotics & Automation Magazine - September 2021 - 35
IEEE Robotics & Automation Magazine - September 2021 - 36
IEEE Robotics & Automation Magazine - September 2021 - 37
IEEE Robotics & Automation Magazine - September 2021 - 38
IEEE Robotics & Automation Magazine - September 2021 - 39
IEEE Robotics & Automation Magazine - September 2021 - 40
IEEE Robotics & Automation Magazine - September 2021 - 41
IEEE Robotics & Automation Magazine - September 2021 - 42
IEEE Robotics & Automation Magazine - September 2021 - 43
IEEE Robotics & Automation Magazine - September 2021 - 44
IEEE Robotics & Automation Magazine - September 2021 - 45
IEEE Robotics & Automation Magazine - September 2021 - 46
IEEE Robotics & Automation Magazine - September 2021 - 47
IEEE Robotics & Automation Magazine - September 2021 - 48
IEEE Robotics & Automation Magazine - September 2021 - 49
IEEE Robotics & Automation Magazine - September 2021 - 50
IEEE Robotics & Automation Magazine - September 2021 - 51
IEEE Robotics & Automation Magazine - September 2021 - 52
IEEE Robotics & Automation Magazine - September 2021 - 53
IEEE Robotics & Automation Magazine - September 2021 - 54
IEEE Robotics & Automation Magazine - September 2021 - 55
IEEE Robotics & Automation Magazine - September 2021 - 56
IEEE Robotics & Automation Magazine - September 2021 - 57
IEEE Robotics & Automation Magazine - September 2021 - 58
IEEE Robotics & Automation Magazine - September 2021 - 59
IEEE Robotics & Automation Magazine - September 2021 - 60
IEEE Robotics & Automation Magazine - September 2021 - 61
IEEE Robotics & Automation Magazine - September 2021 - 62
IEEE Robotics & Automation Magazine - September 2021 - 63
IEEE Robotics & Automation Magazine - September 2021 - 64
IEEE Robotics & Automation Magazine - September 2021 - 65
IEEE Robotics & Automation Magazine - September 2021 - 66
IEEE Robotics & Automation Magazine - September 2021 - 67
IEEE Robotics & Automation Magazine - September 2021 - 68
IEEE Robotics & Automation Magazine - September 2021 - 69
IEEE Robotics & Automation Magazine - September 2021 - 70
IEEE Robotics & Automation Magazine - September 2021 - 71
IEEE Robotics & Automation Magazine - September 2021 - 72
IEEE Robotics & Automation Magazine - September 2021 - 73
IEEE Robotics & Automation Magazine - September 2021 - 74
IEEE Robotics & Automation Magazine - September 2021 - 75
IEEE Robotics & Automation Magazine - September 2021 - 76
IEEE Robotics & Automation Magazine - September 2021 - 77
IEEE Robotics & Automation Magazine - September 2021 - 78
IEEE Robotics & Automation Magazine - September 2021 - 79
IEEE Robotics & Automation Magazine - September 2021 - 80
IEEE Robotics & Automation Magazine - September 2021 - 81
IEEE Robotics & Automation Magazine - September 2021 - 82
IEEE Robotics & Automation Magazine - September 2021 - 83
IEEE Robotics & Automation Magazine - September 2021 - 84
IEEE Robotics & Automation Magazine - September 2021 - 85
IEEE Robotics & Automation Magazine - September 2021 - 86
IEEE Robotics & Automation Magazine - September 2021 - 87
IEEE Robotics & Automation Magazine - September 2021 - 88
IEEE Robotics & Automation Magazine - September 2021 - 89
IEEE Robotics & Automation Magazine - September 2021 - 90
IEEE Robotics & Automation Magazine - September 2021 - 91
IEEE Robotics & Automation Magazine - September 2021 - 92
IEEE Robotics & Automation Magazine - September 2021 - 93
IEEE Robotics & Automation Magazine - September 2021 - 94
IEEE Robotics & Automation Magazine - September 2021 - 95
IEEE Robotics & Automation Magazine - September 2021 - 96
IEEE Robotics & Automation Magazine - September 2021 - 97
IEEE Robotics & Automation Magazine - September 2021 - 98
IEEE Robotics & Automation Magazine - September 2021 - 99
IEEE Robotics & Automation Magazine - September 2021 - 100
IEEE Robotics & Automation Magazine - September 2021 - 101
IEEE Robotics & Automation Magazine - September 2021 - 102
IEEE Robotics & Automation Magazine - September 2021 - 103
IEEE Robotics & Automation Magazine - September 2021 - 104
IEEE Robotics & Automation Magazine - September 2021 - 105
IEEE Robotics & Automation Magazine - September 2021 - 106
IEEE Robotics & Automation Magazine - September 2021 - 107
IEEE Robotics & Automation Magazine - September 2021 - 108
IEEE Robotics & Automation Magazine - September 2021 - 109
IEEE Robotics & Automation Magazine - September 2021 - 110
IEEE Robotics & Automation Magazine - September 2021 - 111
IEEE Robotics & Automation Magazine - September 2021 - 112
IEEE Robotics & Automation Magazine - September 2021 - 113
IEEE Robotics & Automation Magazine - September 2021 - 114
IEEE Robotics & Automation Magazine - September 2021 - 115
IEEE Robotics & Automation Magazine - September 2021 - 116
IEEE Robotics & Automation Magazine - September 2021 - 117
IEEE Robotics & Automation Magazine - September 2021 - 118
IEEE Robotics & Automation Magazine - September 2021 - 119
IEEE Robotics & Automation Magazine - September 2021 - 120
IEEE Robotics & Automation Magazine - September 2021 - 121
IEEE Robotics & Automation Magazine - September 2021 - 122
IEEE Robotics & Automation Magazine - September 2021 - 123
IEEE Robotics & Automation Magazine - September 2021 - 124
IEEE Robotics & Automation Magazine - September 2021 - 125
IEEE Robotics & Automation Magazine - September 2021 - 126
IEEE Robotics & Automation Magazine - September 2021 - 127
IEEE Robotics & Automation Magazine - September 2021 - 128
IEEE Robotics & Automation Magazine - September 2021 - 129
IEEE Robotics & Automation Magazine - September 2021 - 130
IEEE Robotics & Automation Magazine - September 2021 - 131
IEEE Robotics & Automation Magazine - September 2021 - 132
IEEE Robotics & Automation Magazine - September 2021 - 133
IEEE Robotics & Automation Magazine - September 2021 - 134
IEEE Robotics & Automation Magazine - September 2021 - 135
IEEE Robotics & Automation Magazine - September 2021 - 136
IEEE Robotics & Automation Magazine - September 2021 - 137
IEEE Robotics & Automation Magazine - September 2021 - 138
IEEE Robotics & Automation Magazine - September 2021 - 139
IEEE Robotics & Automation Magazine - September 2021 - 140
IEEE Robotics & Automation Magazine - September 2021 - 141
IEEE Robotics & Automation Magazine - September 2021 - 142
IEEE Robotics & Automation Magazine - September 2021 - 143
IEEE Robotics & Automation Magazine - September 2021 - 144
IEEE Robotics & Automation Magazine - September 2021 - 145
IEEE Robotics & Automation Magazine - September 2021 - 146
IEEE Robotics & Automation Magazine - September 2021 - 147
IEEE Robotics & Automation Magazine - September 2021 - 148
IEEE Robotics & Automation Magazine - September 2021 - 149
IEEE Robotics & Automation Magazine - September 2021 - 150
IEEE Robotics & Automation Magazine - September 2021 - 151
IEEE Robotics & Automation Magazine - September 2021 - 152
IEEE Robotics & Automation Magazine - September 2021 - 153
IEEE Robotics & Automation Magazine - September 2021 - 154
IEEE Robotics & Automation Magazine - September 2021 - 155
IEEE Robotics & Automation Magazine - September 2021 - 156
IEEE Robotics & Automation Magazine - September 2021 - 157
IEEE Robotics & Automation Magazine - September 2021 - 158
IEEE Robotics & Automation Magazine - September 2021 - 159
IEEE Robotics & Automation Magazine - September 2021 - 160
IEEE Robotics & Automation Magazine - September 2021 - 161
IEEE Robotics & Automation Magazine - September 2021 - 162
IEEE Robotics & Automation Magazine - September 2021 - 163
IEEE Robotics & Automation Magazine - September 2021 - 164
IEEE Robotics & Automation Magazine - September 2021 - 165
IEEE Robotics & Automation Magazine - September 2021 - 166
IEEE Robotics & Automation Magazine - September 2021 - 167
IEEE Robotics & Automation Magazine - September 2021 - 168
IEEE Robotics & Automation Magazine - September 2021 - 169
IEEE Robotics & Automation Magazine - September 2021 - 170
IEEE Robotics & Automation Magazine - September 2021 - 171
IEEE Robotics & Automation Magazine - September 2021 - 172
IEEE Robotics & Automation Magazine - September 2021 - 173
IEEE Robotics & Automation Magazine - September 2021 - 174
IEEE Robotics & Automation Magazine - September 2021 - 175
IEEE Robotics & Automation Magazine - September 2021 - 176
IEEE Robotics & Automation Magazine - September 2021 - 177
IEEE Robotics & Automation Magazine - September 2021 - 178
IEEE Robotics & Automation Magazine - September 2021 - 179
IEEE Robotics & Automation Magazine - September 2021 - 180
IEEE Robotics & Automation Magazine - September 2021 - 181
IEEE Robotics & Automation Magazine - September 2021 - 182
IEEE Robotics & Automation Magazine - September 2021 - 183
IEEE Robotics & Automation Magazine - September 2021 - 184
IEEE Robotics & Automation Magazine - September 2021 - 185
IEEE Robotics & Automation Magazine - September 2021 - 186
IEEE Robotics & Automation Magazine - September 2021 - 187
IEEE Robotics & Automation Magazine - September 2021 - 188
IEEE Robotics & Automation Magazine - September 2021 - 189
IEEE Robotics & Automation Magazine - September 2021 - 190
IEEE Robotics & Automation Magazine - September 2021 - 191
IEEE Robotics & Automation Magazine - September 2021 - 192
IEEE Robotics & Automation Magazine - September 2021 - Cover3
IEEE Robotics & Automation Magazine - September 2021 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2010
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2010
https://www.nxtbookmedia.com