IEEE Robotics & Automation Magazine - June 2019 - 104

Figure 1. Undergraduates work to attach a gripper arm to Misty.
(Source: Misty Robotics.)

code for robots is the hurdles robot
hardware often throws at them. Most
of these developers don't have the
knowledge or desire to get down into
the firmware and write low-level
driver code just to make the robot do
basic things. Misty definitely breaks
down this barrier by providing a fully
capable hardware stack that's pro-
grammable in JavaScript, C Sharp, or

Figure 3. A developer programs Misty.
(Source: Misty Robotics.)

Figure 4. Misty poses for photos at San
Francisco DeveloperWeek. (Source: Misty
Robotics.)

104

IEEE ROBOTICS & AUTOMATION MAGAZINE

JUNE 2019

Figure 2. A group of students explores Misty's hardwareexpandability packs. (Source: Misty Robotics.)

Python, along with well-documented
application programming interfaces
(APIs) and architectures that are
familiar to software developers. Com-
plicated features, like simultaneous
localization and mapping (SLAM),
audio localization, face recognition,
and more, are built right into the
robot's software. For those who are
more hardware-curious, Misty can
wear a backpack that supports uni-
versal asynchronous receiver/trans-
mitter communication (Figure 2).
This allows developers to extend
Misty's native capabilities by incorpo-
rating external hardware, like an
Arduino or Raspberry Pi, into their
work. This combination of advanced
capabilities, open software, and exten-
sible hardware makes Misty an ideal
robot for software developers who
want to innovate outside of mobile
and web applications.
Srinivasan: Could you please elaborate
on some of the things developers can
look forward to in Misty straight out of
the box?
Sridhar: Having worked with many
robots, my personal favorite feature is
Misty's programmable personality.
Developers can choose how they want
their Misty to look, sound, and be-
have, and they have the freedom to
alter how she responds to humans in a
variety of ways. This includes simple
things, like having Misty make an
angry face and say "ah" when someone
grips her handle or having her make a
happy face and a corresponding sound
when someone touches her cheek.

These types of seemingly basic
social interactions have never failed to
draw people toward Misty when she is
out and about in public. Other amaz-
ing features include mapping and
navigation, audio recording (and the
ability to know where those sounds
come from), video recording, face
detection and recognition, capacitive-
touch sensors, wireless charging, and
the ability to talk to any number of
cloud services that might be running
on Microsoft, Google, Amazon, IBM,
and so on. Additionally, developers
can hack Misty's physicality by creat-
ing new arms, headpieces, backpacks,
and other attachments for her. But
above everything else is the Misty
skill system that makes her extremely
easy to code. We've seen early devel-
opers build cool skills for Misty in
just 30 min (Figure 3).
Srinivasan: What are the features of
Misty that make her a good platform for
SAR development?
Sridhar: Given a specific focus on
SAR development, I would say audio
input to the robot is most valuable.
Developers can choose to pipe the
audio through a recurrent neural net-
works model, or they can use a service
to extract intent and act on this infor-
mation later to have Misty behave
accordingly. This could be used for
actions like toggling the state of a con-
nected device or making a phone call.
These scenarios can be enhanced
with SLAM, face recognition, and audio
localization-Misty can actually look at
you, know your name, and even follow

IEEE Robotics & Automation Magazine - June 2019

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2019