IEEE Robotics & Automation Magazine - March 2016 - 35

the problem of generating motor control behaviors for legged
robotic systems remains a very active area of research.
To perform agile motions, especially those that might include
airborne phases, a robot needs to be equipped with sufficiently
powerful actuators. Fortunately, the development of high-performance drives for legged robots has seen significant progress in recent years. For instance, hydraulic systems, such as Boston
Dynamic's quadrupeds [1] (BidDog, AlphaDog, and WildCat)
and Italian Institute of Technology's HyQ [2], feature great performance and enable dynamic gaits, such as the flying trot, bound,
and gallop. Likewise, research on electric actuators contributed to
significant improvements in the motor skills of legged robots over
the past few years. The Massachusetts Institute of Technology
(MIT) Cheetah [3], for instance, uses electric motors with low
gear reduction, and is skilled enough to bound over obstacles.
Other systems employ biologically inspired actuators that make
use of compliant elements, inspired by the fact that humans and
animals leverage the elastic nature of their musculoskeletal structures [4]. By exploiting the compliance of the actuation system, the
power and velocity output of electric motors can be amplified, as
we showed for ScarlETH, a mechanical leg powered by serieselastic actuators (SEAs) [5] (Figure 1). A number of robots, including Marc Raibert's early machines [6], achieve dynamic
maneuvers thanks to the compliance of the system.
As SEAs combine many features essential for legged robots, such as torque control, lightweight design, and robustness against impacts, we built a fully articulated quadruped
robot, StarlETH [7], with highly compliant SEAs. To increase
StarlETH's repertoire of motion skills, we are interested in developing flexible control strategies for various agile maneuvers, including running and jumping. One important main
goal of our control strategy is to take the full advantage of the
actuators' compliance.

Raibert's seminal work [6] on monopod hoppers with telescopic legs showed that a simple collection of control rules is
applicable to a large set of dynamic motions and robots. Inspired by nature, contact force profiles were manually designed
to enable the MIT Cheetah robot [3] to bound over obstacles.
Alternatively, optimization algorithms can be used to generate
various locomotion tasks by automatically finding appropriate
values for parameterized control policies. Such direct policy
search methods have been used, for instance, to generate galloping motions in simulation [8], to control muscle activations
for simulated bipeds [9], to stabilize a planar bipedal robot
[10], and to generate leaping motions for the wheeled quadruped robot PAW [11]. In previous work, we also applied direct
policy search methods for motion synthesis. In particular, we
employed PI2 and ROCK* [12] to generate jumping and hopping maneuvers for our single-legged robot, ScarlETH.
Our work shows that policy search can be successfully applied to complex dynamical systems with a high-dimensional
state and action space, enabling agile maneuvers. Our work is
based on a generic set of motion primitives encoded through
cost functions. Inspired by motor learning principles observed in nature, the cost functions are optimized through a
trial-and-error process that requires the repeated execution of
slight variations of a motion skill. Since executing these practice runs is time-consuming and could possibly damage the
hardware, our goal is to utilize the policy search method in
simulation only. However, to ensure that the optimized control policies also work on the hardware platform, we must
bridge the gap between the simulation and the real world.
Consequently, an accurate model of the robot is essential. In
our previous work [13], we modeled the rigid-body dynamics
of the quadruped, but neglected the SEAs. For highly agile
motions, such as jumping, the actuator dynamics start to play
Hip Ad-/Abduction
Hip Flexion/Extension
(HFE)

Knee Flexion/Extension (KFE)
Chain Drive
Cable
Pulley
Bump Stop
Deflection
Encoder
Precompressed
Knee Spring
Figure 1. A quadruped robot StarlETH, equipped with SEAs, weighing 28 kg with a total leg length of 0.5 m. (Photo courtesy of
François Pomerleau.)

march 2016

IEEE ROBOTICS & AUTOMATION MAGAZINE

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - March 2016