IEEE Robotics & Automation Magazine - December 2016 - 104

Figure 9. Snapshots from an unloading sequence in a challenging cluttered scenario. Frames 1-3: starting from a sensing pose, the robot
approaches, grasps, and extracts a parcel. Frames 4 and 5: successful grasps of a deformable teddy bear and a five-liter barrel.

worker, yet they pose significant challenges to an autonomous system due to the complexity of the scenes. Therefore, in this scenario, we did not aim to match the speed
of manual container unloading, but rather concentrated on
evaluating the robustness of the system to various objectstacking configurations. The system was evaluated on a
multitude of different container configurations, one of
which is shown in Figure 7(b). A video of a sample system
run is available on the YouTube channel RobLog (https://
youtu.be/34ZXK6L1ixY).
Test scenes for this scenario contained, on average,
between 20 and 25 objects from seven to nine different
object classes, five of which were modeled in the object
database, i.e., the scenes also contained a substantial
amount of unknown objects to be unloaded. On average,
the system was capable of successfully unloading 80% of
the target objects at a cycle time of 200.8 s (σ = 19.3 s,
median 198.9 s, N = 50), i.e., about 3.5 min. Most of the
unloading failures were due to failures in finding collisionfree grasping trajectories for objects placed in difficult
configurations, e.g., in
proximity to the container walls or tightly
Most likely, the main
packed with other
objects. Only a handful
reason for the success
of objects were not recognized by any of the
of the two technology
modules in the recognition pipeline, and, in
demonstrators is
very few cases, this problem persisted between
the modular system
successive cycles. Figure 8(b) shows the modarchitecture.
ule-wise break-down of
run times in an unloading cycle. Other than in
OR, the system also expended a lot of effort on motion
planning and subsequently on motion execution. One of
the reasons for the long execution times is that, in the interest of safety, the robot was operating at a reduced speed
(10% of the maximum drive capacity). The second and
often more prominent cause is linked to the poor quality of
the motion plans obtained for some of the trajectories.
Overall, the system performed within good time bounds
and success rates for a research prototype, and it did so
104

IEEE ROBOTICS & AUTOMATION MAGAZINE

December 2016

with virtually no damage to the unloaded goods. To the
best of our knowledge, no comprehensive results of a complete autonomous manipulation system, encompassing all
steps from perception through grasping and unloading in
similarly complex settings, have been reported previously.
To stress test the system and explore the boundaries of
the proposed approach, we also evaluated the performance
on a significantly more challenging cluttered configuration
of goods in the container, as shown in Figure 7(c).
A sequence of still shots from various stages of unloading
in this challenging setup are shown in Figure 9. Overall,
the system performance gracefully degrades without suffering from a significant failure. As there are many
objects in the scene, both recognition and grasp/motion
planning take significantly longer, increasing the average
cycle time to 313 s (σ = 63 s, median 300 s, N = 26). The
complete unloading of all objects in the container in this
scenario is not always possible, as the likelihood of
objects stacking in unreachable or collision-risky configurations increases significantly. As discussed in the next
section, this limitation is a strong argument in favor of
using compliant, force-limited robots for unloading tasks
in future applications.
Lessons Learned
In this section, we discuss some of the lessons learned over
the four years spent in designing, implementing, and testing
the proposed automation framework and the two demonstrators. From one perspective, the design decisions made
proved to be sound, as was evident by the performance on
challenging and realistic test scenarios. On the other hand,
over the course of development, it was evident that some of
the system components could not fully cope with the hard
requirements posed by the container-unloading problem.
Some of the design decisions that did benefit the demonstrators' performances were as follows:
● Modular approach: Most likely, the main reason for the
success of the two technology demonstrators is the
modular system architecture. In addition to enabling
easier integration between components, this approach
was critical in the fast migration of code between the
two hardware platforms, as well as the rapid deployment of new system components.
● Perception and manipulation: Another main beneficial
system feature was the synergy between perception

http://https:// http://www.youtu.be/34ZXK6L1ixY

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2016