IEEE Robotics & Automation Magazine - June 2021 - 98

●
Grasping strategy failure: Any other type of failure occurred
during the grasping phase of the pick-and-place pipeline;
e.g., the hand might be unable to cage a tool, or a grasp is
not robust enough to lift the tool from the bench.
●
Placement strategy failure: Errors occurred during the
placement phase; e.g., the robots might drop tools outside
of the toolbox because of incorrect decisions on the placement
location or erroneous toolbox pose detection.
The number of times each of the failures occurred per hand
and object is presented in Figure 6, while the corresponding
success-based performance metrics are illustrated in Figure 7.
Both figures highlight the effect that object geometry has on
system performance. More specifically, it is apparent that big
objects like torches and cutters are always detected by the visual
system. This is not always the case for smaller tools like
brushes and screwdrivers. This is because bigger objects are
more likely to have features that persist through multiple pooling
layers of the deep neural network model, whereas smaller
visual features might be occasionally skipped.
This observation is also confirmed by the fact that attempt
rates for torches and cutters are higher compared to brushes and
screwdrivers (Figure 7). Regarding grasping strategy failures, we
observe that grasp success depends on hand morphology and,
more specifically, on the ability of a hand to restrict object displacement
while closing. As a consequence, the qb SoftHand performed
better for grasping very flat tools like the brush. This is
confirmed by the precision and success rate metrics for brushes
in Figure 7. Finally, a few placement failures occurred for long
objects due to unwanted object-toolbox collisions, which are not
mitigated by the force-repulsion component.
Table 6. The overall performance metrics for the
dual-arm system.
Metric
Task completion
score (max of 8)
MPPH
Mean
6.27
37.90
Standard
Deviation
1.56
4.25
Overall, the experimental results verify that the proposed
methodology leads to a well-integrated system. More specifically,
the absence of unexpected system errors and crashes
signifies successful high-level integration. Regarding low-level
integration, all of the system components were able to interoperate
with each other as needed and performed well
enough given the development stage of the evaluated system
and the challenging nature of the task.
Integration Time Prediction Evaluation
This section evaluates whether our integration methodology can
predict the integration time. Table 7 depicts the expected and
actual integration time for the components discussed in the " Systems
Integration Case Study " section. The expected integration
time was converted from the efforts in Tables 4 and 5 using Table
1 and equation (2). The actual integration time was derived from
the logs of the agile management software used [Jira (https://
www.atlassian.com/software/jira)]. The logs detail the start and
completion date of each integration task. The time reported
refers exclusively to the integration of the components, ignoring
the implementation time of the algorithms. Overall, our integration
methodology predicted the integration time reliably.
Discussion
In this article, we propose an integration methodology, summarized
by the workflow in Figure 3, that assesses the effort and
impact of integrating a robotic component in a complex system.
We showcase the application of our approach to an industrial case
study: a dual-arm manipulation system that consists of a diverse
set of research components. We use the system to demonstrate
how to practically apply our methodology and address real-world
integration problems. By following the flowchart (Figure 3), it is
possible to understand which requirements are key to facilitate the
adoption of academic research into industrial systems.
We evaluate the performance of the system with a benchTorch
4
5
6
7
1
2
3
1
1
2
1
Cutter
Brush
Screwdriver
Visual Detection Failures
Grasping Strategy Failures
Placement Strategy Failures
1
3
2
1
21
3
2
2
mark designed to measure the robustness, reliability, and operational
speed of the whole system as well as the relative
effectiveness of the two employed end effectors when integrated
within the same system. Our integration methodology was
employed alongside the agile scrum [13] development methodology
since it helps in estimating the time required for the
integration of a component. This is a key factor in every agile
methodology, where time is expected to be correctly estimated
by the developers. Nevertheless, our methodology is agnostic
of the development methodology adopted by a team.
System integration is a topic rarely addressed by the robotics
Figure 6. The number of system failures per hand and tool.
98 * IEEE ROBOTICS & AUTOMATION MAGAZINE * JUNE 2021
research community; it is considered an overhead rather than
an important part of a system. However, a robotic system needs
to be integrated in a coherent structure to be applied to a realworld
use case. Additionally, valuable scientific contributions
might be lost if it is unfeasible to integrate them. For a successful
translation of robotic research into industrial applications,
the research community needs to apply established integration
methods. Moreover, the integration methodology has to be
grounded by the development of real-world robotic systems
with a high technology-readiness level.
Total Number of Failures
qb
ARMAR-6
qb
ARMAR-6
qb
ARMAR-6
qb
ARMAR-6
http://www.atlassian.com/software/jira http://www.atlassian.com/software/jira
IEEE Robotics & Automation Magazine - June 2021

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2021
