IEEE Robotics & Automation Magazine - June 2021 - 55

Data Acquisition
We followed the semiautomated procedure summarized earlier
to develop the data set for training Mask R-CNN. We
started by collecting 400 images for each of the material types
considered in the study, namely, aluminum [Figure 3(a), first
row], paper and cardboard [Figure 3(a), second row], polyethylene
terephthalate (PET) bottles [Figure 3(a), third row],
and nylon [Figure 3(a), fourth row]. Then, we followed
a manual procedure to collect red-green-blue images.
All images had the same size, i.e., 800 × 800 pixels, and
depicted a single recyclable against a black background. It is
noted that, to have a representative data set of recyclables
images, we also considered deformed and dirty materials.
These images were processed by following step one (mentioned
previously) to identify masks describing regions of the
identified objects. We called this the Basic data set. Then, we
applied step two (from the preceding) to gather 2,000 images
of each material type. We called this the Synthetic Single data
set. Finally, following the third processing step, we developed
a data set that artificially described complex cases of multiple
and overlapping objects. This was the Synthetic Complex
data set. In particular, we developed 16,000 images for the
training set and 5,600 for the validation set. Figure 4 includes
four random images from the data set that have been successfully
created and annotated following the data augmentation
process.
Experimental Results
The robotic module and the material categorization module
are integrated into a composite system that is deployed and
validated in the industrial research setup summarized in the
" Industrial Research Setup " section. The two modules run
in full parallel, with the visual detection module adding
object locations to the list of potential targets managed by
the robot.
Validation of Recyclable Robotic Manipulation
To assess the success of the two alternative vacuum gripping
systems, i.e., the Venturi generator and the blower, we examined
their ability to pick up 80 representative recyclables
(20 for each class) that consisted of aluminum cans in flat,
crumpled, and cylindrical shapes; paper and nylon in flat,
crumpled, and ball shapes; and PET bottles in flat, arbitrarily
crumpled, and cylindrical shapes, some of which contained
liquid. Indicative object shapes appear in Figure 3(a). The
Table 1. The PnP performance evaluation (%).
Approach
Venturi
Pump
Pick success
Transfer fail
Pick success
Transfer fail
Aluminum
68
82
experimental procedure ran in a fully automated mode
through five cycles using the composite set of 80 representative
recyclables (400 picks in total). We used the Mask
R-CNN recyclable identification module (see the following)
to identify the location of objects on the moving conveyor
belt. Table 1 summarizes the results, documenting the ability
of each vacuum system to pick up and transfer items (successful
and unsuccessful trials, respectively). As shown in the
table, in most cases, the blower outperformed the Venturi
Height ×
Width = 800 × 800 Paper/Cardboard
Aluminum
Height ×
Width = 800 × 800
Bottle
Aluminum
Height ×
Width = 800 × 800
Nylon
Aluminum
Height ×
Width = 800 × 800
Nylon
Paper/Cardboard
Figure 4. Four samples from the training data set presented with
annotated masks. The first column contains original images. The
second and the third columns show each material mask and its
class identification as a subtitle.
Paper/ Cardboard
85
1
96
Bottle
81
3
84
4
Nylon
98
99
Average
83
1
90.2
1
JUNE 2021 * IEEE ROBOTICS & AUTOMATION MAGAZINE *
55
IEEE Robotics & Automation Magazine - June 2021

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2021
