IEEE Robotics & Automation Magazine - June 2021 - 56

generator. This was particularly true for lightweight recyclables.
However, when we considered heavy items (e.g., 2-kg
PET bottles full of water), the blower was unable to grab the
objects. In contrast, the Venturi solution was much more successful
because of its ability to form an effective seal. The two
systems produced similar results in the case of 1-kg PET bottles,
but the blower outperformed the Venturi when dealing
with 0.5-kg and empty bottles, which typically have unstructured
surfaces. After examining the results and considering
the fact that the vast majority of recyclables are lightweight,
we selected the blower.
Validation of Visual Categorization
Training Process
The development of a vision-based module capable of categorizing
recyclables into different material types was
accomplished using Mask R-CNN [15], which has been
successfully employed to tackle a wide range of object
identification and categorization tasks. Mask R-CNN is a
deep CNN that simultaneously predicts recyclable object
bounding boxes, masks, and material types. We used a
public Mask R-CNN implementation [18] that was trained
on the data sets in the " Vision-Based Material Categorization "
section, using 30% of the images for model learning
validation and the remaining 70% for model training. To
increase the image processing frame rate, we incorporated
ResNet-50 for feature extraction across entire photos.
Moreover, a region proposal network (RPN) was employed
to scan images in a sliding window manner to identify
areas that contained objects. The RPN employed a set of
boxes, called anchors, with predefined image locations and
scales (there were five scales and three aspect ratios in our
implementation) to figure out the size and location of an
object on the feature map. Specifying the correct anchor
span is critical for obtaining successful classification
results. To estimate recyclable objects' bounding boxes,
masks, and material types, dedicated subnetworks known
Table 2. The training parameter configuration.
Parameter
Value
BACKBONE
BATCH_SIZE
DETECTION_MIN_CONFIDENCE
FPN_CLASSIF_FC_LAYERS_SIZE
RPN_ANCHOR_RATIOS
RPN_ANCHOR_SCALES
RPN_ANCHOR_STRIDE
RPN_BBOX_STD_DEV
RPN_NMS_THRESHOLD
ResNet-50
1
0.85
1,024
[0.5, 1, 2]
(32, 64, 128, 256, 512)
1
[0.1 0.1 0.2 0.2]
0.7
RPN_TRAIN_ANCHORS_PER_IMAGE 256
●
●
as heads work on identified regions of interest to shape the
final output. After extensive experimentation with the validation
set, we identified the Mask R-CNN training parameters
that worked effectively. They are summarized in
Table 2.
Before using Mask R-CNN for recyclable identification
and classification, a learning process was applied to improve
the applicability of the deep neural network to the problem.
To bootstrap the learning process, a network pretrained on
the Common Objects in Context data set was used. This
approach, introduced in [19], speeds up learning and ensures
the minimum quality of the results in a reasonable training
time. Following this, only the Mask R-CNN heads are typically
trained for a problem, keeping the general object feature
extraction mechanism unchanged. To adapt Mask R-CNN to
the classification task, we developed and tested four customized
implementations: Net 1, Net 2, Net 3, and Net 4. They
were trained on a high-performance computing infrastructure
with TensorFlow GPU support.
Training Analysis
Mask R-CNN training is based on a complex loss function
that is calculated as the weighted sum of different partial
losses at every training state. The partial losses considered
in the current implementation are described in
the following.
●
mrcnn_bbox_loss corresponds to the success at localizing
the bounding boxes of objects that belong to a given
class. This loss metric is increased when the object categorization
is correct but the bounding box localization
is not precise.
●
mrcnn_class_loss describes the loss due to the improper
categorization of an object identified by the RPN. This
provides an indication of how well Mask R-CNN recognizes
each class of an identified object. This metric
is increased when an object is detected in an image
but misclassified.
●
mrcnn_mask_loss is the metric that summarizes the success
at implementing masks across a processed image that correspond
to the pixel area covered by the identified objects.
The training data set includes masks for all objects of interest
in input images, thus providing the ground truth for the
evaluation of masks predicted by Mask R-CNN.
rpn_bbox_loss corresponds to the localization accuracy of
the RPN, in other words, how well the RPN identifies
individual objects. High values indicate miscalculations
when an object is detected but the bounding box needs to
be corrected.
rpn_class_loss is an RPN performance metric that is
increased when an object is undetected at the final output
and decreased when the RPN successfully identifies
an item.
Performance Analysis
To assess the performance of each trained network, we used a
typical evaluation procedure that considered three metrics,
56 * IEEE ROBOTICS & AUTOMATION MAGAZINE * JUNE 2021

IEEE Robotics & Automation Magazine - June 2021

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2021

Contents
IEEE Robotics & Automation Magazine - June 2021 - Cover1
IEEE Robotics & Automation Magazine - June 2021 - Cover2
IEEE Robotics & Automation Magazine - June 2021 - Contents
IEEE Robotics & Automation Magazine - June 2021 - 2
IEEE Robotics & Automation Magazine - June 2021 - 3
IEEE Robotics & Automation Magazine - June 2021 - 4
IEEE Robotics & Automation Magazine - June 2021 - 5
IEEE Robotics & Automation Magazine - June 2021 - 6
IEEE Robotics & Automation Magazine - June 2021 - 7
IEEE Robotics & Automation Magazine - June 2021 - 8
IEEE Robotics & Automation Magazine - June 2021 - 9
IEEE Robotics & Automation Magazine - June 2021 - 10
IEEE Robotics & Automation Magazine - June 2021 - 11
IEEE Robotics & Automation Magazine - June 2021 - 12
IEEE Robotics & Automation Magazine - June 2021 - 13
IEEE Robotics & Automation Magazine - June 2021 - 14
IEEE Robotics & Automation Magazine - June 2021 - 15
IEEE Robotics & Automation Magazine - June 2021 - 16
IEEE Robotics & Automation Magazine - June 2021 - 17
IEEE Robotics & Automation Magazine - June 2021 - 18
IEEE Robotics & Automation Magazine - June 2021 - 19
IEEE Robotics & Automation Magazine - June 2021 - 20
IEEE Robotics & Automation Magazine - June 2021 - 21
IEEE Robotics & Automation Magazine - June 2021 - 22
IEEE Robotics & Automation Magazine - June 2021 - 23
IEEE Robotics & Automation Magazine - June 2021 - 24
IEEE Robotics & Automation Magazine - June 2021 - 25
IEEE Robotics & Automation Magazine - June 2021 - 26
IEEE Robotics & Automation Magazine - June 2021 - 27
IEEE Robotics & Automation Magazine - June 2021 - 28
IEEE Robotics & Automation Magazine - June 2021 - 29
IEEE Robotics & Automation Magazine - June 2021 - 30
IEEE Robotics & Automation Magazine - June 2021 - 31
IEEE Robotics & Automation Magazine - June 2021 - 32
IEEE Robotics & Automation Magazine - June 2021 - 33
IEEE Robotics & Automation Magazine - June 2021 - 34
IEEE Robotics & Automation Magazine - June 2021 - 35
IEEE Robotics & Automation Magazine - June 2021 - 36
IEEE Robotics & Automation Magazine - June 2021 - 37
IEEE Robotics & Automation Magazine - June 2021 - 38
IEEE Robotics & Automation Magazine - June 2021 - 39
IEEE Robotics & Automation Magazine - June 2021 - 40
IEEE Robotics & Automation Magazine - June 2021 - 41
IEEE Robotics & Automation Magazine - June 2021 - 42
IEEE Robotics & Automation Magazine - June 2021 - 43
IEEE Robotics & Automation Magazine - June 2021 - 44
IEEE Robotics & Automation Magazine - June 2021 - 45
IEEE Robotics & Automation Magazine - June 2021 - 46
IEEE Robotics & Automation Magazine - June 2021 - 47
IEEE Robotics & Automation Magazine - June 2021 - 48
IEEE Robotics & Automation Magazine - June 2021 - 49
IEEE Robotics & Automation Magazine - June 2021 - 50
IEEE Robotics & Automation Magazine - June 2021 - 51
IEEE Robotics & Automation Magazine - June 2021 - 52
IEEE Robotics & Automation Magazine - June 2021 - 53
IEEE Robotics & Automation Magazine - June 2021 - 54
IEEE Robotics & Automation Magazine - June 2021 - 55
IEEE Robotics & Automation Magazine - June 2021 - 56
IEEE Robotics & Automation Magazine - June 2021 - 57
IEEE Robotics & Automation Magazine - June 2021 - 58
IEEE Robotics & Automation Magazine - June 2021 - 59
IEEE Robotics & Automation Magazine - June 2021 - 60
IEEE Robotics & Automation Magazine - June 2021 - 61
IEEE Robotics & Automation Magazine - June 2021 - 62
IEEE Robotics & Automation Magazine - June 2021 - 63
IEEE Robotics & Automation Magazine - June 2021 - 64
IEEE Robotics & Automation Magazine - June 2021 - 65
IEEE Robotics & Automation Magazine - June 2021 - 66
IEEE Robotics & Automation Magazine - June 2021 - 67
IEEE Robotics & Automation Magazine - June 2021 - 68
IEEE Robotics & Automation Magazine - June 2021 - 69
IEEE Robotics & Automation Magazine - June 2021 - 70
IEEE Robotics & Automation Magazine - June 2021 - 71
IEEE Robotics & Automation Magazine - June 2021 - 72
IEEE Robotics & Automation Magazine - June 2021 - 73
IEEE Robotics & Automation Magazine - June 2021 - 74
IEEE Robotics & Automation Magazine - June 2021 - 75
IEEE Robotics & Automation Magazine - June 2021 - 76
IEEE Robotics & Automation Magazine - June 2021 - 77
IEEE Robotics & Automation Magazine - June 2021 - 78
IEEE Robotics & Automation Magazine - June 2021 - 79
IEEE Robotics & Automation Magazine - June 2021 - 80
IEEE Robotics & Automation Magazine - June 2021 - 81
IEEE Robotics & Automation Magazine - June 2021 - 82
IEEE Robotics & Automation Magazine - June 2021 - 83
IEEE Robotics & Automation Magazine - June 2021 - 84
IEEE Robotics & Automation Magazine - June 2021 - 85
IEEE Robotics & Automation Magazine - June 2021 - 86
IEEE Robotics & Automation Magazine - June 2021 - 87
IEEE Robotics & Automation Magazine - June 2021 - 88
IEEE Robotics & Automation Magazine - June 2021 - 89
IEEE Robotics & Automation Magazine - June 2021 - 90
IEEE Robotics & Automation Magazine - June 2021 - 91
IEEE Robotics & Automation Magazine - June 2021 - 92
IEEE Robotics & Automation Magazine - June 2021 - 93
IEEE Robotics & Automation Magazine - June 2021 - 94
IEEE Robotics & Automation Magazine - June 2021 - 95
IEEE Robotics & Automation Magazine - June 2021 - 96
IEEE Robotics & Automation Magazine - June 2021 - 97
IEEE Robotics & Automation Magazine - June 2021 - 98
IEEE Robotics & Automation Magazine - June 2021 - 99
IEEE Robotics & Automation Magazine - June 2021 - 100
IEEE Robotics & Automation Magazine - June 2021 - 101
IEEE Robotics & Automation Magazine - June 2021 - 102
IEEE Robotics & Automation Magazine - June 2021 - 103
IEEE Robotics & Automation Magazine - June 2021 - 104
IEEE Robotics & Automation Magazine - June 2021 - 105
IEEE Robotics & Automation Magazine - June 2021 - 106
IEEE Robotics & Automation Magazine - June 2021 - 107
IEEE Robotics & Automation Magazine - June 2021 - 108
IEEE Robotics & Automation Magazine - June 2021 - 109
IEEE Robotics & Automation Magazine - June 2021 - 110
IEEE Robotics & Automation Magazine - June 2021 - 111
IEEE Robotics & Automation Magazine - June 2021 - 112
IEEE Robotics & Automation Magazine - June 2021 - 113
IEEE Robotics & Automation Magazine - June 2021 - 114
IEEE Robotics & Automation Magazine - June 2021 - 115
IEEE Robotics & Automation Magazine - June 2021 - 116
IEEE Robotics & Automation Magazine - June 2021 - 117
IEEE Robotics & Automation Magazine - June 2021 - 118
IEEE Robotics & Automation Magazine - June 2021 - 119
IEEE Robotics & Automation Magazine - June 2021 - 120
IEEE Robotics & Automation Magazine - June 2021 - 121
IEEE Robotics & Automation Magazine - June 2021 - 122
IEEE Robotics & Automation Magazine - June 2021 - 123
IEEE Robotics & Automation Magazine - June 2021 - 124
IEEE Robotics & Automation Magazine - June 2021 - 125
IEEE Robotics & Automation Magazine - June 2021 - 126
IEEE Robotics & Automation Magazine - June 2021 - 127
IEEE Robotics & Automation Magazine - June 2021 - 128
IEEE Robotics & Automation Magazine - June 2021 - 129
IEEE Robotics & Automation Magazine - June 2021 - 130
IEEE Robotics & Automation Magazine - June 2021 - 131
IEEE Robotics & Automation Magazine - June 2021 - 132
IEEE Robotics & Automation Magazine - June 2021 - 133
IEEE Robotics & Automation Magazine - June 2021 - 134
IEEE Robotics & Automation Magazine - June 2021 - 135
IEEE Robotics & Automation Magazine - June 2021 - 136
IEEE Robotics & Automation Magazine - June 2021 - 137
IEEE Robotics & Automation Magazine - June 2021 - 138
IEEE Robotics & Automation Magazine - June 2021 - 139
IEEE Robotics & Automation Magazine - June 2021 - 140
IEEE Robotics & Automation Magazine - June 2021 - Cover3
IEEE Robotics & Automation Magazine - June 2021 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2010
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2010
https://www.nxtbookmedia.com