IEEE Robotics & Automation Magazine - September 2021 - 31

objective is to put first-generation greenhouse harvesting
robots on the market.
UAVs are increasingly used in many agricultural robotics
applications, e.g., for tree 3D reconstruction and canopy estimation
[6], fruit counting [7], yield estimation [8], and automated
monitoring using lightweight devices [9].
On the industry side, several start-ups have been initiated,
and many more are expected to be funded. The major
services provided are UGVs for weed removal [10]-[12] and
in-season data analytics or early pest and disease detection
from aerial or satellite imagery.
Experimental Platforms
The Flourish project exploited existing state-of-the-art
farming and aerial robots, extending them in various
ways to improve both autonomous navigation and
environment modeling capabilities and enable them to
perform robust plant classification and/or selective weedremoval
operations.
Multirotor Used in the Flourish Project
The main UAV platform in the project is a fully sensorized
DJI Matrice 100 [see Figure 2(a) and (b)]. The platform
includes an Intel NUC i7 computer for onboard processing,
a NVIDIA TX2 GPU for real-time weed detection, a
GPS module, and a visual-inertial (VI) system for egomotion
estimation. We employed a VI sensor developed at the
Autonomous Systems Lab [13] and also tested and integrated
a commercially available sensor, the Intel ZR300,
for wider usage.
Ground Vehicle
The BoniRob Farming Robot
Bosch Deepfield Robotics' BoniRob [see Figure 2(c)] is a
flexible research platform for agricultural robotics. Its four
wheels can be independently rotated around the vertical
axis-resulting in omnidirectional driving capabilities-
and are mounted at the end of lever arms, letting the robot
adjust its track width from 1 to 2 m. To execute complex
tasks, the BoniRob carries a multitude of sensors: GPS,
real-time kinematic (RTK) GPS, a push broom lidar, two
omnidirectional lidars, red-green-blue (RGB) cameras, a
VI system, hyperspectral cameras, wheel odometers, and
so forth. These sensors are directly connected to a set of
onboard PCs that run the Robot Operating System (ROS)
and communicate through an internal network. The BoniRob's
batteries are complemented by a backup generator
that facilitates long-term field application.
Weed-Intervention Module
Supporting the target use case of selective weed intervention,
the robot is equipped with an extension module, the
weed-intervention module (Figure 3), which consists of a
perception system for weed classification, multimodal
actuation systems, and their supporting aggregates.
The main design objectives of this unit are high weed
throughput, precise treatment, and flexibility. The weeds
are treated mechanically with two ranks of stampers or
chemically with one rank of sprayers. The weeds are
detected and tracked in real time using three cameras with
nonoverlapping fields of view (FoV).
The weed unit's perception system consists of three
ground-facing global shutter cameras and three narrowbeam
sonars. To protect this perception system from natural
light sources, the weed-control unit is covered, and
artificial lights are installed to control the illumination. A
first RGB + near-infrared response camera is used for
weed detection and tracking, while the other two RGB
cameras are used for tracking. The sonars help recover the
absolute scale of the camera images. Further details about
the weed-intervention module can be found in the " Selective
Weed Removal " section.
Data Analysis and Interpretation
in a Farming Scenario
Precision farming applications aim to improve farm productivity
while reducing the use of fertilizers, herbicides,
and pesticides. To meet these challenges, in-field measurements
of plant-vitality indicators and weed density are
required. We addressed both of these requirements from a
" robotic " point of view by proposing a set of methods to 1)
accurately detect plants and classify them as crops and
weeds (see the " Crop and Weed Detection " and " Automatic
Synthetic Data Set Generation " sections) and 2) automatically
analyze the N status of crops from multispectral aerial
images (see the " Multispectral N-Status Detection and
Phenotyping " section).
Crop and Weed Detection
A prerequisite for selective and plant-specific treatments
with farming robots is an effective plant-classification system
that provides the robot with information on where
and when to trigger its actuators to perform the desired
action in real time.
In the Flourish project, we focus on vision-based
approaches for plant classification and use machine learning
techniques to effectively cope with the large variety of
different crops and weeds as well as with changing environmental
conditions. Figure 4(a)-(c) illustrates the
results obtained by our plant-classification systems for
both the UGV and UAV platforms. The further distinction
between weeds and grass-like weeds allows our system
to perform different treatments in a targeted manner
depending on the type of weed. For example, local
mechanical treatments are most effective when applied to
the stem location of the plants. In contrast, grass-like
weeds can be effectively treated by spraying herbicides on
their leaf surfaces.
We developed several data-driven plant-classification
systems, ranging from approaches based on handcrafted
features and random forests [14], [15] to deep learning
SEPTEMBER 2021 * IEEE ROBOTICS & AUTOMATION MAGAZINE *
31

IEEE Robotics & Automation Magazine - September 2021

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2021

Contents
IEEE Robotics & Automation Magazine - September 2021 - Cover1
IEEE Robotics & Automation Magazine - September 2021 - Cover2
IEEE Robotics & Automation Magazine - September 2021 - Contents
IEEE Robotics & Automation Magazine - September 2021 - 2
IEEE Robotics & Automation Magazine - September 2021 - 3
IEEE Robotics & Automation Magazine - September 2021 - 4
IEEE Robotics & Automation Magazine - September 2021 - 5
IEEE Robotics & Automation Magazine - September 2021 - 6
IEEE Robotics & Automation Magazine - September 2021 - 7
IEEE Robotics & Automation Magazine - September 2021 - 8
IEEE Robotics & Automation Magazine - September 2021 - 9
IEEE Robotics & Automation Magazine - September 2021 - 10
IEEE Robotics & Automation Magazine - September 2021 - 11
IEEE Robotics & Automation Magazine - September 2021 - 12
IEEE Robotics & Automation Magazine - September 2021 - 13
IEEE Robotics & Automation Magazine - September 2021 - 14
IEEE Robotics & Automation Magazine - September 2021 - 15
IEEE Robotics & Automation Magazine - September 2021 - 16
IEEE Robotics & Automation Magazine - September 2021 - 17
IEEE Robotics & Automation Magazine - September 2021 - 18
IEEE Robotics & Automation Magazine - September 2021 - 19
IEEE Robotics & Automation Magazine - September 2021 - 20
IEEE Robotics & Automation Magazine - September 2021 - 21
IEEE Robotics & Automation Magazine - September 2021 - 22
IEEE Robotics & Automation Magazine - September 2021 - 23
IEEE Robotics & Automation Magazine - September 2021 - 24
IEEE Robotics & Automation Magazine - September 2021 - 25
IEEE Robotics & Automation Magazine - September 2021 - 26
IEEE Robotics & Automation Magazine - September 2021 - 27
IEEE Robotics & Automation Magazine - September 2021 - 28
IEEE Robotics & Automation Magazine - September 2021 - 29
IEEE Robotics & Automation Magazine - September 2021 - 30
IEEE Robotics & Automation Magazine - September 2021 - 31
IEEE Robotics & Automation Magazine - September 2021 - 32
IEEE Robotics & Automation Magazine - September 2021 - 33
IEEE Robotics & Automation Magazine - September 2021 - 34
IEEE Robotics & Automation Magazine - September 2021 - 35
IEEE Robotics & Automation Magazine - September 2021 - 36
IEEE Robotics & Automation Magazine - September 2021 - 37
IEEE Robotics & Automation Magazine - September 2021 - 38
IEEE Robotics & Automation Magazine - September 2021 - 39
IEEE Robotics & Automation Magazine - September 2021 - 40
IEEE Robotics & Automation Magazine - September 2021 - 41
IEEE Robotics & Automation Magazine - September 2021 - 42
IEEE Robotics & Automation Magazine - September 2021 - 43
IEEE Robotics & Automation Magazine - September 2021 - 44
IEEE Robotics & Automation Magazine - September 2021 - 45
IEEE Robotics & Automation Magazine - September 2021 - 46
IEEE Robotics & Automation Magazine - September 2021 - 47
IEEE Robotics & Automation Magazine - September 2021 - 48
IEEE Robotics & Automation Magazine - September 2021 - 49
IEEE Robotics & Automation Magazine - September 2021 - 50
IEEE Robotics & Automation Magazine - September 2021 - 51
IEEE Robotics & Automation Magazine - September 2021 - 52
IEEE Robotics & Automation Magazine - September 2021 - 53
IEEE Robotics & Automation Magazine - September 2021 - 54
IEEE Robotics & Automation Magazine - September 2021 - 55
IEEE Robotics & Automation Magazine - September 2021 - 56
IEEE Robotics & Automation Magazine - September 2021 - 57
IEEE Robotics & Automation Magazine - September 2021 - 58
IEEE Robotics & Automation Magazine - September 2021 - 59
IEEE Robotics & Automation Magazine - September 2021 - 60
IEEE Robotics & Automation Magazine - September 2021 - 61
IEEE Robotics & Automation Magazine - September 2021 - 62
IEEE Robotics & Automation Magazine - September 2021 - 63
IEEE Robotics & Automation Magazine - September 2021 - 64
IEEE Robotics & Automation Magazine - September 2021 - 65
IEEE Robotics & Automation Magazine - September 2021 - 66
IEEE Robotics & Automation Magazine - September 2021 - 67
IEEE Robotics & Automation Magazine - September 2021 - 68
IEEE Robotics & Automation Magazine - September 2021 - 69
IEEE Robotics & Automation Magazine - September 2021 - 70
IEEE Robotics & Automation Magazine - September 2021 - 71
IEEE Robotics & Automation Magazine - September 2021 - 72
IEEE Robotics & Automation Magazine - September 2021 - 73
IEEE Robotics & Automation Magazine - September 2021 - 74
IEEE Robotics & Automation Magazine - September 2021 - 75
IEEE Robotics & Automation Magazine - September 2021 - 76
IEEE Robotics & Automation Magazine - September 2021 - 77
IEEE Robotics & Automation Magazine - September 2021 - 78
IEEE Robotics & Automation Magazine - September 2021 - 79
IEEE Robotics & Automation Magazine - September 2021 - 80
IEEE Robotics & Automation Magazine - September 2021 - 81
IEEE Robotics & Automation Magazine - September 2021 - 82
IEEE Robotics & Automation Magazine - September 2021 - 83
IEEE Robotics & Automation Magazine - September 2021 - 84
IEEE Robotics & Automation Magazine - September 2021 - 85
IEEE Robotics & Automation Magazine - September 2021 - 86
IEEE Robotics & Automation Magazine - September 2021 - 87
IEEE Robotics & Automation Magazine - September 2021 - 88
IEEE Robotics & Automation Magazine - September 2021 - 89
IEEE Robotics & Automation Magazine - September 2021 - 90
IEEE Robotics & Automation Magazine - September 2021 - 91
IEEE Robotics & Automation Magazine - September 2021 - 92
IEEE Robotics & Automation Magazine - September 2021 - 93
IEEE Robotics & Automation Magazine - September 2021 - 94
IEEE Robotics & Automation Magazine - September 2021 - 95
IEEE Robotics & Automation Magazine - September 2021 - 96
IEEE Robotics & Automation Magazine - September 2021 - 97
IEEE Robotics & Automation Magazine - September 2021 - 98
IEEE Robotics & Automation Magazine - September 2021 - 99
IEEE Robotics & Automation Magazine - September 2021 - 100
IEEE Robotics & Automation Magazine - September 2021 - 101
IEEE Robotics & Automation Magazine - September 2021 - 102
IEEE Robotics & Automation Magazine - September 2021 - 103
IEEE Robotics & Automation Magazine - September 2021 - 104
IEEE Robotics & Automation Magazine - September 2021 - 105
IEEE Robotics & Automation Magazine - September 2021 - 106
IEEE Robotics & Automation Magazine - September 2021 - 107
IEEE Robotics & Automation Magazine - September 2021 - 108
IEEE Robotics & Automation Magazine - September 2021 - 109
IEEE Robotics & Automation Magazine - September 2021 - 110
IEEE Robotics & Automation Magazine - September 2021 - 111
IEEE Robotics & Automation Magazine - September 2021 - 112
IEEE Robotics & Automation Magazine - September 2021 - 113
IEEE Robotics & Automation Magazine - September 2021 - 114
IEEE Robotics & Automation Magazine - September 2021 - 115
IEEE Robotics & Automation Magazine - September 2021 - 116
IEEE Robotics & Automation Magazine - September 2021 - 117
IEEE Robotics & Automation Magazine - September 2021 - 118
IEEE Robotics & Automation Magazine - September 2021 - 119
IEEE Robotics & Automation Magazine - September 2021 - 120
IEEE Robotics & Automation Magazine - September 2021 - 121
IEEE Robotics & Automation Magazine - September 2021 - 122
IEEE Robotics & Automation Magazine - September 2021 - 123
IEEE Robotics & Automation Magazine - September 2021 - 124
IEEE Robotics & Automation Magazine - September 2021 - 125
IEEE Robotics & Automation Magazine - September 2021 - 126
IEEE Robotics & Automation Magazine - September 2021 - 127
IEEE Robotics & Automation Magazine - September 2021 - 128
IEEE Robotics & Automation Magazine - September 2021 - 129
IEEE Robotics & Automation Magazine - September 2021 - 130
IEEE Robotics & Automation Magazine - September 2021 - 131
IEEE Robotics & Automation Magazine - September 2021 - 132
IEEE Robotics & Automation Magazine - September 2021 - 133
IEEE Robotics & Automation Magazine - September 2021 - 134
IEEE Robotics & Automation Magazine - September 2021 - 135
IEEE Robotics & Automation Magazine - September 2021 - 136
IEEE Robotics & Automation Magazine - September 2021 - 137
IEEE Robotics & Automation Magazine - September 2021 - 138
IEEE Robotics & Automation Magazine - September 2021 - 139
IEEE Robotics & Automation Magazine - September 2021 - 140
IEEE Robotics & Automation Magazine - September 2021 - 141
IEEE Robotics & Automation Magazine - September 2021 - 142
IEEE Robotics & Automation Magazine - September 2021 - 143
IEEE Robotics & Automation Magazine - September 2021 - 144
IEEE Robotics & Automation Magazine - September 2021 - 145
IEEE Robotics & Automation Magazine - September 2021 - 146
IEEE Robotics & Automation Magazine - September 2021 - 147
IEEE Robotics & Automation Magazine - September 2021 - 148
IEEE Robotics & Automation Magazine - September 2021 - 149
IEEE Robotics & Automation Magazine - September 2021 - 150
IEEE Robotics & Automation Magazine - September 2021 - 151
IEEE Robotics & Automation Magazine - September 2021 - 152
IEEE Robotics & Automation Magazine - September 2021 - 153
IEEE Robotics & Automation Magazine - September 2021 - 154
IEEE Robotics & Automation Magazine - September 2021 - 155
IEEE Robotics & Automation Magazine - September 2021 - 156
IEEE Robotics & Automation Magazine - September 2021 - 157
IEEE Robotics & Automation Magazine - September 2021 - 158
IEEE Robotics & Automation Magazine - September 2021 - 159
IEEE Robotics & Automation Magazine - September 2021 - 160
IEEE Robotics & Automation Magazine - September 2021 - 161
IEEE Robotics & Automation Magazine - September 2021 - 162
IEEE Robotics & Automation Magazine - September 2021 - 163
IEEE Robotics & Automation Magazine - September 2021 - 164
IEEE Robotics & Automation Magazine - September 2021 - 165
IEEE Robotics & Automation Magazine - September 2021 - 166
IEEE Robotics & Automation Magazine - September 2021 - 167
IEEE Robotics & Automation Magazine - September 2021 - 168
IEEE Robotics & Automation Magazine - September 2021 - 169
IEEE Robotics & Automation Magazine - September 2021 - 170
IEEE Robotics & Automation Magazine - September 2021 - 171
IEEE Robotics & Automation Magazine - September 2021 - 172
IEEE Robotics & Automation Magazine - September 2021 - 173
IEEE Robotics & Automation Magazine - September 2021 - 174
IEEE Robotics & Automation Magazine - September 2021 - 175
IEEE Robotics & Automation Magazine - September 2021 - 176
IEEE Robotics & Automation Magazine - September 2021 - 177
IEEE Robotics & Automation Magazine - September 2021 - 178
IEEE Robotics & Automation Magazine - September 2021 - 179
IEEE Robotics & Automation Magazine - September 2021 - 180
IEEE Robotics & Automation Magazine - September 2021 - 181
IEEE Robotics & Automation Magazine - September 2021 - 182
IEEE Robotics & Automation Magazine - September 2021 - 183
IEEE Robotics & Automation Magazine - September 2021 - 184
IEEE Robotics & Automation Magazine - September 2021 - 185
IEEE Robotics & Automation Magazine - September 2021 - 186
IEEE Robotics & Automation Magazine - September 2021 - 187
IEEE Robotics & Automation Magazine - September 2021 - 188
IEEE Robotics & Automation Magazine - September 2021 - 189
IEEE Robotics & Automation Magazine - September 2021 - 190
IEEE Robotics & Automation Magazine - September 2021 - 191
IEEE Robotics & Automation Magazine - September 2021 - 192
IEEE Robotics & Automation Magazine - September 2021 - Cover3
IEEE Robotics & Automation Magazine - September 2021 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2010
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2010
https://www.nxtbookmedia.com