Tech Briefs Magazine - June 2022 - MD-19

Their applications include fabrication
of structures on orbit or on the lunar
surface including habitats, power
trusses, science equipment, telescope
supports, refueling stations, aeroshells,
antennae, gravity-generator facilities, or
solar arrays, as well as advanced industrial
manufacturing and assembly.
NASA is actively seeking licensees to comtechnology.
Please contact
mercialize this
NASA's Licensing Concierge at AgencyPatent-Licensing@mail.nasa.gov
or call at
202-358-7432 to initiate licensing discussions.
For more information, visit https://
technology.nasa.gov/patent/LAR-TOPS-345.
An Artificial Intelligence System to Identify Objects in
Cluttered Scenes
The technology enables detecting visible, partially occluded, and unseen objects in a single framework.
Gwangju Institute of Science and Technology, South Korea
R
obotic vision has come a long way,
reaching a level of sophistication with
applications in complex and demanding
tasks, such as autonomous driving and
object manipulation. However, it still
struggles to identify individual objects in
cluttered scenes where some objects are
partially or completely hidden behind
others. Typically, when dealing with such
scenes, robotic vision systems are trained
to identify the occluded object based
only on its visible parts. But such training
requires large datasets of objects and
can be tedious.
Associate Professor Kyoobin Lee and
Ph.D. student Seunghyeok Back from the
Gwangju Institute of Science and
Technology in South Korea found themselves
facing this problem when they were
developing an artificial intelligence system
to identify and sort objects in cluttered
scenes. " We expect a
robot to recognize and manipulate
objects they have
not encountered before or
been trained to recognize.
In reality, however, we need
to manually collect and label
data one by one as the generalizability
of deep neural
networks depends highly on
the quality and quantity of
the training dataset, " said
Back.
In a new study,
led by
Prof. Lee and Back, they developed
a model called " unseen
object amodal instance
segmentation " (UOAIS) for
detecting occluded objects
in cluttered scenes. To train
the model in identifying object
geometry, they developed
a database containing
Motion Design, June 2022
45,000 photorealistic synthetic images
containing depth information. With this
(limited) training data, the model was
able to detect a variety of occluded objects.
Upon encountering a cluttered
scene, it first picked out the object of interest
and then determines if the object is
occluded by segmenting the object into a
" visible mask " and an " amodal mask. "
The researchers were excited by the
results. " Previous methods are limited
to either detecting only specific types of
objects or detecting only the visible regions
without explicitly reasoning over
occluded areas. By contrast, our method
can infer the hidden regions of occluded
objects like a human vision system.
This enables a reduction in data
collection efforts while improving performance
in a complex environment, "
said Back.
Artificial Intelligence for
Amodal Identification of
Occluded Objects
Segmentation of occluded objects is a
challenging task for robotic vision systems
A cluttered scene
with objects
Partially hidden
or occluded object
While methods have been
developed to include occluded
objects, they only involve perceiving
the visible parts of objects
Can robotic vision systems be
trained for amodal perception?
To enable " occlusion reasoning " in
their system, the researchers introduced
a " hierarchical occlusion modeling "
(HOM) scheme, which assigned a hierarchy
to the combination of multiple
extracted features and their prediction
order. By testing their model against
three benchmarks, they validated the
effectiveness of the HOM scheme, which
achieved state-of-the-art performance.
The researchers are hopeful about the
future prospects of their method.
" Perceiving unseen objects in a cluttered
environment is essential for amodal robotic
manipulation. Our UOAIS method
could serve as a baseline on this front, "
said Back.
For more information, contact Seulhye
Kim, Gwangju Institute of Science and
Technology, at seulhyekim@gist.ac.kr; 82627-156-253.
Unseen
object amodal instance segmentation (UOAIS)
Occlusion prediction + Object segmentation
Feature extraction
(RGB + depth)
Object of interest
bounding box
Visible mask
Amodal mask
RGB-D fusion backbone
(visible + occluded mask)
Occlusion
Hierarchical occlusion modeling + Hierarchical fusion
Model trained with 45,000
synthetic RGB-D images
Joint detection of visible and
amodal regions
State-of-the-art performance
in UOASIS tasks
Reduced training data and time
needed
The developed UOAIS system can segment both visible and occluded regions of
unseen objects, making it ideal for applications in amodal robotic manipulation
Unseen Object Amodal Instance Segmentation
via Hierarchiacal Occlusuin Modeling
Back et al. (2022)
IEEE International Conference on Robotics and Automation l DOI: 10-48550/arXiv. 2109.11103
Much like human vision, the method allows for the detection of visible, partially occluded, and unseen objects in a single
framework. (Image: Gwangju Institute of Science and Technology)
19
MD Tech Briefs 0622_2.indd 19
Cov
ToC
5/18/22 10:17 AM
https://technology.nasa.gov/patent/LAR-TOPS-345 http://info.hotims.com/82322-850

Tech Briefs Magazine - June 2022

Table of Contents for the Digital Edition of Tech Briefs Magazine - June 2022

Tech Briefs Magazine - June 2022 - Intro
Tech Briefs Magazine - June 2022 - Sponsor
Tech Briefs Magazine - June 2022 - Band1
Tech Briefs Magazine - June 2022 - Band2
Tech Briefs Magazine - June 2022 - Cov1
Tech Briefs Magazine - June 2022 - Cov2
Tech Briefs Magazine - June 2022 - 1
Tech Briefs Magazine - June 2022 - 2
Tech Briefs Magazine - June 2022 - 3
Tech Briefs Magazine - June 2022 - 4
Tech Briefs Magazine - June 2022 - 5
Tech Briefs Magazine - June 2022 - 6
Tech Briefs Magazine - June 2022 - 7
Tech Briefs Magazine - June 2022 - 8
Tech Briefs Magazine - June 2022 - 9
Tech Briefs Magazine - June 2022 - 10
Tech Briefs Magazine - June 2022 - 11
Tech Briefs Magazine - June 2022 - 12
Tech Briefs Magazine - June 2022 - 13
Tech Briefs Magazine - June 2022 - 14
Tech Briefs Magazine - June 2022 - 15
Tech Briefs Magazine - June 2022 - 16
Tech Briefs Magazine - June 2022 - 17
Tech Briefs Magazine - June 2022 - 18
Tech Briefs Magazine - June 2022 - 19
Tech Briefs Magazine - June 2022 - 20
Tech Briefs Magazine - June 2022 - 21
Tech Briefs Magazine - June 2022 - 22
Tech Briefs Magazine - June 2022 - 23
Tech Briefs Magazine - June 2022 - 24
Tech Briefs Magazine - June 2022 - 25
Tech Briefs Magazine - June 2022 - 26
Tech Briefs Magazine - June 2022 - 27
Tech Briefs Magazine - June 2022 - 28
Tech Briefs Magazine - June 2022 - 29
Tech Briefs Magazine - June 2022 - 30
Tech Briefs Magazine - June 2022 - 31
Tech Briefs Magazine - June 2022 - 32
Tech Briefs Magazine - June 2022 - 33
Tech Briefs Magazine - June 2022 - 34
Tech Briefs Magazine - June 2022 - 35
Tech Briefs Magazine - June 2022 - 36
Tech Briefs Magazine - June 2022 - 37
Tech Briefs Magazine - June 2022 - 38
Tech Briefs Magazine - June 2022 - 39
Tech Briefs Magazine - June 2022 - 40
Tech Briefs Magazine - June 2022 - 41
Tech Briefs Magazine - June 2022 - 42
Tech Briefs Magazine - June 2022 - 43
Tech Briefs Magazine - June 2022 - 44
Tech Briefs Magazine - June 2022 - 45
Tech Briefs Magazine - June 2022 - 46
Tech Briefs Magazine - June 2022 - 47
Tech Briefs Magazine - June 2022 - 48
Tech Briefs Magazine - June 2022 - 49
Tech Briefs Magazine - June 2022 - 50
Tech Briefs Magazine - June 2022 - 51
Tech Briefs Magazine - June 2022 - 52
Tech Briefs Magazine - June 2022 - 53
Tech Briefs Magazine - June 2022 - 54
Tech Briefs Magazine - June 2022 - 55
Tech Briefs Magazine - June 2022 - 56
Tech Briefs Magazine - June 2022 - Cov3
Tech Briefs Magazine - June 2022 - Cov4
Tech Briefs Magazine - June 2022 - MD-Cov1
Tech Briefs Magazine - June 2022 - MD-Cov2
Tech Briefs Magazine - June 2022 - MD-1
Tech Briefs Magazine - June 2022 - MD-2
Tech Briefs Magazine - June 2022 - MD-3
Tech Briefs Magazine - June 2022 - MD-4
Tech Briefs Magazine - June 2022 - MD-5
Tech Briefs Magazine - June 2022 - MD-6
Tech Briefs Magazine - June 2022 - MD-7
Tech Briefs Magazine - June 2022 - MD-8
Tech Briefs Magazine - June 2022 - MD-9
Tech Briefs Magazine - June 2022 - MD-10
Tech Briefs Magazine - June 2022 - MD-11
Tech Briefs Magazine - June 2022 - MD-12
Tech Briefs Magazine - June 2022 - MD-13
Tech Briefs Magazine - June 2022 - MD-14
Tech Briefs Magazine - June 2022 - MD-15
Tech Briefs Magazine - June 2022 - MD-16
Tech Briefs Magazine - June 2022 - MD-17
Tech Briefs Magazine - June 2022 - MD-18
Tech Briefs Magazine - June 2022 - MD-19
Tech Briefs Magazine - June 2022 - MD-20
Tech Briefs Magazine - June 2022 - MD-21
Tech Briefs Magazine - June 2022 - MD-22
Tech Briefs Magazine - June 2022 - MD-23
Tech Briefs Magazine - June 2022 - MD-24
Tech Briefs Magazine - June 2022 - MD-Cov3
Tech Briefs Magazine - June 2022 - MD-Cov4
Tech Briefs Magazine - June 2022 - Sensor-Cov1
Tech Briefs Magazine - June 2022 - Sensor-Cov2
Tech Briefs Magazine - June 2022 - Sensor-1
Tech Briefs Magazine - June 2022 - Sensor-2
Tech Briefs Magazine - June 2022 - Sensor-3
Tech Briefs Magazine - June 2022 - Sensor-4
Tech Briefs Magazine - June 2022 - Sensor-5
Tech Briefs Magazine - June 2022 - Sensor-6
Tech Briefs Magazine - June 2022 - Sensor-7
Tech Briefs Magazine - June 2022 - Sensor-8
Tech Briefs Magazine - June 2022 - Sensor-9
Tech Briefs Magazine - June 2022 - Sensor-10
Tech Briefs Magazine - June 2022 - Sensor-11
Tech Briefs Magazine - June 2022 - Sensor-12
Tech Briefs Magazine - June 2022 - Sensor-13
Tech Briefs Magazine - June 2022 - Sensor-14
Tech Briefs Magazine - June 2022 - Sensor-15
Tech Briefs Magazine - June 2022 - Sensor-16
Tech Briefs Magazine - June 2022 - Sensor-17
Tech Briefs Magazine - June 2022 - Sensor-18
Tech Briefs Magazine - June 2022 - Sensor-19
Tech Briefs Magazine - June 2022 - Sensor-20
Tech Briefs Magazine - June 2022 - Sensor-21
Tech Briefs Magazine - June 2022 - Sensor-22
Tech Briefs Magazine - June 2022 - Sensor-23
Tech Briefs Magazine - June 2022 - Sensor-24
Tech Briefs Magazine - June 2022 - Sensor-Cov3
Tech Briefs Magazine - June 2022 - Sensor-Cov4
https://www.nxtbook.com/smg/techbriefs/22TB09
https://www.nxtbook.com/smg/techbriefs/22TB08
https://www.nxtbook.com/smg/techbriefs/22TB07
https://www.nxtbook.com/smg/techbriefs/22TB06
https://www.nxtbook.com/smg/techbriefs/22TB05-P
https://www.nxtbook.com/smg/techbriefs/22TB05-D
https://www.nxtbook.com/smg/techbriefs/22TB04
https://www.nxtbook.com/smg/techbriefs/22TB03
https://www.nxtbook.com/smg/techbriefs/22TB02
https://www.nxtbook.com/smg/techbriefs/22TB01
https://www.nxtbook.com/smg/techbriefs/21TB12
https://www.nxtbook.com/smg/techbriefs/21TB11
https://www.nxtbook.com/smg/techbriefs/21TB10
https://www.nxtbook.com/smg/techbriefs/21TB09
https://www.nxtbook.com/smg/techbriefs/21TB08
https://www.nxtbook.com/smg/techbriefs/21TB07
https://www.nxtbook.com/smg/techbriefs/21TB06
https://www.nxtbook.com/smg/techbriefs/21TB05
https://www.nxtbook.com/smg/techbriefs/21TB04
https://www.nxtbook.com/smg/techbriefs/21BT03
https://www.nxtbook.com/smg/techbriefs/21TB02
https://www.nxtbookmedia.com