IEEE Robotics & Automation Magazine - June 2021 - 73

Conclusions
The article presented the application of a robotic system in refilling
a supermarket shelf. The main lesson learned from this experience
is that the successful exploitation of robots in a real
supermarket depends highly on the availability of different technologies
and methodologies, from perception to planning and
control. If the perception is of paramount relevance, using (visual)
perception for object detection only before the planning phase
is definitely not enough to accomplish the task. Only a multimodal
(visual and tactile) sensor-based control allowed the robot
to safely grasp the detected item to be placed on the shelf layer.
Another step change was the tight integration between the
planning and control modules. Planning using the sole dexterity
of the robot manipulator was successful in elementary situations
only. The motion planning became successful in almost
all conditions only by including a further degree of dexterity
provided by the low-level grasp controller of the robot gripper.
The final result was evaluated by the REFILLS end user (dmdrogerie
markt GmbH), which gave very positive feedback,
and the current technology readiness level is TRL 5.
Acknowledgments
This work received funding from the European Commission
under the H2020 Framework Programme, REFILLS project
GA 731590. This article has supplementary downloadable material
available at https://doi.org/10.1109/MRA.2021.3064754, provided
by the authors.
References
[1] R. Bogue, " Strong prospects for robots in retail, " Ind. Robot., vol. 46,
no. 3, pp. 326-331, 2019. doi: 10.1108/IR-01-2019-0023.
[2] " Universal depalletizer. " Photoneo. Accessed: Dec. 2020. [Online].
Available: https://www.photoneo.com/wp-content/uploads/2020/07/
Universal-Depalletizer.pdf
[3] R. Caccavale et al., " A flexible robotic depalletizing system for
supermarket logistics, " IEEE Robot. Automat. Lett., vol. 5, no. 3,
pp. 4471-4476, 2020. doi: 10.1109/LRA.2020.3000427.
[4] H. Kuhn and M. Sternbeck, " Integrative retail logistics: An exploratory
study, " Oper. Manage. Res., vol. 6, nos. 1-2, pp. 2-18, 2013. doi:
10.1007/s12063-012-0075-9.
[5] J. Winkler et al., " Knowledge-enabled robotic agents for shelf replenishment
in cluttered retail environments, " in Proc. Int. Conf. Autonom. Agents
Multiagents, May 2016, pp. 1421-1422.
[6] R. Sakai et al., " A mobile dual-arm manipulation robot system for
stocking and disposing of items in a convenience store by using universal
vacuum grippers for grasping items, " Adv. Robot., vol. 34, nos.
3-4, pp. 219-234, 2020. doi: 10.1080/01691864.2019.1705909.
[7] G. A. G. Ricardez et al., " Restock and straightening system for retail
automation using compliant and mobile manipulation, " Adv. Robot.,
vol. 34, nos. 3-4, pp. 235-249, 2020. doi: 10.1080/01691864.2019.1698460.
[8] " Future convenience store challenge. " World Robotic Summit.
Accessed: Dec. 2020. [Online]. Available: https://worldrobotsummit.org/en/
wrs2020/challenge/service/fcsc.html
[9] " Robotics enabling fullyintegrated logistics lines for supermarkets. "
The REFILLS Project. Accessed: Dec. 2020. [Online]. Available: http://www
.refills-project.eu/
[10] M. Tenorth and M. Beetz, " Knowrob: A knowledge processing
infrastructure for cognition-enabled robots, " Int. J. Robotics Res., vol.
32, no. 5, pp. 566-590, 2013. doi: 10.1177/0278364913481635.
[11] C. Wang et al., " Densefusion: 6d object pose estimation by iterative
dense fusion, " in Proc. IEEE/CVF Conf. Comput. Vision Pattern Recogn.
(CVPR), 2019, pp. 3338-3347. doi: 10.1109/CVPR.2019.00346.
[12] Y. Xiang, T. Schmidt, V. Narayanan, and D. Fox, " PoseCNN: A convolutional
neural network for 6d object pose estimation in cluttered
scenes, " 2018, arXiv:1711.00199v3.
[13] J. Redmon and A. Farhadi, " Yolov3: An incremental improvement, "
2018. [Online]. Available: https://arxiv.org/pdf/1804.02767.pdf
[14] Q. Li, O. Kroemer, Z. Su, F. F. Veiga, M. Kaboli, and H. J. Ritter, " A
review of tactile information: Perception and action through touch, " IEEE
Trans. Robot., vol. 36, no. 6, pp. 1-16, 2020. doi: 10.1109/TRO.2020.3003230.
[15] M. Costanzo, G. De Maria, C. Natale, and S. Pirozzi, " Design and
calibration of a force/tactile sensor for dexterous manipulation, " Sensors,
vol. 19, no. 4, pp. 1-23, 2019. doi: 10.3390/s19040966.
[16] M. Costanzo, G. De Maria, and C. Natale, " Two-fingered in-hand
object handling based on force/tactile feedback, " IEEE Trans. Robot.,
vol. 36, no. 1, pp. 157-173, Feb. 2020. doi: 10.1109/TRO.2019.2944130.
[17] M. Costanzo, G. De Maria, G. Lettera, and C. Natale, " Grasp control
for enhancing dexterity of parallel grippers, " in Proc. IEEE Int.
Conf. Robot. Automat., Paris, France, June 2020, pp. 524-530.
[18] A. Cavallo, M. Costanzo, G. De Maria, and C. Natale, " Modeling
and slipping control of a planar slider, " Automatica, vol. 115, p. 108,875,
2020. doi: 10.1016/j.automatica.2020.108875.
[19] S. Chitta, I. Sucan, and S. Cousins, " MoveIt! [ROS topics], " IEEE
Robot. Automat. Mag., vol. 19, no. 1, pp. 18-19, Mar. 2012. doi: 10.1109/
MRA.2011.2181749.
[20] M. Costanzo et al., " Manipulation planning and control for shelf
replenishment, " IEEE Robot. Automat. Lett., vol. 5, no. 2, pp. 1595-1601,
Apr. 2020. doi: 10.1109/LRA.2020.2969179.
[21] E. Marchand, F. Spindler, and F. Chaumette, " ViSP for visual serving:
A generic software platform with a wide class of robot control
skills, " IEEE Robot. Automat. Mag., vol. 12, no. 4, pp. 40-52, Dec. 2005.
doi: 10.1109/MRA.2005.1577023.
[22] R. D. Howe and M. R. Cutkosky, " Practical force-motion models for
sliding manipulation, " Int. J. Robot. Res., vol. 15, no. 6, pp. 557-572, 1996.
doi: 10.1177/027836499601500603.
Marco Costanzo, Department of Engineering, Università degli
Studi della Campania Luigi Vanvitelli, Aversa (CE), 81031,
Italy. Email: marco.costanzo@unicampania.it.
Giuseppe De Maria, Department of Engineering, Università
degli Studi della Campania Luigi Vanvitelli, Aversa (CE),
81031, Italy. Email: giuseppe.demaria@unicampania.it.
Gaetano Lettera, Department of Engineering, Università degli
Studi della Campania Luigi Vanvitelli, Aversa (CE), 81031,
Italy. Email: gaetano.lettera@unicampania.it.
Ciro Natale, Department of Engineering, Università degli
Studi della Campania Luigi Vanvitelli, Aversa (CE), 81031,
Italy. Email: ciro.natale@unicampania.it.
JUNE 2021 * IEEE ROBOTICS & AUTOMATION MAGAZINE *
73
https://www.arxiv.org/pdf/1804.02767.pdf https://www.doi.org/10.1109/MRA.2021.3064754 https://www.photoneo.com/wp-content/uploads/2020/07/Universal-Depalletizer.pdf https://www.photoneo.com/wp-content/uploads/2020/07/Universal-Depalletizer.pdf https://worldrobotsummit.org/en/wrs2020/challenge/service/fcsc.html https://worldrobotsummit.org/en/wrs2020/challenge/service/fcsc.html http://www.refills-project.eu/ http://www.refills-project.eu/
IEEE Robotics & Automation Magazine - June 2021

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2021
