IEEE Robotics & Automation Magazine - December 2021 - 20

divergence to quantitatively compare the obtained policies,
enabling us to consolidate earlier empirical findings. Most
importantly, we succeeded in deploying controllers learned in
simulation to a commercial robotic platform on real-world
staircases. The robot was able to successfully ascend and
descend while respecting the underlying criteria.
Our framework can accommodate improvements that
would increase the autonomy of the deployed system.
Although we managed to train effective controllers for normal
situations, accidents might still happen, as the robot may
encounter situations not explored in training; hence, incremental
learning is an interesting perspective. Also, full 3D
navigation autonomy implies the development of multiple
individual controllers that need to be coordinated with one
another. We seek to address these challenges in the future as
well as to explore the possibility of using raw sensory data as
input to the learned controllers.
Acknowledgments
The work was performed in the context of the REACT and
Vaincre l'Isolement par les TIC par l'Ambient Assisted Living
projects and financed by Brest Metropole, the region of Brittany,
and the European Regional Development Fund. This article
has supplementary downloadable material available at https://
doi.org/10.1109/MRA.2021.3114105, provided by the authors.
References
[1] K. Chatzilygeroudis, V. Vassiliades, F. Stulp, S. Calinon, and J.
Mouret, " A survey on policy search algorithms for learning robot controllers
in a handful of trials, " IEEE Trans. Robot., vol. 36, no. 2, pp. 328-
347, 2020. doi: 10.1109/TRO.2019.2958211.
[2] F. Torabi, G. Warnell, and P. Stone, " Behavioral cloning from observation, "
in Proc. 27th Int. Joint Conf. Artif. Intell., 2018, pp. 4950-4957.
[3] J. Ho and S. Ermon, " Generative adversarial imitation learning, " in Proc.
Adv. Neural Inf. Process. Syst., Curran Associates, 2016, vol. 29, pp. 4565-4573.
[4] A. Mitriakov, P. Papadakis, S. M. Nguyen, and S. Garlatti, " Staircase
traversal via reinforcement learning for active reconfiguration of assistive
robots, " in Proc. IEEE Int. Conf. Fuzzy Syst., 2020, pp. 1-8. doi:
10.1109/FUZZ48607.2020.9177581.
[5] A. Mitriakov, P. Papadakis, S. M. Nguyen, and S. Garlatti, " Staircase
negotiation learning for articulated tracked robots with varying
degrees of freedom, " in Proc. Int. Symp. Saf., Security Rescue Robot.,
2020, pp. 394-400. doi: 10.1109/SSRR50563.2020.9292594.
[6] S. G. Tzafestas, " Mobile robot control and navigation: A global overview, "
J. Intell. Robot. Syst., vol. 91, no. 1, pp. 35-58, 2018. doi: 10.1007/
s10846-018-0805-9.
[7] A. I. Mouriakis, N. Trawny, S. I. Roumeliotis, D. M. Helmick, and L.
Matthies, " Autonomous stair climbing for tracked vehicles, " Int. J. Robot.
Res., vol. 26, no. 7, pp. 737-758, 2007. doi: 10.1177/0278364907080423.
[8] K. Nagatani, A. Yamasaki, K. Yoshida, T. Yoshida, and E. Koyanagi,
" Semi-autonomous traversal on uneven terrain for a tracked vehicle using
autonomous control of active flippers, " in Proc. IEEE/RSJ Int. Conf. Intell.
Robots Syst., 2008, pp. 2667-2672. doi: 10.1109/IROS.2008.4650643.
[9] P. Ben-Tzvi, S. Ito, and A. A. Goldenberg, " A mobile robot with
autonomous climbing and descending of stairs, " Robotica, vol. 27, no. 2,
pp. 171-188, 2009. doi: 10.1017/S0263574708004426.
20 * IEEE ROBOTICS & AUTOMATION MAGAZINE * DECEMBER 2021
[10] H. Zhang and A. Song, " System centroid position based tipover stability
enhancement method for a tracked search and rescue robot, "
Adv. Robot., vol. 28, no. 23, pp. 1571-1585, 2014. doi: 10.1080/01691864.
2014.976654.
[11] M. Pecka, V. Šalanský, K. Zimmermann, and T. Svoboda, " Autonomous
flipper control with safety constraints, " in Proc. IEEE/RSJ Int.
Conf. Intell. Robots Syst., 2016, pp. 2889-2894. doi: 10.1109/
IROS.2016.7759447.
[12] D. Endo, A. Watanabe, and K. Nagatani, " Stair climbing control for
4-dof tracked vehicle based on internal sensors, " J. Robot., vol. 2017, pp.
1-18, Oct. 2017. doi: 10.1155/2017/3624589.
[13] M. M. Ejaz, T. B. Tang, and C. Lu, " Vision-based autonomous navigation
approach for a tracked robot using deep reinforcement learning, "
IEEE Sensors J., vol. 21, no. 2, pp. 2230-2240, 2021. doi: 10.1109/
JSEN.2020.3016299.
[14] S. Suzuki, S. Hasegawa, and M. Okugawa, " Remote control system
of disaster response robot with passive sub-crawlers considering falling
down avoidance, " ROBOMECH J., vol. 1, no. 1, p. 20, 2014. doi:
10.1186/s40648-014-0020-9.
[15] D. C. Guastella and G. Muscato, " Learning-based methods of perception
and navigation for ground vehicles in unstructured environments: A
review, " Sensors, vol. 21, no. 1, p. 73, 2020. doi: 10.3390/s21010073.
[16] K. Zimmermann, P. Zuzanek, M. Reinstein, and V. Hlavac, " Adaptive
traversability of unknown complex terrain with obstacles for
mobile robots, " in Proc. IEEE Int. Conf. Robot. Automat., 2014, pp.
5177-5182. doi: 10.1109/ICRA.2014.6907619.
[17] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction.
Cambridge, MA: MIT Press, 2018.
[18] A. Azar et al., " Drone deep reinforcement learning: A review, " Electronics,
vol. 10, no. 9, p. 999, 2021. doi: 10.3390/electronics10090999.
[19] E. Garcia, J. Estremera, and P. Gonzalez-de-Santos, " A classification
of stability margins for walking robots, " Robotica, vol. 20, no. 6, pp. 595-
606, 2002. doi: 10.1017/S0263574702004502.
[20] M. Pecka, K. Zimmermann, and T. Svoboda, " Fast simulation of vehicles
with non-deformable tracks, " in Proc. 2017 IEEE/RSJ Int. Conf. Intell. Robots
Syst. (IROS), pp. 6414-6419. doi: 10.1109/IROS.2017.8206546.
Andrei Mitriakov, Research Laboratory in Information and
Communication Science and Technology, IMT Atlantique,
Lab-STICC, UMR 6285, team RAMBO, Brest, F-29238,
France. Email: andrei.mitriakov@imt-atlantique.fr.
Panagiotis Papadakis, Research Laboratory in Information and
Communication Science and Technology, IMT Atlantique,
Lab-STICC, UMR 6285, team RAMBO, Brest, F-29238,
France. Email: panagiotis.papadakis@imt-atlantique.fr.
Jérôme Kerdreux, Research Laboratory in Information and
Communication Science and Technology, IMT Atlantique,
Lab-STICC, UMR 6285, team RAMBO, Brest, F-29238,
France. Email: jerome.kerdreux@imt-atlantique.fr.
Serge Garlatti, Research Laboratory in Information and Communication
Science and Technology, IMT Atlantique, LabSTICC,
UMR 6285, team RAMBO, Brest, F-29238. France.
Email: serge.garlatti@imt-atlantique.fr.
http://www.doi.org/10.1109/MRA.2021.3114105 http://www.doi.org/10.1109/MRA.2021.3114105
IEEE Robotics & Automation Magazine - December 2021

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2021
