IEEE Robotics & Automation Magazine - March 2016 - 105
[5] E. Greensmith, P. L. Bartlett, and J. Baxter, "Variance reduction techniques
for gradient estimates in reinforcement learning," J. Machine Learning Res.,
vol. 5, pp. 1471-1530, Nov. 2004.
[6] F. Sehnke, C. Osendorfer, T. Rückstiess, A. Graves, J. Peters, and J. Schmidhuber, "Parameter-exploring policy gradients," Neural Netw., vol. 23, no. 4, pp.
551-559, 2010.
[7] G. Cheng, S. Hyon, J. Morimoto, A. Ude, G. H. Joshua, G. Colvin, W.
Scroggin, and C. J. Stephen, "Cb: A humanoid research platform for exploring
neuroscience," Adv. Robotics, vol. 21, no. 10, pp. 1097-1114, 2007.
[8] G. S. Fishman, Monte Carlo: Concepts, Algorithms, and Applications. Berlin, Germany: Springer-Verlag, 1996.
[9] H. Hachiya, J. Peters, and M. Sugiyama, "Reward weight regression with
sample reuse for direct policy search in reinforcement learning," Neural Comput., vol. 23, no. 11, pp. 2798-2832, 2011.
[10] J. Peters and S. Schaal, "Policy gradient methods for robotics," in Proc.
IEEE/RSJ Int. Conf. Intelligent Robots Systems, 2006, pp. 2219-2225.
[11] J. Peters and S. Schaal, "Reinforcement learning by reward-weighted regression for operational space control," in Proc. Int. Conf. Machine Learning,
2007, pp. 745-750.
[12] J. Peters, K. Mülling, and Y. Altün, "Relative Entropy Policy Search," in
[24] R. J. Williams, "Toward a theory of reinforcement-learning connectionist
systems," Tech. Rep. NU-CCS-88-3, College of Computer Science, Northeastern Univ., Boston, MA, 1988.
[25] R. J. Williams, "Simple statistical gradient-following algorithms for connectionist reinforcement learning," Mach. Learn., vol. 8, no. 3, pp. 229-256, 1992.
[26] R. S. Sutton, "Temporal credit assignment in reinforcement learning,"
Ph.D. dissertation, Univ. Massachusetts, 1984.
[27] R. S. Sutton and G. A. Barto, Reinforcement Learning: An Introduction.
Cambridge, MA: MIT Press, 1998.
[28] S. J. Pan and Q. Yang, "A survey on transfer learning," IEEE Trans. Knowledge Data Eng., vol. 22, no. 10, pp. 1345-1359, 2010.
[29] S. Kakade, "A natural policy gradient," in Proc. Advances Neural Information Processing Systems 14, 2002, pp. 1531-1538.
[30] S. Schaal and C. G. Atkeson, "Constructive incremental learning from
only local information," Neural Comput., vol. 10, no. 8, pp. 2047-2084, 1998.
[31] S. Schaal, "The SL simulation and real-time control software package,"
Univ. of Southern California, Los Angeles, CA, Tech. Rep., 2009.
[32] T. Matsubara, J. Morimoto, J. Nakanishi, M. Sato, and K. Doya, "Learning
CPG-based biped locomotion with a policy gradient method," Robot. Auton.
Syst., vol. 54, no. 11, pp. 911-920, 2006.
Proc. 24th AAAI Conf. Artificial Intelligence, 2010, pp. 1607-1612.
[13] J. Moody and C. J. Darken, "Fast learning in networks of locally-tuned
processing units," Neural Comput., vol. 1, no. 2, pp. 281-294, 1989.
[14] J. Morimoto, C. G. Atkeson, "Nonparametric representation of an approximated Poincare map for learning biped locomotion," Auton. Robots, vol. 27,
no. 2, pp. 131-144, 2009.
[15] J. Peters and S. Schaal, "Natural actor-critic," Neurocomputing, vol. 71, no.
79, pp. 1180-1190, 2008.
[16] L. Weaver and N. Tao, "The optimal reward baseline for gradient-based
reinforcement learning," in Proc. 7th Conf. Uncertainty Artificial Intelligence,
2001, pp. 538-545.
[17] M. P. Deisenroth and C. E. Rasmussen, "PILCO: A model-based and data
efficient approach to policy search," in Proc. Int. Conf. Machine Learning,
2011, , pp. 465-472.
[18] M. P. Deisenroth, D. Fox, and C. E. Rasmussen, "Gaussian processes for
data-efficient learning in robotics and control," IEEE Trans. Pattern Anal.
Mach. Intell., vol. 37, no. 2, pp. 408-423, 2015.
[19] N. Sugimoto and J. Morimoto, "Phase-dependent trajectory optimization
for CPG-based biped walking using path integral reinforcement learning," in
Proc. IEEE/RAS Int. Conf. Humanoid Robots, 2011, pp. 255-260.
[20] N. Sugimoto, M. Haruno, K. Doya, and M. Kawato, "MOSAIC for multiple-reward environments," Neural Comput., vol. 24, no. 3, pp. 577-606, 2012.
[21] N. Sugimoto, J. Morimoto, S. Hyon, and M. Kawato, "eMOSAIC Model for
Humanoid Robot Control," Neural Netw., vol. 29-30, pp. 8-19, May 2012.
[22] N. Sugimoto and J. Morimoto, "Trajectory-model-based reinforcement
Learning: Application to bimanual humanoid motor learning with a closedchain constraint," in Proc. IEEE-RAS Int. Conf. Humanoid Robots, 2013, pp.
429-434.
[23] N. Sugimoto, V. Tangkaratt, T. Wensveen, T. Zhao, M. Sugiyama, and J.
Morimoto, "Efficient reuse of previous experiences in humanoid motor learning," in Proc. IEEE-RAS Int. Conf. Humanoid Robots, 2014, pp. 554-559.
[33] T. Zhao, H. Hachiya, G. Niu, and M. Sugiyama, "Analysis and improvement of policy gradient estimation," Neural Netw., vol. 26, pp. 118-129, Feb.
2012.
[34] T. Zhao, H. Hachiya, V. Tangkaratt, J. Morimoto, and M. Sugiyama, "Efficient sample reuse in policy gradients with parameter-based exploration,"
Neural Comput., vol. 25, no. 6, pp. 1512-1547, 2013.
[35] V. Tangkaratt, S. Mori, T. Zhao, J. Morimoto, and M. Sugiyama, "Modelbased policy gradients with parameter-based exploration by least-squares conditional density estimation," Neural Netw., vol. 57, pp. 128-140, Sept. 2014.
[36] S. Schaal, C. G. Atkeson, "Learning control in robotics," IEEE Robot. Automat. Mag., vol. 17, no. 2, pp. 20-29, 2010.
Norikazu Sugimoto, National Institute of Information and
Communications Technology, Osaka, Japan. E-mail: xsugi@
nict.go.jp.
Voot Tangkaratt, The University of Tokyo, Japan. E-mail:
voot@ms.k.u-tokyo.ac.jp.
Thijs Wensveen, Delft University of Technology, The Netherlands.
E-mail: thijswensveen@gmail.com.
Tingting Zhao, the Tianjin University of Science and Technology, China. E-mail: tingting@tust.edu.cn.
Masashi Sugiyama, The University of Tokyo, Japan. E-mail:
sugi@k.u-tokyo.ac.jp.
Jun Morimoto, ATR Computational Neuroscience Labs, Kyoto,
Japan. E-mail: xmorimo@atr.jp.
march 2016
*
IEEE ROBOTICS & AUTOMATION MAGAZINE
*
105
Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - March 2016
IEEE Robotics & Automation Magazine - March 2016 - Cover1
IEEE Robotics & Automation Magazine - March 2016 - Cover2
IEEE Robotics & Automation Magazine - March 2016 - 1
IEEE Robotics & Automation Magazine - March 2016 - 2
IEEE Robotics & Automation Magazine - March 2016 - 3
IEEE Robotics & Automation Magazine - March 2016 - 4
IEEE Robotics & Automation Magazine - March 2016 - 5
IEEE Robotics & Automation Magazine - March 2016 - 6
IEEE Robotics & Automation Magazine - March 2016 - 7
IEEE Robotics & Automation Magazine - March 2016 - 8
IEEE Robotics & Automation Magazine - March 2016 - 9
IEEE Robotics & Automation Magazine - March 2016 - 10
IEEE Robotics & Automation Magazine - March 2016 - 11
IEEE Robotics & Automation Magazine - March 2016 - 12
IEEE Robotics & Automation Magazine - March 2016 - 13
IEEE Robotics & Automation Magazine - March 2016 - 14
IEEE Robotics & Automation Magazine - March 2016 - 15
IEEE Robotics & Automation Magazine - March 2016 - 16
IEEE Robotics & Automation Magazine - March 2016 - 17
IEEE Robotics & Automation Magazine - March 2016 - 18
IEEE Robotics & Automation Magazine - March 2016 - 19
IEEE Robotics & Automation Magazine - March 2016 - 20
IEEE Robotics & Automation Magazine - March 2016 - 21
IEEE Robotics & Automation Magazine - March 2016 - 22
IEEE Robotics & Automation Magazine - March 2016 - 23
IEEE Robotics & Automation Magazine - March 2016 - 24
IEEE Robotics & Automation Magazine - March 2016 - 25
IEEE Robotics & Automation Magazine - March 2016 - 26
IEEE Robotics & Automation Magazine - March 2016 - 27
IEEE Robotics & Automation Magazine - March 2016 - 28
IEEE Robotics & Automation Magazine - March 2016 - 29
IEEE Robotics & Automation Magazine - March 2016 - 30
IEEE Robotics & Automation Magazine - March 2016 - 31
IEEE Robotics & Automation Magazine - March 2016 - 32
IEEE Robotics & Automation Magazine - March 2016 - 33
IEEE Robotics & Automation Magazine - March 2016 - 34
IEEE Robotics & Automation Magazine - March 2016 - 35
IEEE Robotics & Automation Magazine - March 2016 - 36
IEEE Robotics & Automation Magazine - March 2016 - 37
IEEE Robotics & Automation Magazine - March 2016 - 38
IEEE Robotics & Automation Magazine - March 2016 - 39
IEEE Robotics & Automation Magazine - March 2016 - 40
IEEE Robotics & Automation Magazine - March 2016 - 41
IEEE Robotics & Automation Magazine - March 2016 - 42
IEEE Robotics & Automation Magazine - March 2016 - 43
IEEE Robotics & Automation Magazine - March 2016 - 44
IEEE Robotics & Automation Magazine - March 2016 - 45
IEEE Robotics & Automation Magazine - March 2016 - 46
IEEE Robotics & Automation Magazine - March 2016 - 47
IEEE Robotics & Automation Magazine - March 2016 - 48
IEEE Robotics & Automation Magazine - March 2016 - 49
IEEE Robotics & Automation Magazine - March 2016 - 50
IEEE Robotics & Automation Magazine - March 2016 - 51
IEEE Robotics & Automation Magazine - March 2016 - 52
IEEE Robotics & Automation Magazine - March 2016 - 53
IEEE Robotics & Automation Magazine - March 2016 - 54
IEEE Robotics & Automation Magazine - March 2016 - 55
IEEE Robotics & Automation Magazine - March 2016 - 56
IEEE Robotics & Automation Magazine - March 2016 - 57
IEEE Robotics & Automation Magazine - March 2016 - 58
IEEE Robotics & Automation Magazine - March 2016 - 59
IEEE Robotics & Automation Magazine - March 2016 - 60
IEEE Robotics & Automation Magazine - March 2016 - 61
IEEE Robotics & Automation Magazine - March 2016 - 62
IEEE Robotics & Automation Magazine - March 2016 - 63
IEEE Robotics & Automation Magazine - March 2016 - 64
IEEE Robotics & Automation Magazine - March 2016 - 65
IEEE Robotics & Automation Magazine - March 2016 - 66
IEEE Robotics & Automation Magazine - March 2016 - 67
IEEE Robotics & Automation Magazine - March 2016 - 68
IEEE Robotics & Automation Magazine - March 2016 - 69
IEEE Robotics & Automation Magazine - March 2016 - 70
IEEE Robotics & Automation Magazine - March 2016 - 71
IEEE Robotics & Automation Magazine - March 2016 - 72
IEEE Robotics & Automation Magazine - March 2016 - 73
IEEE Robotics & Automation Magazine - March 2016 - 74
IEEE Robotics & Automation Magazine - March 2016 - 75
IEEE Robotics & Automation Magazine - March 2016 - 76
IEEE Robotics & Automation Magazine - March 2016 - 77
IEEE Robotics & Automation Magazine - March 2016 - 78
IEEE Robotics & Automation Magazine - March 2016 - 79
IEEE Robotics & Automation Magazine - March 2016 - 80
IEEE Robotics & Automation Magazine - March 2016 - 81
IEEE Robotics & Automation Magazine - March 2016 - 82
IEEE Robotics & Automation Magazine - March 2016 - 83
IEEE Robotics & Automation Magazine - March 2016 - 84
IEEE Robotics & Automation Magazine - March 2016 - 85
IEEE Robotics & Automation Magazine - March 2016 - 86
IEEE Robotics & Automation Magazine - March 2016 - 87
IEEE Robotics & Automation Magazine - March 2016 - 88
IEEE Robotics & Automation Magazine - March 2016 - 89
IEEE Robotics & Automation Magazine - March 2016 - 90
IEEE Robotics & Automation Magazine - March 2016 - 91
IEEE Robotics & Automation Magazine - March 2016 - 92
IEEE Robotics & Automation Magazine - March 2016 - 93
IEEE Robotics & Automation Magazine - March 2016 - 94
IEEE Robotics & Automation Magazine - March 2016 - 95
IEEE Robotics & Automation Magazine - March 2016 - 96
IEEE Robotics & Automation Magazine - March 2016 - 97
IEEE Robotics & Automation Magazine - March 2016 - 98
IEEE Robotics & Automation Magazine - March 2016 - 99
IEEE Robotics & Automation Magazine - March 2016 - 100
IEEE Robotics & Automation Magazine - March 2016 - 101
IEEE Robotics & Automation Magazine - March 2016 - 102
IEEE Robotics & Automation Magazine - March 2016 - 103
IEEE Robotics & Automation Magazine - March 2016 - 104
IEEE Robotics & Automation Magazine - March 2016 - 105
IEEE Robotics & Automation Magazine - March 2016 - 106
IEEE Robotics & Automation Magazine - March 2016 - 107
IEEE Robotics & Automation Magazine - March 2016 - 108
IEEE Robotics & Automation Magazine - March 2016 - 109
IEEE Robotics & Automation Magazine - March 2016 - 110
IEEE Robotics & Automation Magazine - March 2016 - 111
IEEE Robotics & Automation Magazine - March 2016 - 112
IEEE Robotics & Automation Magazine - March 2016 - 113
IEEE Robotics & Automation Magazine - March 2016 - 114
IEEE Robotics & Automation Magazine - March 2016 - 115
IEEE Robotics & Automation Magazine - March 2016 - 116
IEEE Robotics & Automation Magazine - March 2016 - 117
IEEE Robotics & Automation Magazine - March 2016 - 118
IEEE Robotics & Automation Magazine - March 2016 - 119
IEEE Robotics & Automation Magazine - March 2016 - 120
IEEE Robotics & Automation Magazine - March 2016 - Cover3
IEEE Robotics & Automation Magazine - March 2016 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2010
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2010
https://www.nxtbookmedia.com