IEEE Robotics & Automation Magazine - September 2019 - 77

For almost all authors across all research areas, the model is
extremely confident ( p 2 .95) that the intercept is less than 1:1
(below 0 on the log odds scale and within the bounds of the
vertical line). This means that, outside of the NRI, almost all
authors are unlikely to publish in these areas. At the same time,
for many authors across the research areas, the model is confident ( p 2 .75) or at least more confident than not ( p 2 .5) of
a positive NRI effect. In some cases, the model is confident of a
substantial positive effect, corresponding to a change of 2:1
odds or greater. For several areas-most especially decision
science and psychology-the model is highly confident of
large effects, but the uncertainty over these estimates ranges
over several orders of magnitude.
Discussion
Based on our bibliometric analysis, in its first five years of funding, the NRI has supported novel collaborations and stimulated interdisciplinary collaboration and publishing. We found
that 31 pairs of NRI-funded collaborators at different institutions have never published together before (22% of all NRIfunded collaborations). Additionally, 26 NRI awards (8% of all
awards) make substantial references to prior research in fields
such as biology, medicine, neuroscience, Earth science, materials science, physics, and psychology. NRI-funded researchers
also tend to publish more in these nonrobotics fields.
Our findings indicate that the NRI has begun to influence
the robotics research community, supporting novel collaborations and interdisciplinary research. NRI 2.0's central
themes-ubiquitous corobots, greater scalability and variety,
and lowered barriers to robotics research-mean that the program will require still more novel and interdisciplinary collaboration. We therefore expect the trends we have identified here
to continue. In addition, over the next decade, we expect that
the NRI will start to have a significant societal impact as foundational research is translated into consumer products.
Acknowledgments
Work on this article was conducted while Daniel Hicks was a
Science and Technology Policy Fellow with the American
Association for the Advancement of Science and Reid Simmons was serving as a program director at the U.S. National
Science Foundation (NSF) in the Information and Intelligent
Systems division of Computer and Information Science and
Engineering. Hicks' fellowship was funded under NSF award
1643294. Henrick Christensen helpfully provided documents
on the early history of the National Robotics Initiative (NRI).
Current and past NSF staff who played a substantial role in
shaping the NRI include Jeff Trinkle, Satyandra Gupta, Richard Voyles, Howard Wactlar, and Lynne Parker. We thank Steven Thomson (U.S. Department of Agriculture), Rodrigo
Rimando (U.S. Department of Energy), and Grace Peng (the
National Institutes of Health) for providing us with information on the NRI awards funded by their agencies. The views
expressed in this article are those of the authors and should not
be interpreted as the official recommendations or policy of the
NSF or other author-affiliated organizations.

References
[1] Computing Community Consortium, "From Internet to robotics: The
next transformative technology." Accessed on: July 10, 2017. [Online].
Available: http://archive2.cra.org/ccc/visioning/visioning-activities/robotics
[2] Computing Community Consortium and Computing Research Association, "A roadmap for US robotics: From Internet to robotics," May 21,
2009. [Online]. Available: http://www.us-robotics.us/reports/CCCReport
.pdf
[3] J. L. King, V. Gurbaxani, K. L. Kraemer, F. W. McFarlan, K. S. Raman,
and C. S. Yap, "Institutional factors in information technology innovation,"
Inform. Syst. Res., vol. 5, no. 2, pp. 139-169, 1994. doi: 10.1287/isre.5.2.139.
[4] National Science Foundation, "National Robotics Initiative (NRI):
The realization of co-robots acting in direct support of individuals
and groups," NSF 11-553, 2011. Accessed on: July 10, 2017. [Online]. Available: https://www.nsf.gov/pubs/2011/nsf11553/nsf11553.htm
[5] E. Messina, B. A. Weiss, and R. Madhavan, "Co-x panel discussion,"
in Proc. 10th Performance Metrics for Intelligent Systems Workshop,
2010, pp. 361-363.
[6] National Science Foundation, "National Robotics Initiative 2.0: Ubiquitous collaborative robots (NRI-2.0)," NSF 17-518, 2017. Accessed on:
Aug. 21, 2017. [Online]. Available: https://www.nsf.gov/pubs/2017/nsf17518/
nsf17518.htm
[7] P. Lin, "Why ethics matters for autonomous cars," in Autonomes
Fahren: Technische, Rechtliche und Gesellschaftliche Aspekte, M. Maurer, J. C. Gerdes, B. Lenz, and H. Winner, Eds. Berlin: Springer-Verlag,
2015, pp. 69-85.
[8] K. Crawford and R. Calo, "There is a blind spot in AI research,"
Nature, vol. 538, no. 7625, pp. 311-313, 2016. doi: 10.1038/538311a.
[9] Executive Office of the President of the United States. "Artificial intelligence, automation, and the economy," Dec. 20, 2016. [Online]. Available:
https://obamawhitehouse.archives.gov/sites/whitehouse.gov/files/
documents/Artificial-Intelligence-Automation-Economy.PDF
[10] S. Vallor, "Moral deskilling and upskilling in a new machine age:
Reflections on the ambiguous future of character," Philosophy Technol.,
vol. 28, no. 1, pp. 107-124, 2015. doi: 10.1007/s13347-014-0156-9.
[11] Elsevier, "Content." Accessed on: July 5, 2017. [Online]. Available:
https://www.elsevier.com/solutions/scopus/content
[12] J. Mingers and L. Leydesdorff, "A review of theory and practice in
scientometrics," Eur. J. Oper. Res., vol. 246, no. 1, pp. 1-19, 2015. doi:
10.1016/j.ejor.2015.04.002.
[13] A. Gelman and J. Hill, Data analysis using regression and multilevel/hierarchical models. Cambridge, U.K.: Cambridge Univ. Press,
2006.
[14] J. Gabry et al., "rstanarm: Bayesian applied regression modeling
via Stan," The Comprehensive R Archive Network, 2017. [Online]. Available:
https://cran.r-project.org/web/packages/rstanarm/index.html
[15] Stan. Accessed on: June 5, 2019. [Online]. Available: https://mc-stan
.org/

Daniel J. Hicks, American Association for the Advancement of
Science, Alexandria, Virginia. Email: hicks.daniel.j@gmail
.com.
Reid Simmons, National Science Foundation, Alexandria,
Virginia. Email: reids@cs.cmu.edu.

SEPTEMBER 2019

IEEE ROBOTICS & AUTOMATION MAGAZINE

http://archive2.cra.org/ccc/visioning/visioning-activities/robotics http://www.us-robotics.us/reports/CCCReport.pdf http://www.us-robotics.us/reports/CCCReport.pdf https://www.nsf.gov/pubs/2011/nsf11553/nsf11553.htm https://www.nsf.gov/pubs/2017/nsf17518/nsf17518.htm https://www.nsf.gov/pubs/2017/nsf17518/nsf17518.htm https://obamawhitehouse.archives.gov/sites/whitehouse.gov/files/documents/Artificial-Intelligence-Automation-Economy.PDF https://www.elsevier.com/solutions/scopus/content https://cran.r-project.org/web/packages/rstanarm/index.html https://mc-stan.org https://mc-stan.org

IEEE Robotics & Automation Magazine - September 2019

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2019