IEEE Technology and Society Magazine - September 2019 - 67

I and II rockets). Furthermore, Hindsight cautioned
that conclusions about the role of basic and applied
science may not apply to non-DOD systems.
A focus on the public values and the public good
that emerges from engineering developments would
magnify the benefits of a future Hindsight-like study [4].
For example, a study of the knowledge required to
solve some public problem would complement Hindsight's focus on innovations needed to make a new
weapon system. Additional analysis categories surrounding R XD events could reflect social values,
change, and opposition, teasing out more of the rationale underlying a given decision. Future case studies
that trace how efforts to achieve public good succeeded in engineering systems would potentially provide
guidance for engineers trying to similarly guide their
technical systems to positive outcomes. Structured
ways for engineers to learn of public values and infuse
them into design decisions could be one potential outcome from expanding a Hindsight-like approach [20].
Such potential analysis should deeply acknowledge that
many pathways for policy and social change can proceed without any role for engineering.
The political prospects for Hindsight-like case studies may be stronger than they appear, especially given
a different scope and focus. For example, case studies
may proceed more easily in other fields of engineering
outside of the defense sector. A study of artificial intelligence might examine how new research and technologies measure up against their claims of benefit and
relate them to values displaced by such systems. Engineering managers have the discretion to initiate smaller case studies on the sources of innovation in their
domain areas. Smaller scale analyses reduce the risk
of Hindsight-level controversy - big budgets draw big
interest, after all. Private sector companies could also
conduct a Hindsight-like study internally to avoid public
controversy. Were the engineering community to share
knowledge from case studies in different areas, we
might gain insights to help better manage programs as
a whole and provide greater benefit to society.

Acknowledgment
The authors thank the thoughtful anonymous reviewers
as well as Dr. Marci Baranski for comments on the draft.
An earlier version of this article was presented at the
2018 IEEE International Symposium on Technology and
Society (ISTAS'18) in Washington, DC, and was substantially modified for this publication.
14
These figures are scans of graphs that appear in the original HINDSIGHT
report.
15
These figures are scans of graphs that appear in the original TRACES
report, Volume 1.

SEPTEMBER 2019

∕

This paper reflects the authors' personal views and
does not necessarily reflect the views of NASA and the
United States Government.

Author Information
Zachary Pirtle is with NASA Headquarters, Washington,
DC, U.S.A. Email: zpirtle@nasa.gov.
Jared Moore is with the Wadhwani Institute for
Artificial Intelligence, Mumbai, India. Email: jared@
jaredmoore.org.

References
[1] S. Toulmin, "The complexity of scientific choice: A Stocktaking,"
Minerva, vol. 2, no. 3, p. 343-359, 1964.
[2] V. Bush, Science, The Endless Frontier: A Report to the President. U.S. GPO, 1945.
[3] D. Mowery and N. Rosenberg, "The influence of market demand
upon innovation: A critical review of some recent empirical studies," Research Policy, vol. 8, no. 2, pp. 102-153, 1979.
[4] B. Bozeman and D. Sarewitz, "Public value mapping and science
policy evaluation," Minerva, vol. 49, no. 1, pp. 1-23, 2011.
[5] Z. Szajnfarber, "Innovation pathways in technology intensive
government organizations: Insights from NASA, M.I.T. Libraries,
2011.
[6] R.S. Isenson, Project Hindsight: Final Report. Washington,
DC: Office Of The Director of Defense Research and Engineering, 1969,
[7] C.W. Sherwin and R.S. Isenson, First Interim Report on Project
Hindsight (Summary). Washington, DC: Office Of The Director of
Defense Research and Engineering, 1966.
[8] C.W. Sherwin and R.S. Isenson, "Project Hindsight," Science,
vol. 156, no. 3782, pp. 1571-1577, 1967.
[9] D.S. Greenberg, "'Hindsight': DOD study examines return on
investment in research," Science, vol. 154, no. 3751, pp. 872-873,
1966.
[10] I.R. Institute, Technology in retrospect and critical events in
science, Volume 2. 1969.
[11] I.R. Institute, Technology in retrospect and critical events in
science, Volume 1. 1968.
[12] V.W. Ruttan, Is War Necessary For Economic Growth?: Military Procurement and Technology Development. Oxford, U.K.:
Oxford Univ. Press, 2006.
[13] Z. Pirtle, Engineering Innovation: Energy, Policy, and the
Role of Engineering, in Philosophy and Engineering: Reflections on Practice, Principles and Process. Springer, 2013,
p. 377-390.
[14] E. Layton, "Mirror-image twins: The communities of science
and technology in 19th-century America," Technology and Culture,
vol. 12, no. 4, pp. 562-580,1971.
[15] V.T.O. Narayanamurti, Cycles of Invention and Discovery.
Cambridge, MA: Harvard Univ. Press, 2016.
[16] K. Kreilkamp, "Hindsight and the real world of science policy,"
Science Studies, vol. 1, no. 1, pp. 43-66, 1971.
[17] E. Fisher, R.L. Mahajan, and C. Mitcham, "Midstream modulation of technology: Governance from within," Bull. Science, Technology & Society, vol. 26, no. 6, pp. 485-496, 2006.
[18] D.H. Guston, Between Politics and Science: Assuring the
Integrity and Productivity of Research. Cambridge, U.K.: Cambridge Univ. Press, 2000.
[19] U.S. House of Representiatives, C.o.S.a.T.T.F.o.S.P., A History
of Science Policy in the United States, 1950-1985. Background Rep.
no. 1. 99th Congress, Second Session, 1986.
[20] P. Bertrand, Z. Pirtle, and D. Tomblin, "Participatory technology assessment for Mars mission planning: Public values and rationales," Space Policy, vol. 42, pp. 41-53, 2017.

IEEE TECHNOLOGY AND SOCIETY MAGAZINE

67
IEEE Technology and Society Magazine - September 2019

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - September 2019
