IEEE Technology and Society Magazine - September 2019 - 63

horizon much longer than the twenty year period
examined in the HINDSIGHT study. These studies
concluded that earlier science events, unrelated to
military considerations, were of much greater
importance relative to technology events as sources of technical change (Mowery and Rosenberg
1979 [3]; Thirtle and Ruttan 1985)."
In comparing Hindsight and TRACES, we note several
key differences, some of which may be more important
than the time horizon. They differ in period of study,
types of innovation studied, classification of research
events, and in positionality of researchers. First, as
noted, Hindsight was strictly bound by the 20-year window before the development of a technology, while
TRACES was not. Indeed, the TRACES authors found
that 75-80% of non-mission research had occurred
15-20 years prior to the introduction of the innovations
studied (see Figures 3 and 4, which are Figures 3 and 5
in the TRACES report). Figure 4 (Figure 5 in the TRACES
report) shows a marked difference between the distribution of non-mission and mission-oriented research
events. At 30, 20, and 10 years prior to an innovation,
the authors find that, respectively, 30, 70, and 85 percent of non-mission and 22, 38, and 62 percent of mission-orient events had occurred.
Notably, mission-oriented events increased dramatically about 25 years prior to the innovation. This may be
explained by project deadlines (which are arguably more
constrained in mission-oriented research), the fact that
mission-oriented research requires previous non-mission
work (imagine a threshold of 80% completion of nonmission events in order to see a spike in mission-oriented
events), or by the onset of research developments in
WWII - developments which the authors acknowledge.
Nevertheless, TRACES classified about 70% of total tracings as non-mission, and stopped measuring events after
each innovation. If we take the conclusions of Hindsight to
be true, that the majority of, in their terms, engineering
and technology events occurred after an innovation, it
then appears likely that the distribution of mission-oriented research events in Figure 4 are skewed results because
of undersampling.
Secondly, the TRACES report examined only novel
civilian technologies and only up to the point of the
introduction of each technology. This is in contrast to the
Hindsight report, which examined only existing military
technologies and did so through the point of innovation
or system development. Thirdly, the TRACES report
allowed for great discretion in event classification. Each
technology's team of researchers classified events independently. TRACES presents no rubric as comprehensive as that of Hindsight; as Table 3 shows, its
findings might agree with Hindsight given different

SEPTEMBER 2019

∕

interpretations of the classifications. In particular,
overcounting of non-mission events could have
occurred given a strict reading of the phrase research
"for a specific application." In comparison, Hindsight
RXD events were defined as "the occurrence of a novel
idea and subsequent engineering application," a definition that could undercount contributing events that
had no engineering application.
Fourthly, the researchers differed in the institutional
goals of the funding and implementing organizations.
The TRACES report directly responds to the preliminary
Hindsight report. The DOD had efficiency and the production of technologies in mind. In contrast, the NSF
takes non-mission research as its aim, where the rationale for such funding might be limited if the Hindsight
results were true. The editors state:
"The relationship of nonmission research to our
economy is, at best, only generally understood.
The way in which nonmission research appears to
be used to achieve economic gain is analogous to
capital wealth. We establish a fund of knowledge
against which withdrawals can be made to
achieve innovation at a rate satisfactory to society." [10, p. 21].

A Defense of the Hindsight 20-Year Time Horizon
We further expand on why simply utilizing a longer timeframe does not necessarily imply that TRACES had a
more sound methodology. There are two reasons why
this might not be the case.
TRACES' proposal for a longer time horizon may
not support continued basic science research in
general. The timelines of RXD events for each of Hindsight's systems show a unilateral decrease in events
prior to five years before the system development. This
decrease is present in enough systems that it reinforces
Hindsight's argument that 20 years was a plausible

TABLE 4. TRACES Classifications [11, p. ix].
TRACES
Classification Description
Non-mission

motivated by the search for knowledge and
scientific understanding without special
regard for its application

Mission

performed to develop information for a specific
application concept prior to development of a
prototype product or engineering design

Development
and
application

involving prototype development and
engineering design directed toward the
demonstration of a specific process or
product for purposes of marketing

IEEE TECHNOLOGY AND SOCIETY MAGAZINE

IEEE Technology and Society Magazine - September 2019

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - September 2019