Quality Progress - March 2017 - 47


Looking at several possible
models and selecting one to
focus on has some potential
for selecting incorrectly and
drawing false conclusions.
Hence, if several models look
reasonable based on the data
and other knowledge, it can
be beneficial to continue to
explore the results for all of
these competitive models.
In this case, we considered
results from both models. If
further exploration or data collection were performed to find
the minimum density, continuing to evaluate concentrations
between 9.5 and 11.5 is likely
merited, not just close to 10.8
as the cubic model suggests.
Second, by comparing
results from several possible
models, we can assess the relative contributions of the two
types of uncertainty. If we had
just looked at Figure 1, it was
not easy to see that this model
might not be the best possible
that we could find.
By plotting the two estimated models with their CIs in
the same plot in Figure 3, we
can better see the subtle differences that distinguish them.
Third, it is helpful- when
possible-to report several
alternatives. For some of the
cases involving reliability that
I have worked on, the worstcase reliability from all of the
leading models is presented
as an overall lower bound for
possible reliability.
This can be a helpful bound
when the consequences of
an error in overestimating
reliability are large. Alternate
strategies in the statistics literature for acknowledging and

NEW STATISTICS BOOK A new ASQ Quality Press book features a
collection of 90 Statistics Roundtable columns previously published
in QP. Statistical Roundtables, edited by Christine Anderson-Cook
and Lu Lu, is broken into nine chapters on topics such as statistical
engineering, data quality and measurement, data collection and key
statistical tools. The 480-page book costs $60 for ASQ members
and $99 for nonmembers. For more details, visit http://tinyurl.com/
stats-roundtables.

incorporating model uncertainty include Bayesian model
averaging5 and propagating
model uncertainty.6

One more model

Another common version of
model uncertainty occurs
when we have multiple explanatory variables. In this case, we
may have several models using
different subsets of the explanatory variables that perform
similarly well.
Here, the risk of choosing
a single model and ignoring
other contenders is potentially
even greater. If we dismiss
an explanatory variable from
further consideration, we risk
losing track of a potential
mechanism that might be driving changes in our response.
Christine M. Anderson-Cook,
Jerome Morzinski and Kenneth
D. Blecker describe a process
for considering multiple models and identifying a subset of
leading candidates.7
A final comment about
different types of uncertainty:
When we are designing experiments, it is important to build
in the ability to assess the
quality of the fit of our model.8
If we only design our experiment to perform well for the
assumed model, and it turns
that the model is incorrect
and perhaps too simplistic,
a poorly chosen experiment

might not allow us to discover
the mistake.
Choosing a well-designed
experiment to allow for
adequate balance between
good estimating if the model
is correct and protection if the
model is wrong, as well as the
capability for checking lack of
fit, is a large topic for discussion.9, 10
Imagine if our engineer had
not had the ability to explore
the cubic model. This could
have hidden this source of
uncertainty from further investigation and led to suboptimal
conclusions.
REFERENCES AND NOTE
1. For more details about the
difference between confidence
and prediction intervals, see
Christine M. Anderson-Cook,
"Interval Training: Answering
the Right Question With the
Right Interval," Quality Progress,
October 2009, pp. 58-60.
2. Douglas C. Montgomery,
Elizabeth A. Peck and G. Geoff rey
Vining, Introduction to Linear
Regression Analysis, third edition,
Wiley, 2001, pp. 152-154.
3. Christine M. Anderson-Cook,
Yongtao Cao and Lu Lu, "Maximize,
Minimize or Target," Quality
Progress, April 2016, pp. 52-55.

4. Montgomery, Introduction to
Linear Regression Analysis, see
reference 2.
5. Jennifer A. Hoeting, David
Madigan, Adrian E. Raftery and
Chris T. Volinsky, "Bayesian Model
Averaging: A Tutorial," Statistical
Science, Vol. 14, No. 4, 1999, pp.
382-401.
6. David Draper, "Assessment
and Propagation of Model
Uncertainty," Journal of the Royal
Statistical Society B, Vol. 57, No. 1,
1995, pp. 45-97.
7. Christine M. AndersonCook, Jerome Morzinski and
Kenneth D. Blecker, "Statistical
Model Selection for Better
Prediction and Discovering
Science Mechanism That Affect
Reliability" Systems, Vol. 3, No. 3,
2015, pp. 109-132.
8. Christine M. Anderson-Cook,
"A Matter of Trust: Balance
Confidence in Your Model
While Avoiding Pitfalls," Quality
Progress, March 2010, pp. 56-58.
9. Lu Lu, Christine M. AndersonCook, Timothy J. Robinson,
"Optimization of Designed
Experiments Based on Multiple
Criteria Utilizing a Pareto
Frontier," Technometrics, Vol. 53,
No. 4, 2011, pp. 353-365.
10. Lu Lu, Christine M. AndersonCook, "Rethinking the Optimal
Response Surface Design
for a First-Order Model With
Two-Factor Interactions, When
Protecting Against Curvature,"
Quality Engineering, Vol. 24, No.
3, 2012, pp. 404-422.

Christine M. Anderson-Cook is a research
scientist in the Statistical Sciences Group
at Los Alamos National Laboratory in Los
Alamos, NM. She earned a doctorate in
statistics from the University of Waterloo
in Ontario. Anderson-Cook is a fellow
of ASQ and the American Statistical
Association.

qualityprogress.com ❘ March 2017

QP 47


http://www.tinyurl.com/ http://www.qualityprogress.com

Table of Contents for the Digital Edition of Quality Progress - March 2017

Seen and Heard
Expert Answers
Progress Report
Mr. Pareto Head
Field Notes
Innovation Imperative
Hard Wired
Propel Forward
Life After Disruption
Work Smarter, Not Harder
Statistics Spotlight
Standard Issues
Marketplace
Footnotes
Back to Basics
Quality Progress - March 2017 - cover1
Quality Progress - March 2017 - cover2
Quality Progress - March 2017 - 1
Quality Progress - March 2017 - 2
Quality Progress - March 2017 - 3
Quality Progress - March 2017 - 4
Quality Progress - March 2017 - 5
Quality Progress - March 2017 - Seen and Heard
Quality Progress - March 2017 - 7
Quality Progress - March 2017 - Expert Answers
Quality Progress - March 2017 - 9
Quality Progress - March 2017 - Progress Report
Quality Progress - March 2017 - 11
Quality Progress - March 2017 - 12
Quality Progress - March 2017 - Mr. Pareto Head
Quality Progress - March 2017 - Field Notes
Quality Progress - March 2017 - 15
Quality Progress - March 2017 - 16
Quality Progress - March 2017 - 17
Quality Progress - March 2017 - Innovation Imperative
Quality Progress - March 2017 - 19
Quality Progress - March 2017 - 20
Quality Progress - March 2017 - 21
Quality Progress - March 2017 - Hard Wired
Quality Progress - March 2017 - 23
Quality Progress - March 2017 - 24
Quality Progress - March 2017 - 25
Quality Progress - March 2017 - 26
Quality Progress - March 2017 - 27
Quality Progress - March 2017 - Propel Forward
Quality Progress - March 2017 - 29
Quality Progress - March 2017 - 30
Quality Progress - March 2017 - 31
Quality Progress - March 2017 - 32
Quality Progress - March 2017 - 33
Quality Progress - March 2017 - Life After Disruption
Quality Progress - March 2017 - 35
Quality Progress - March 2017 - 36
Quality Progress - March 2017 - 37
Quality Progress - March 2017 - 38
Quality Progress - March 2017 - 39
Quality Progress - March 2017 - Work Smarter, Not Harder
Quality Progress - March 2017 - 41
Quality Progress - March 2017 - 42
Quality Progress - March 2017 - 43
Quality Progress - March 2017 - Statistics Spotlight
Quality Progress - March 2017 - 45
Quality Progress - March 2017 - 46
Quality Progress - March 2017 - 47
Quality Progress - March 2017 - Standard Issues
Quality Progress - March 2017 - 49
Quality Progress - March 2017 - Marketplace
Quality Progress - March 2017 - 51
Quality Progress - March 2017 - Footnotes
Quality Progress - March 2017 - 53
Quality Progress - March 2017 - 54
Quality Progress - March 2017 - 55
Quality Progress - March 2017 - Back to Basics
Quality Progress - March 2017 - cover3
Quality Progress - March 2017 - cover4
https://www.nxtbook.com/naylor/ASQM/ASQM0719
https://www.nxtbook.com/naylor/ASQM/ASQM0619
https://www.nxtbook.com/naylor/ASQM/ASQM0519
https://www.nxtbook.com/naylor/ASQM/ASQM0419
https://www.nxtbook.com/naylor/ASQM/ASQM0319
https://www.nxtbook.com/naylor/ASQM/ASQM0219
https://www.nxtbook.com/naylor/ASQM/ASQM0119
https://www.nxtbook.com/naylor/ASQM/ASQM1218
https://www.nxtbook.com/naylor/ASQM/ASQM1118
https://www.nxtbook.com/naylor/ASQM/ASQM1018
https://www.nxtbook.com/naylor/ASQM/ASQM0918
https://www.nxtbook.com/naylor/ASQM/ASQM0818
https://www.nxtbook.com/naylor/ASQM/ASQM0718
https://www.nxtbook.com/naylor/ASQM/ASQM0618
https://www.nxtbook.com/naylor/ASQM/ASQM0518
https://www.nxtbook.com/naylor/ASQM/ASQM0418
https://www.nxtbook.com/naylor/ASQM/ASQM0318
https://www.nxtbook.com/naylor/ASQM/ASQM0218
https://www.nxtbook.com/naylor/ASQM/ASQM0118
https://www.nxtbook.com/naylor/ASQM/ASQM1217
https://www.nxtbook.com/naylor/ASQM/ASQM1117
https://www.nxtbook.com/naylor/ASQM/ASQM1017
https://www.nxtbook.com/naylor/ASQM/ASQM0917
https://www.nxtbook.com/naylor/ASQM/ASQM0817
https://www.nxtbook.com/naylor/ASQM/ASQM0717
https://www.nxtbook.com/naylor/ASQM/ASQM0617
https://www.nxtbook.com/naylor/ASQM/ASQM0517
https://www.nxtbook.com/naylor/ASQM/ASQM0417
https://www.nxtbook.com/naylor/ASQM/ASQC12518
https://www.nxtbook.com/naylor/ASQM/ASQM0317
https://www.nxtbook.com/naylor/ASQM/ASQM0217
https://www.nxtbook.com/naylor/ASQM/ASQM0117
https://www.nxtbook.com/naylor/ASQM/ASQM1216
https://www.nxtbook.com/naylor/ASQM/ASQM1116
https://www.nxtbook.com/naylor/ASQM/ASQM1016
https://www.nxtbook.com/naylor/ASQM/ASAC0016
https://www.nxtbook.com/naylor/ASQM/ASQM0916
https://www.nxtbook.com/naylor/ASQM/ASQA0016
https://www.nxtbook.com/naylor/ASQM/ASQM0816
https://www.nxtbook.com/naylor/ASQM/ASQM0716
https://www.nxtbook.com/naylor/ASQM/ASQM0616
https://www.nxtbook.com/naylor/ASQM/ASQM0516
https://www.nxtbook.com/naylor/ASQM/ASQM0416
https://www.nxtbook.com/naylor/ASQM/ASQM0316
https://www.nxtbook.com/naylor/ASQM/ASQM0216
https://www.nxtbook.com/naylor/ASQM/ASQM0116
https://www.nxtbook.com/naylor/ASQM/ASQM1215
https://www.nxtbook.com/naylor/ASQM/ASQM1115
https://www.nxtbook.com/naylor/ASQM/ASQM1015
https://www.nxtbook.com/naylor/ASQM/ASQM0915
https://www.nxtbook.com/naylor/ASQM/ASQM0815
https://www.nxtbook.com/naylor/ASQM/ASQM0715
https://www.nxtbook.com/naylor/ASQM/ASQM0615
https://www.nxtbook.com/naylor/ASQM/ASQM0515
https://www.nxtbook.com/naylor/ASQM/ASQM0315
https://www.nxtbook.com/naylor/ASQM/ASQM0215
https://www.nxtbook.com/naylor/ASQM/ASQM0115
https://www.nxtbook.com/naylor/ASQM/ASQM1214
https://www.nxtbook.com/naylor/ASQM/ASQM1114
https://www.nxtbook.com/naylor/ASQM/ASQM1014
https://www.nxtbook.com/naylor/ASQM/ASQM0914
https://www.nxtbook.com/naylor/ASQM/ASQM0814
https://www.nxtbook.com/naylor/ASQM/ASQM0714
https://www.nxtbook.com/naylor/ASQM/ASQM0614
https://www.nxtbook.com/naylor/ASQM/ASQM0514
https://www.nxtbook.com/naylor/ASQM/ASQM0414
https://www.nxtbook.com/naylor/ASQM/ASQM0314
https://www.nxtbook.com/naylor/ASQM/ASQM0214
https://www.nxtbook.com/naylor/ASQM/ASQM0114
https://www.nxtbook.com/naylor/ASQM/ASQM1213
https://www.nxtbook.com/naylor/ASQM/ASQM1113
https://www.nxtbook.com/naylor/ASQM/ASQM1013
https://www.nxtbook.com/naylor/ASQM/ASQM0913
https://www.nxtbook.com/naylor/ASQM/ASQM0813
https://www.nxtbook.com/naylor/ASQM/ASQM0713
https://www.nxtbook.com/naylor/ASQM/ASQM0613
https://www.nxtbook.com/naylor/ASQM/ASQM0513
https://www.nxtbook.com/naylor/ASQM/ASQM0413
https://www.nxtbook.com/naylor/ASQM/ASQM0313
https://www.nxtbook.com/nxtbooks/naylor/ASQM0213
https://www.nxtbook.com/nxtbooks/naylor/ASQM0113
https://www.nxtbook.com/nxtbooks/naylor/ASQM1212
https://www.nxtbook.com/nxtbooks/naylor/ASQM1112
https://www.nxtbook.com/nxtbooks/naylor/ASQM1012
https://www.nxtbook.com/nxtbooks/naylor/ASQM0912
https://www.nxtbookmedia.com