Training Industry Magazine - March/April 2017 - 48

MEASURING
I M P A C T

THE KEY TO
QUALITY COMMENTS
I S ASK ING THE R I G H T QU E ST I ON S
BY DR. REANNA PONCHERI HARMAN

Anyone who has participated in a training
event is familiar with open-ended survey items
like this one: "Please provide any additional
comments you have about the training program
you just completed."
After getting into the rhythm of clicking bubble
after bubble in response to closed-ended survey
items, many trainees come to a roadblock when
provided with a blank box of space and asked
to provide feedback in their own words. While
trainees tend to complete all or most of the
closed-ended items on organizational surveys,
the same is not true for open-ended items. A
recently published series of studies investigated
responses to open-ended comments on training
evaluation surveys and found that 28 to 53
percent of trainees provided a response to openended items.
Reaction measures are widely used in the
training industry for evaluation and are typically
measured on surveys that include both closedended items and open-ended items. But, why are
there less responses to open-ended items and
what are the practical implications?
At the annual meeting of the Society for Industrial
and Organizational Psychology in 2007, Poncheri
and Thompson presented a study that openended items are viewed as less important than
closed-ended items on web-based surveys.
Additionally, open-ended items carry a higher
response burden for trainees when compared to
closed-ended items because it takes more effort

| 48

for the trainee to respond. If trainees think that
providing comments is not important or that it
takes a lot of effort to respond, the combination
could be enough to discourage response.
Despite what trainees may perceive, training
managers and other decision makers highly
value input provided in response to open-ended
items because the comments are in the trainees'
own words. Comments are very rich and can be
very powerful. Even a highly negative, critical
comment may sway a decision-maker to cancel
or make significant changes to a program, while
highly positive comments can provide value to
market training or justify additional investment.
Because of their value, comments should be
interpreted with care. Research shows that
comments are negative in tone and that
dissatisfied trainees tend to provide comments.
Comments most likely do not represent the
opinions of all trainees, but may reflect input
from the most dissatisfied trainees. Training
managers who read comments should interpret
them appropriately as they may only be hearing
from the dissatisfied trainees.
Knowing how to get the most out of comments
can provide key strategic advantages
for organizations if used and interpreted
appropriately. Here are four evidence-based tips
for training managers and other key decisionmakers to improve training survey practices and
to better leverage the feedback provided.

http://psycnet.apa.org/journals/apl/100/3/894/ http://journals.sagepub.com/doi/abs/10.1177/1094428106295504

Table of Contents for the Digital Edition of Training Industry Magazine - March/April 2017