Training Industry Magazine - July/August 2019 - 49

ARE INSTRUCTORS STILL RELEVANT?
With so much focus on asynchronous,
technology-delivered learning, it is
understandable to question whether
instructors and instructor-led training (ILT)
are still relevant. The short answer is yes!

recent research (What Learners Want:
Strategies for Training Delivery) found that
63% and 28% of learners, respectively,
participated in at least one ILT course and
in at least one VILT in the past 12 months.

Approximately 67% of formal learning
hours available in 2017 were instructor-led
(53% traditional, 9% virtual and 5% nononline remote classroom), according to
ATD research. Training Industry research
concurs, finding on average companies
deliver 64% of their training portfolios via
ILT (39%) or virtual ILT (VILT; 25%). Other

Training Industry research found that,
over the next 12 months, 21% and 31%
of companies plan to increase their
use of ILT and VILT, while only 10% and
8% plan to decrease their use. Thus, we
see a place for ILT and VILT in training
portfolios and a role for instructors into
the foreseeable future.

Now,
we
successfully
answered
the question, "How do we improve
learning, graduation rates, and program
effectiveness?" and provided a mechanism
to use formative evaluation, analytics and
feedback to drive improvement.
Over time, instructor performance and
effectiveness increased, and variability in
instructor performance decreased. Thus,
the program's effectiveness increased,
producing more capability.

INSTRUCTOR-FOCUSED
FORMATIVE EVALUATIONS
AND ANALYTICS ARE
TYPICALLY NEGLECTED.
ARE YOU READY TO TRY
THIS APPROACH?
Formative evaluation focused on levers,
such as instructor performance, can drive
continuous improvement and optimize
the learning process and its outcomes.
Every L&D program is different, so tailor
the process as needed and let your
findings guide its implementation. Before
you get started, however, it is important to
do the prep-work:
* Ask if the training program is meeting
its objectives. Asking questions about

effectiveness allows stakeholders to
identify gaps between actual and desired
outcomes linked to their roles and
objectives. Prioritize outcomes desired
by multiple stakeholders. If there are no
gaps, stop. If stakeholders are satisfied
with current performance, stop.
* Ask if there is opportunity for
improvement. Then, determine if
improvement is possible given the
context, stakeholders' cooperation
and the outcome's measurement. If
not, stop.
* Develop
questions
related
to
improvement, such as "How can I
impact the focal outcome?" or "What
factors drive the focal outcome?"
Training effectiveness research and
models identify factors that typically
influence
learning
outcomes.
Statistical techniques can identify
sources that influence outcome
measures to narrow the candidates.
Instructor performance is just one
potential factor. Select factors to
investigate that are easily measured.
* Develop and pilot metrics for the
selected factors, choosing the
most appropriate data sources,
measurement methods and collection
times to test the impact on the focal
outcome. Determine if the metrics
function as designed, meeting both
validity and reliability standards. If
not, repeat until they do.

* Collect and analyze data on these
metrics along with learning outcomes.
Determine if there is a relationship
between the factor(s) and learning
outcome(s). If not, stop.
* Determine if the factor is suitable
to be used in an intervention. Is the
factor actionable? Does the factor's
measurement occur before the focal
outcome's measurement? Is there
time for a change in the factor to
impact the outcome? Determine if the
evidence supports use of the factor as
an intervention. If not, stop.
* Develop and implement an analytics
intervention to improve the relevant
factors and associated outcomes.
Evaluate and adjust over time.

FINAL THOUGHTS
Our case demonstrates the "two questions"
approach in driving evaluation, analytics
and feedback practice. Specifically, it
provides an example of how instructor
performance was identified as a key lever
impacting learning, and how instructor
performance measurement, analytics and
feedback were used to improve instruction
and its impact on learning outcomes.
Instructors are impact multipliers
through their influence on learners
and need insights, tools and support to
maximize their impact. Analytics help
supervisors have timely performance
conversations, coach instructors and
provide support based on data and
insights. Ultimately, analytics and
development tools provide instructors
agency over their professional and
career development. Timely, analyticsbased feedback empowers instructors
to adjust their practice in process,
sharpen their craft and create more
value for themselves, their learners and
their employers.
Dr. Eric Surface is CEO and Dr. Reanna Harman
is VP for Practice at ALPS Insights. They have
35 years of combined L&D and consulting
experience. ALPS Insights provides L&D
evaluation, analytics and insights through
its software platform, ALPS IbexTM, as well as
consulting and analytics services. Email Eric
and Reanna.

T R A I N I N G I N DUSTR Y MAGAZ INE -LEARNING ANALYTICS 20 19 I WWW. T RAININGINDU S T RY . C OM/ MAGAZ I NE

| 49


https://www2.trainingindustry.com/what-learners-want-strategies-for-training-delivery https://www2.trainingindustry.com/what-learners-want-strategies-for-training-delivery http://www.alpsinsights.com https://www.trainingindustry.com/magazine

Training Industry Magazine - July/August 2019

Table of Contents for the Digital Edition of Training Industry Magazine - July/August 2019

Learning Analytics: Table Stakes for Business Success
Table of Contents
Let's Build a Measurement Practice for L&D
Brain-Based Learning Analytics and Evaluation Practices
Who [Else] Cares About Learning Impact Data
Justifying the Investment in Leadership Training
Is Training Analytics Smoke and Mirrors?
Leveraging Analytical Tools to Transform L&D
Levearging Learning Analytics to Improve Business Outcomes: Where to Start
Was It Worth It? Measuring the Impact and ROI of Leadership Training
How to Make the Transition to a Data-Driven Learning Culture
How Personal Skills AI Assistants Might Disrupt Corporate Training
Machines Are the Future of Training: How Data-Driven Feedback Is Fostering Improvement and Enhancing Human Relationships
Improving Instructor Impact on Learning with Analytics
A Year in the Life: Evolving L&D Metrics From a Reporting Managed Service to Business Insight Partner
Localizing E-Learning Programs for an International Audience
Learning Analytics Is More Than Data Collection
Using Business Data to Drive Training Objectives
Quality versus Quantity: How Analytics Can Improve Training Effectiveness
Coursera Reaches Unicorn Status and Accelerates Enterprise Product Innovation
Company News
Training Industry Magazine - July/August 2019 - Cover1
Training Industry Magazine - July/August 2019 - 2
Training Industry Magazine - July/August 2019 - 3
Training Industry Magazine - July/August 2019 - 4
Training Industry Magazine - July/August 2019 - Learning Analytics: Table Stakes for Business Success
Training Industry Magazine - July/August 2019 - Table of Contents
Training Industry Magazine - July/August 2019 - 7
Training Industry Magazine - July/August 2019 - 8
Training Industry Magazine - July/August 2019 - 9
Training Industry Magazine - July/August 2019 - 10
Training Industry Magazine - July/August 2019 - Let's Build a Measurement Practice for L&D
Training Industry Magazine - July/August 2019 - 12
Training Industry Magazine - July/August 2019 - Brain-Based Learning Analytics and Evaluation Practices
Training Industry Magazine - July/August 2019 - 14
Training Industry Magazine - July/August 2019 - Who [Else] Cares About Learning Impact Data
Training Industry Magazine - July/August 2019 - 16
Training Industry Magazine - July/August 2019 - Justifying the Investment in Leadership Training
Training Industry Magazine - July/August 2019 - Is Training Analytics Smoke and Mirrors?
Training Industry Magazine - July/August 2019 - 19
Training Industry Magazine - July/August 2019 - 20
Training Industry Magazine - July/August 2019 - 21
Training Industry Magazine - July/August 2019 - Leveraging Analytical Tools to Transform L&D
Training Industry Magazine - July/August 2019 - 23
Training Industry Magazine - July/August 2019 - 24
Training Industry Magazine - July/August 2019 - 25
Training Industry Magazine - July/August 2019 - Levearging Learning Analytics to Improve Business Outcomes: Where to Start
Training Industry Magazine - July/August 2019 - 27
Training Industry Magazine - July/August 2019 - 28
Training Industry Magazine - July/August 2019 - 29
Training Industry Magazine - July/August 2019 - 30
Training Industry Magazine - July/August 2019 - 31
Training Industry Magazine - July/August 2019 - 32
Training Industry Magazine - July/August 2019 - Was It Worth It? Measuring the Impact and ROI of Leadership Training
Training Industry Magazine - July/August 2019 - 34
Training Industry Magazine - July/August 2019 - 35
Training Industry Magazine - July/August 2019 - How to Make the Transition to a Data-Driven Learning Culture
Training Industry Magazine - July/August 2019 - 37
Training Industry Magazine - July/August 2019 - How Personal Skills AI Assistants Might Disrupt Corporate Training
Training Industry Magazine - July/August 2019 - 39
Training Industry Magazine - July/August 2019 - 40
Training Industry Magazine - July/August 2019 - 41
Training Industry Magazine - July/August 2019 - Machines Are the Future of Training: How Data-Driven Feedback Is Fostering Improvement and Enhancing Human Relationships
Training Industry Magazine - July/August 2019 - 43
Training Industry Magazine - July/August 2019 - 44
Training Industry Magazine - July/August 2019 - 45
Training Industry Magazine - July/August 2019 - 46
Training Industry Magazine - July/August 2019 - Improving Instructor Impact on Learning with Analytics
Training Industry Magazine - July/August 2019 - 48
Training Industry Magazine - July/August 2019 - 49
Training Industry Magazine - July/August 2019 - A Year in the Life: Evolving L&D Metrics From a Reporting Managed Service to Business Insight Partner
Training Industry Magazine - July/August 2019 - 51
Training Industry Magazine - July/August 2019 - Localizing E-Learning Programs for an International Audience
Training Industry Magazine - July/August 2019 - 53
Training Industry Magazine - July/August 2019 - 54
Training Industry Magazine - July/August 2019 - Learning Analytics Is More Than Data Collection
Training Industry Magazine - July/August 2019 - 56
Training Industry Magazine - July/August 2019 - Using Business Data to Drive Training Objectives
Training Industry Magazine - July/August 2019 - 58
Training Industry Magazine - July/August 2019 - Quality versus Quantity: How Analytics Can Improve Training Effectiveness
Training Industry Magazine - July/August 2019 - Coursera Reaches Unicorn Status and Accelerates Enterprise Product Innovation
Training Industry Magazine - July/August 2019 - Company News
Training Industry Magazine - July/August 2019 - Cover4
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20201112
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20200910
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20200708
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20200506
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20200304
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20200102
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20191112
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20190910
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20190708
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20190506
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20190304
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20190102
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20181112
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20180910
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20180708
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20180506
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20180304
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20180102
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20171112
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20170910
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20170708
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20170506
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20170304
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20171112_se
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20161112
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2016fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2016summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2016sales
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2016spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2016winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2015outsourcing
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2015fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2015summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2015leadership
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2015spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2015winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2014fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2014summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2014spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2014winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2013fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2013summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2013spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2013winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2012fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2012summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2012spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2012winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2011fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2011summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2011spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2011winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2010fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2010summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2010spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2010winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2009fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2009summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2009spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2009winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2008fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2008summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2008spring
https://www.nxtbookmedia.com