Training Industry Magazine - July/August 2019 - 34

Using this model (see Figure 1), we can
essentially tell the extent of the training's
impact - from learners' participation in
the program to business results to ROI to
and how to improve the training impact
in the future. Additionally, by creating
this comprehensive, connected and
cohesive picture of impact, we would
be able to show how valuable it was (in
quantifiable terms) for business leaders
and stakeholders to send their new
leaders through the training early in their
role, as opposed to later, which typically
happens when the everyday demands of
the business seem to take priority.

participants' manager and direct reports.
This was to compare and corroborate that
these leadership behaviors did, in fact,
improve over the post-training period.

Develop leaders who
inspire people to perform
at a higher level and
increase organizational
productivity and profits.

The Training
The leadership training was delivered
to over 2,000 first-time managers to
help them transition from individual
contributors to people leaders, and to
help them ramp up their leadership
skills as early as possible in their new
roles. These skills include coaching
their frontline employees, aligning
their teams to organizational priorities
and recognizing the right behaviors.
These managers could be in their role
anywhere from just a few months to
more than a year before they attended
the training, which is why we wanted
to prove the value of early attendance.
Ultimately, the objective is to develop
leaders who inspire people to perform
at a higher level and, therefore, increase
organizational productivity and profits.

How We Measured Impact
Level 1: A survey was administered to
participants immediately after training,
asking participants to rate various factors
like instructor effectiveness, course
content and relevance to their job.
Level 2: We added several learning
questions to the same post-training
survey. Here, we asked participants
whether they gained new knowledge and
skills that were relevant for their roles.
Level 3: We used multi-rater feedback.
First, we administered a survey three
months post-training that asked
participants to rate the improvement
(if any) they observed in very specific
leadership behaviors. We then gave
these same core questions to both the

| 34

Level 4: To define and isolate the impact
training had on the business, we used a
control group design. That is, we compared
the performance improvements of a
trained group of leaders who had been
through the training (within their first
three to six months as a new leader) and
a twin sample or control group of leaders
who had not yet attended the training
(also within their first three to six months
in their role). For each group, we tracked
their performance three months before the
training and three months after training
to see if there was any improvement. The
control group of untrained leaders was

tracked during the same timeframe, and
all other factors were controlled for (e.g.
tenure, location, team size).
Level 5: To measure ROI, we turned the
performance benefits we found at Level
4 into a dollar value (only the metrics we
could monetize), and then compared
them to the fully loaded cost of training.
Level 6: To understand which factors
impacted ROI, we added several
questions to the original survey asking
participants about their immediate work
environments, such as "how supportive
was your immediate manager when you
tried to apply all your learning?" Answers
to these questions were then analyzed
and correlated to the amount of impact
the training had on their behavioral
improvements and performance.

The Results
Level 1: Satisfaction with training: The
average rating was 4.73 out 5.00 stars.
Level 2: Learning new knowledge and
skills: 97% of participants gained new and
valuable knowledge or skills for their job.
Level 3: Behavioral change on the job:
94% of participants showed "some" to

Figure 1: Level 1-6 Training Measurement Model

6
5

Climates that Maximize?
Return on Investment?

4

Impact the Business?

3

Change Behavior?

2
1

Learn Anything?
Like It?



Training Industry Magazine - July/August 2019

Table of Contents for the Digital Edition of Training Industry Magazine - July/August 2019

Learning Analytics: Table Stakes for Business Success
Table of Contents
Let's Build a Measurement Practice for L&D
Brain-Based Learning Analytics and Evaluation Practices
Who [Else] Cares About Learning Impact Data
Justifying the Investment in Leadership Training
Is Training Analytics Smoke and Mirrors?
Leveraging Analytical Tools to Transform L&D
Levearging Learning Analytics to Improve Business Outcomes: Where to Start
Was It Worth It? Measuring the Impact and ROI of Leadership Training
How to Make the Transition to a Data-Driven Learning Culture
How Personal Skills AI Assistants Might Disrupt Corporate Training
Machines Are the Future of Training: How Data-Driven Feedback Is Fostering Improvement and Enhancing Human Relationships
Improving Instructor Impact on Learning with Analytics
A Year in the Life: Evolving L&D Metrics From a Reporting Managed Service to Business Insight Partner
Localizing E-Learning Programs for an International Audience
Learning Analytics Is More Than Data Collection
Using Business Data to Drive Training Objectives
Quality versus Quantity: How Analytics Can Improve Training Effectiveness
Coursera Reaches Unicorn Status and Accelerates Enterprise Product Innovation
Company News
Training Industry Magazine - July/August 2019 - Cover1
Training Industry Magazine - July/August 2019 - 2
Training Industry Magazine - July/August 2019 - 3
Training Industry Magazine - July/August 2019 - 4
Training Industry Magazine - July/August 2019 - Learning Analytics: Table Stakes for Business Success
Training Industry Magazine - July/August 2019 - Table of Contents
Training Industry Magazine - July/August 2019 - 7
Training Industry Magazine - July/August 2019 - 8
Training Industry Magazine - July/August 2019 - 9
Training Industry Magazine - July/August 2019 - 10
Training Industry Magazine - July/August 2019 - Let's Build a Measurement Practice for L&D
Training Industry Magazine - July/August 2019 - 12
Training Industry Magazine - July/August 2019 - Brain-Based Learning Analytics and Evaluation Practices
Training Industry Magazine - July/August 2019 - 14
Training Industry Magazine - July/August 2019 - Who [Else] Cares About Learning Impact Data
Training Industry Magazine - July/August 2019 - 16
Training Industry Magazine - July/August 2019 - Justifying the Investment in Leadership Training
Training Industry Magazine - July/August 2019 - Is Training Analytics Smoke and Mirrors?
Training Industry Magazine - July/August 2019 - 19
Training Industry Magazine - July/August 2019 - 20
Training Industry Magazine - July/August 2019 - 21
Training Industry Magazine - July/August 2019 - Leveraging Analytical Tools to Transform L&D
Training Industry Magazine - July/August 2019 - 23
Training Industry Magazine - July/August 2019 - 24
Training Industry Magazine - July/August 2019 - 25
Training Industry Magazine - July/August 2019 - Levearging Learning Analytics to Improve Business Outcomes: Where to Start
Training Industry Magazine - July/August 2019 - 27
Training Industry Magazine - July/August 2019 - 28
Training Industry Magazine - July/August 2019 - 29
Training Industry Magazine - July/August 2019 - 30
Training Industry Magazine - July/August 2019 - 31
Training Industry Magazine - July/August 2019 - 32
Training Industry Magazine - July/August 2019 - Was It Worth It? Measuring the Impact and ROI of Leadership Training
Training Industry Magazine - July/August 2019 - 34
Training Industry Magazine - July/August 2019 - 35
Training Industry Magazine - July/August 2019 - How to Make the Transition to a Data-Driven Learning Culture
Training Industry Magazine - July/August 2019 - 37
Training Industry Magazine - July/August 2019 - How Personal Skills AI Assistants Might Disrupt Corporate Training
Training Industry Magazine - July/August 2019 - 39
Training Industry Magazine - July/August 2019 - 40
Training Industry Magazine - July/August 2019 - 41
Training Industry Magazine - July/August 2019 - Machines Are the Future of Training: How Data-Driven Feedback Is Fostering Improvement and Enhancing Human Relationships
Training Industry Magazine - July/August 2019 - 43
Training Industry Magazine - July/August 2019 - 44
Training Industry Magazine - July/August 2019 - 45
Training Industry Magazine - July/August 2019 - 46
Training Industry Magazine - July/August 2019 - Improving Instructor Impact on Learning with Analytics
Training Industry Magazine - July/August 2019 - 48
Training Industry Magazine - July/August 2019 - 49
Training Industry Magazine - July/August 2019 - A Year in the Life: Evolving L&D Metrics From a Reporting Managed Service to Business Insight Partner
Training Industry Magazine - July/August 2019 - 51
Training Industry Magazine - July/August 2019 - Localizing E-Learning Programs for an International Audience
Training Industry Magazine - July/August 2019 - 53
Training Industry Magazine - July/August 2019 - 54
Training Industry Magazine - July/August 2019 - Learning Analytics Is More Than Data Collection
Training Industry Magazine - July/August 2019 - 56
Training Industry Magazine - July/August 2019 - Using Business Data to Drive Training Objectives
Training Industry Magazine - July/August 2019 - 58
Training Industry Magazine - July/August 2019 - Quality versus Quantity: How Analytics Can Improve Training Effectiveness
Training Industry Magazine - July/August 2019 - Coursera Reaches Unicorn Status and Accelerates Enterprise Product Innovation
Training Industry Magazine - July/August 2019 - Company News
Training Industry Magazine - July/August 2019 - Cover4
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20201112
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20200910
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20200708
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20200506
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20200304
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20200102
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20191112
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20190910
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20190708
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20190506
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20190304
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20190102
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20181112
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20180910
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20180708
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20180506
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20180304
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20180102
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20171112
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20170910
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20170708
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20170506
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20170304
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20171112_se
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20161112
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2016fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2016summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2016sales
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2016spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2016winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2015outsourcing
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2015fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2015summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2015leadership
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2015spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2015winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2014fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2014summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2014spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2014winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2013fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2013summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2013spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2013winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2012fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2012summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2012spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2012winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2011fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2011summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2011spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2011winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2010fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2010summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2010spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2010winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2009fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2009summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2009spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2009winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2008fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2008summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2008spring
https://www.nxtbookmedia.com