Training Industry Magazine - July/August 2019 - 51

FIGURE 1

the course or event, we had standard
metrics across the board:

WHAT IS MEASURED?

GFPR measurements align to Phillips Levels of Evaluation

ROI
BUSINESS
IMPACT

What were program benefits compared to costs?
What business measures were affected?

APPLICATION

LEARNING

REACTION/SATISFACTION
PARTICIPATION/COMPLIANCE

EXTERNAL FOCUS
Over the next year, we balanced completion
and compliance metrics while maturing the
reaction and satisfaction, application, and
business impact measurements - knowing
that in the future we could stand by return
on investment (ROI) measurements.
Changing management served as a key
aspect of our external focus. Changing
the formerly referred to reporting team
to the measurement and analytics team
was significant. To our enablement
organization, this change clarified our
intention to be a partner in the business. To
other industry professionals, this indicated
the level of seriousness our enablement
organization brought to the table. On each
conference call we attended, we no longer
used reporting. Our team scheduled lunchand-learns and attended other team calls to
represent clearly who we were, our mission
and how to engage our support.
SUCCESSES
We knew that as an enablement
organization and as a team, we were
moving toward deeper, more insightful
learning measurement. Each small
victory lead to our larger victories
chronicled below:
Implementation of measurement levels
across enablement organization: The
implementation of Phillips' measurement
methodology was, and remains, vital. The
pyramid of measurement levels became
the symbol for the intended journey both

Is on-the-job behavior changed?
Were knowledge and skills learned?
Was content relevant and
valuable?
Did target audience
complete?

internally and externally. In conversations
with field leaders, we communicate
our intention of providing value to and
partnering with the business.
Introduction of measurable objectives:
Fundamental to our efforts, and helped
in great part by team leaders and partner
centers of excellence, was the demand for
more detailed objectives when planning
for a course, quarterly plans and yearly
goals. Our team provided examples and
worked with teams to turn needs into clear,
measurable objectives. This alone helped
our other enablement teams be clear on
expectations as we planned, as well as
during quarterly business reviews where
we detailed what was accomplished.

OUR TEAM'S FOCUS
WAS TO GUIDE THE
METRICS MATURITY OF
OUR ORGANIZATION.
Standardization of surveys: Although
surveys had been developed, distributed
and reported on previously, there was
no uniformity. Creating survey templates
was among our first tasks as a team. Both
our team and enablement operations
owned creating in the survey platform. We
identified key metrics for measurement.
This way, regardless of the target audience,

* Net promotor score (NPS)
* Confidence change
* Relevancy to role
* Anticipated business impact
Progress from reporting manage service
to a consultative business partner:
Our team's efforts included layering
the beginning measurement level of
completion and compliance with survey
data, and then guiding the enablement
leads towards identifying the behaviors that
would indicate application had occurred.
In many cases, we were able to connect
those to business changes expected by
field leaders. In one instance, we leveraged
data related to the value selling workshops
held in order to enable a methodology
and tool for our sales force. We combined
the available workshop completions with
survey metrics, monitored tool usage for
an application metric across teams and
worked with an extended team to average
deal size to show impact.
With our corporate business insight team,
we partnered to move the measurement of
this program from Level 1 to Level 4. With the
business impact data, our team progressed
through the levels of measurement to
go beyond reporting. We, along with
enablement leads who represented various
participant groups, could analyze these
results for changes over time, achievement
of targets set and correlate the enablement
to the business through these results. These
accomplishments represent our position
as a true partner to our enablement leads
as they seek to satisfy the demands of the
business and their field leaders.
While the team now consists of additional
members and the enablement organization
changes constantly, we continue in the
right direction, with our methodology
grounding our focus to measure and
analyze - not only report.
Erika Robertson has been in the tech space for
12 years, focusing on sales force enablement.
Achieving the Certified ROI Professional (CRP)
designation from ROI Institute allowed her
to refine her instructional design further to
measurement and analytics. Email Erika.

T R A I N I N G I N DUSTR Y MAGAZ INE -LEARNING ANALYTICS 20 19 I WWW. T RAININGINDU S T RY . C OM/ MAGAZ I NE

| 51


https://www.trainingindustry.com/magazine

Training Industry Magazine - July/August 2019

Table of Contents for the Digital Edition of Training Industry Magazine - July/August 2019

Learning Analytics: Table Stakes for Business Success
Table of Contents
Let's Build a Measurement Practice for L&D
Brain-Based Learning Analytics and Evaluation Practices
Who [Else] Cares About Learning Impact Data
Justifying the Investment in Leadership Training
Is Training Analytics Smoke and Mirrors?
Leveraging Analytical Tools to Transform L&D
Levearging Learning Analytics to Improve Business Outcomes: Where to Start
Was It Worth It? Measuring the Impact and ROI of Leadership Training
How to Make the Transition to a Data-Driven Learning Culture
How Personal Skills AI Assistants Might Disrupt Corporate Training
Machines Are the Future of Training: How Data-Driven Feedback Is Fostering Improvement and Enhancing Human Relationships
Improving Instructor Impact on Learning with Analytics
A Year in the Life: Evolving L&D Metrics From a Reporting Managed Service to Business Insight Partner
Localizing E-Learning Programs for an International Audience
Learning Analytics Is More Than Data Collection
Using Business Data to Drive Training Objectives
Quality versus Quantity: How Analytics Can Improve Training Effectiveness
Coursera Reaches Unicorn Status and Accelerates Enterprise Product Innovation
Company News
Training Industry Magazine - July/August 2019 - Cover1
Training Industry Magazine - July/August 2019 - 2
Training Industry Magazine - July/August 2019 - 3
Training Industry Magazine - July/August 2019 - 4
Training Industry Magazine - July/August 2019 - Learning Analytics: Table Stakes for Business Success
Training Industry Magazine - July/August 2019 - Table of Contents
Training Industry Magazine - July/August 2019 - 7
Training Industry Magazine - July/August 2019 - 8
Training Industry Magazine - July/August 2019 - 9
Training Industry Magazine - July/August 2019 - 10
Training Industry Magazine - July/August 2019 - Let's Build a Measurement Practice for L&D
Training Industry Magazine - July/August 2019 - 12
Training Industry Magazine - July/August 2019 - Brain-Based Learning Analytics and Evaluation Practices
Training Industry Magazine - July/August 2019 - 14
Training Industry Magazine - July/August 2019 - Who [Else] Cares About Learning Impact Data
Training Industry Magazine - July/August 2019 - 16
Training Industry Magazine - July/August 2019 - Justifying the Investment in Leadership Training
Training Industry Magazine - July/August 2019 - Is Training Analytics Smoke and Mirrors?
Training Industry Magazine - July/August 2019 - 19
Training Industry Magazine - July/August 2019 - 20
Training Industry Magazine - July/August 2019 - 21
Training Industry Magazine - July/August 2019 - Leveraging Analytical Tools to Transform L&D
Training Industry Magazine - July/August 2019 - 23
Training Industry Magazine - July/August 2019 - 24
Training Industry Magazine - July/August 2019 - 25
Training Industry Magazine - July/August 2019 - Levearging Learning Analytics to Improve Business Outcomes: Where to Start
Training Industry Magazine - July/August 2019 - 27
Training Industry Magazine - July/August 2019 - 28
Training Industry Magazine - July/August 2019 - 29
Training Industry Magazine - July/August 2019 - 30
Training Industry Magazine - July/August 2019 - 31
Training Industry Magazine - July/August 2019 - 32
Training Industry Magazine - July/August 2019 - Was It Worth It? Measuring the Impact and ROI of Leadership Training
Training Industry Magazine - July/August 2019 - 34
Training Industry Magazine - July/August 2019 - 35
Training Industry Magazine - July/August 2019 - How to Make the Transition to a Data-Driven Learning Culture
Training Industry Magazine - July/August 2019 - 37
Training Industry Magazine - July/August 2019 - How Personal Skills AI Assistants Might Disrupt Corporate Training
Training Industry Magazine - July/August 2019 - 39
Training Industry Magazine - July/August 2019 - 40
Training Industry Magazine - July/August 2019 - 41
Training Industry Magazine - July/August 2019 - Machines Are the Future of Training: How Data-Driven Feedback Is Fostering Improvement and Enhancing Human Relationships
Training Industry Magazine - July/August 2019 - 43
Training Industry Magazine - July/August 2019 - 44
Training Industry Magazine - July/August 2019 - 45
Training Industry Magazine - July/August 2019 - 46
Training Industry Magazine - July/August 2019 - Improving Instructor Impact on Learning with Analytics
Training Industry Magazine - July/August 2019 - 48
Training Industry Magazine - July/August 2019 - 49
Training Industry Magazine - July/August 2019 - A Year in the Life: Evolving L&D Metrics From a Reporting Managed Service to Business Insight Partner
Training Industry Magazine - July/August 2019 - 51
Training Industry Magazine - July/August 2019 - Localizing E-Learning Programs for an International Audience
Training Industry Magazine - July/August 2019 - 53
Training Industry Magazine - July/August 2019 - 54
Training Industry Magazine - July/August 2019 - Learning Analytics Is More Than Data Collection
Training Industry Magazine - July/August 2019 - 56
Training Industry Magazine - July/August 2019 - Using Business Data to Drive Training Objectives
Training Industry Magazine - July/August 2019 - 58
Training Industry Magazine - July/August 2019 - Quality versus Quantity: How Analytics Can Improve Training Effectiveness
Training Industry Magazine - July/August 2019 - Coursera Reaches Unicorn Status and Accelerates Enterprise Product Innovation
Training Industry Magazine - July/August 2019 - Company News
Training Industry Magazine - July/August 2019 - Cover4
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20201112
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20200910
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20200708
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20200506
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20200304
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20200102
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20191112
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20190910
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20190708
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20190506
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20190304
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20190102
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20181112
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20180910
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20180708
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20180506
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20180304
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20180102
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20171112
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20170910
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20170708
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20170506
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20170304
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20171112_se
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_20161112
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2016fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2016summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2016sales
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2016spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2016winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2015outsourcing
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2015fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2015summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2015leadership
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2015spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2015winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2014fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2014summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2014spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2014winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2013fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2013summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2013spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2013winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2012fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2012summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2012spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2012winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2011fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2011summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2011spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2011winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2010fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2010summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2010spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2010winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2009fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2009summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2009spring
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2009winter
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2008fall
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2008summer
https://www.nxtbook.com/nxtbooks/trainingindustry/tiq_2008spring
https://www.nxtbookmedia.com