Training Industry Magazine - July/August 2019 - 23

DATA MINING AND ANALYTICS
IN L&D
Data mining is the process of finding
anomalies, patterns and correlations
within large data sets to predict outcomes.
Big data analytics is the process of
collecting, organizing and analyzing large
sets of data (called big data) to discover
patterns and other useful information.
So, how can L&D professionals leverage
data mining and analytics? Figure 1 is
a depiction of what analytics tools can
be leveraged and at what level of the
Kirkpatrick Model.
Let us delve into a little more detail,
with the four Kirkpatrick's levels as the
reference points.

REACTION
Organizations typically collect learner
feedback through either physical or online
survey forms. In the case of e-learning
hosted on learning management systems
(LMSs), feedback forms are often built into
the e-learning course.
The net result is a host of data (from
questions that ask learners to score

parameters such as the relevance of
course content, ease of understanding of
concepts, interactivity or engagement,
etc., on a five or 10-point scale) and/or a
large amount of unstructured information
through comments on various parameters
in the questionnaire or survey forms.

The single biggest
challenge facing
learning leaders today
is evaluating the
effectiveness of their
learning interventions.
To analyze this kind of unstructured data,
there are two analytics tools that, when
used in tandem, are extremely useful: text
mining and sentiment analysis. Imagine
a Python sentiment analyzer that takes

input from the various feedback sources,
the algorithm generates sentiments from
the required or specified surveys, throws
up key themes and generates output that
is continuously appended to the previous
analysis for the same training program.
Then, it generates dashboards with
meaningful, insightful data and shows the
progression of trends over time.
Consider the following example: Let's
say that a large global organization
conducts a new manager induction
program for people who have been newly
promoted to managerial or supervisory
roles from individual contributor roles.
Feedback is gathered and run through
the Python sentiment analyzer every
time the training is conducted. From
each session, the training team obtains
instant insights on what worked well,
what needs improvement, etc. Insights
can be generated in numerous ways,
such as geographically and evaluation
parameter-wise. So, for example, course
content might be working well across
geographies. Logistics might be working
well in one geography but might be a pain
in another. Learner engagement might be
a pain across the board.

FIGURE 1

WHEN

ANALYTICS

HOW

WHAT

Applying analytics for evaluating training effectiveness, personalization and continuous improvement
REACTION

LEARNING

BEHAVIOR + RESULT

Learners' reaction to the
learning intervention

The extent to which learners
acquire the intended knowledge,
skill, attitude, from the
learning intervention

The extent to which learners acquire the
intended knowledge, skill, attitude from the
learning intervention

Feedback surveys or smile
sheets, seeking feedback on
training content, design, trainer

Pre- and post-assessments

Performance assessments, supervisor, mentor,
and 360-degree feedback, performance on
outcomes such as customer satisfaction,
operational metrics, sales

Text mining, sentiment analysis:
Key themes generated via tools
like Python Sentiment Analyzer

Assessment scores: scores,
trends, anomalies; analytics
powered by learning
management systems

Artificial intelligence (AI) and predictive
analytics: Intelligent systems enabling
integration of data from HRMS, LMS, and
other relevant systems, and AI algorithms
driving predictive analytics

After intervention

Before and after intervention

After intervention, over defined
time periods

T R A I N I N G I N D USTR Y MA GA ZIN E -L EARN ING A NA LY TICS 20 19 I WWW. T RAININGINDU S T RY . C OM/ MAGAZ I NE

| 23

https://www.trainingindustry.com/magazine

Training Industry Magazine - July/August 2019

Table of Contents for the Digital Edition of Training Industry Magazine - July/August 2019