Training Industry Magazine - November/December 2016 - 57

body of research that applies a linguistic
approach and best practices in language
testing to identify what really matters in
communication measurement and how to
implement it in business.
There are three main principles:

1| Given the central importance of spoken
English communication, its measurement
needs to be both summative (providing
an overall score that benchmarks a
candidate for employment and appraisal)
and formative (providing an analytical
diagnosis of the strengths and weaknesses
of the CSR's communication skills) so that
training and coaching can be targeted
effectively.
2 | A measurement tool needs to be:
* Valid: The test relates to the
performance required. For example,
giving a written grammar test to
candidates will not demonstrate
whether they can communicate well
on the phone.

measurement tool comprised of key
criteria related to business success and a
set of competency levels that define the
spoken communication ability of the CSR.
These criteria are normally presented in a
table, and each criterion and level has its
own descriptor.
For call centers, the key performance
criteria relate to:
* Content completion of the calls:
Were the business knowledge and
process communicated accurately and
comprehensively?
* Control of the call flow: Was the CSR
able to manage the call professionally,
explaining business issues clearly,
logically and coherently?
* Customer relationship-building:
Was the CSR able to develop a
good customer relationship by, for
example, listening, empathizing and
apologizing?

* Reliable: Assessment personnel
will understand and interpret the
measurement tool in the same way,
leading to consistent scores.

* Overall English vocabulary,
grammar and phonological choices:
Was the CSR able to make appropriate
and accurate choices while being
globally comprehensible?

* Practical: The test is cost- and timeeffective compared to its value to
the user organization.

PROOF OF CONCEPT: TESTING A
RESEARCH-BASED ASSESSMENT TOOL

3 | The washback of a good communications
assessment rubric ensures that coaching
support will be comprehensive and
relevant. Washback means that the
assessment criteria and the training and
coaching are consistent and aligned.
Because English communication is the
core skill for contact center employees,
the business should make summative
and formative assessments every day. In
recruitment departments, for example,
large numbers of applicants are assessed
on a daily basis, with typically fewer than
10 percent making it to employment.
Developing these new employees
through training and coaching ensures
high-quality customer service, and
this training requires formative, or
diagnostic, assessment and feedback.
Both these summative and formative
functions require a common rubric, a

A number of projects have looked at the
validity of this linguistically informed
assessment approach and tool both in
its summative usage in recruitment and
in its formative usage in coaching.

Recruitment
Are the assessment scores in proficiency
and performance domains good predictors
of successful performance?
To answer this question, a study was
carried out in a large outsourcing contact
center in Manila. Researchers selected
a representative sample of 31 high
performers, middle performers and low
performers and measured the correlation
between their performance and their
proficiency and performance assessment
scores at recruitment. CSRs who scored
higher in the performance domains at
recruitment were significantly better
performers on actual service calls, while

T R A I N I N G I N DUSTR Y MA GAZ INE - TRENDS201 7 I WWW.TRAI NINGINDU S T RY . C OM/ MAGAZ I NE

the proficiency-only assessment had no
statistically significant predictive value.

On-the-Job Assessment,
Diagnosis and Coaching Support
Does this approach to measurement and
washback impact business performance?
In a study in a financial shared service
company in India, communications
coaches were trained to use this
research-based approach over a twomonth period in their regular coaching
sessions. The study compared pilot
group CSRs who received this coaching
with another group who received the
usual form of coaching. The researchers
compared the groups based on the
business' key performance indicators
(KPIs): AHT, hold time, quality scorecard
results, sales targets and VOC.
The pilot group showed strong
progression from pre-coaching results to
post-coaching results, with improvement
in all KPIs. The biggest improvement was
in sales. The control group, however,
showed no significant improvement in
the same period. For that group, some
metrics (VOC, hold time) had minor
improvement, while others (sales, AHT)
had a drop in performance.
These results demonstrated strong
quantitative evidence that the approach
succeeded. From a qualitative viewpoint,
surveys and focus group discussions
showed that the coaches preferred this
approach to the previous scorecardbased coaching, saying that the approach
was more "fair, systematic and objective."
Testing proficiency alone provides a
snapshot in time and it makes it very
difficult to develop call center CSRs
effectively. With more research focusing
on workplace language testing and
linguistics, there are opportunities
to use these approaches to make our
organizations - and our jobs - better.
Dr. Jane Lockwood is an associate professor
in the Department of English at the City
University of Hong Kong, where she has
published extensively in workplace training
and measurement. She co-founded
FuturePerfect, a communications and
assessment consultancy. Email Jane.

| 57

http://www.futureperfect.com/ http://www.trainingindustry.com/ezine.aspx

Table of Contents for the Digital Edition of Training Industry Magazine - November/December 2016