The Call - Winter 2016 - 17

and objective. If you have multiple
QAEs, each could interpret the
evaluation criteria differently. That's
why it's absolutely critical to create
and document QA definitions, review
those definitions with your QAEs and
telecommunicators, and calibrate
reviews on an ongoing basis.
Ornberg explains: "If you have one
QAE who's grading more generously and
one who's grading more strictly, your QA
program will definitely fail, because you're
going to have favoritism issues. Even if
you just have one QAE, it's important to
calibrate to make sure the QAE is in line
with management's expectations."
So what can your PSAP do to ensure
QAEs are interpreting evaluation
questions consistently and correctly?
"Evaluators need to be properly trained
and periodically assessed to ensure
that they are being fair and equitable,
and scoring calls consistently within an
agreed‑upon standard deviation across
multiple QAEs," Botz advises.
As a QA program is being rolled out,
senior management should review
QAE evaluations and calibrate them
on a weekly basis, and as the program
matures, transition to bi‑monthly or
monthly calibration reviews.
Senior management should select at
least one call‑taking and one dispatching
incident and score them, and then ask
the QAEs to do the same. If there are
any outliers, management should then
discuss and rectify any discrepancies in
calibration sessions.

Best Practice #10: Let Technology
Do the Work for You
Even today, with the availability of
automated QA solutions, many PSAPs
conduct their QA manually, using
over‑the‑shoulder monitoring, manual
call selection, and paper evaluation
forms and reports.
QA software solutions can eliminate
paper and improve efficiency by
streamlining every aspect of the QA
process. Here's how:

Automated Call-Scheduling Rules

Automated Reporting

Set up rules in the QA system to select
a specific number or percentage of calls
(for each telecommunicator for a specific
time period). Automating this process
means QAEs get a true random sample.
You can further refine automated
call selection based on other criteria:
call length, time of day, day of week,
originating locations, or call type.

The beauty of electronic scoring is that
it also makes it possible to automatically
generate reports that highlight
performance metrics and trends.
These insights can then be applied to
improving performance via one‑on‑one
coaching sessions or training on a
broader scale.

Audio Analytics for Finding
Important Calls to QA
Use the audio analytics engine to
automatically identify and categorize
calls containing keywords or phrases
relating to specific incident types; for
example, domestic violence, homicides,
or heart attacks. For the latter, key words
like "heart attack," "cardiac arrest," or
"chest pain" would trigger the recording
to be pulled and categorized for review.

Audio Analytics for
Ensuring Compliance
Use audio analytics to "listen"
to 100 percent of 9‑1‑1 calls and
isolate problem calls (e.g. containing
unacceptable language), or to detect the
presence or absence of keywords/phrases
to determine if protocols were followed.

More Efficient, Streamlined
Call Review
Thousands of PSAPs nationwide
now use protocol‑driven software, like
Priority Dispatch's ProQA and AQUA,
to guide telecommunicators through
responses for police, fire, and EMS
calls, and to evaluate those same calls
after‑the‑fact. Having call recording
integrated directly into the Priority
Dispatch software interface can cut QA
review time in half, because QAEs can
automatically pull up all calls and review
them in one interface.

Pre-Programmed QA Templates
and Form Builders
Start from a library of pre‑built forms,
or use form builders to quickly create
QA forms for different protocols, roles,
and incident types, first setting up form
sections and questions, then assigning
weighting factors. The QA software
automatically tabulates the scores as
evaluations are completed.

The ten best practices offered above
are a guideline to help make your PSAP's
QA/QI program more effective and
efficient. If you're already implementing
a QA program in your 9‑1‑1 center,
I encourage you to review your current
processes and apply as many of these
best practices as possible. If you're just
getting your QA program off the ground,
you may be hard pressed to implement
all of these best practices, but it's
important to start somewhere.
"I've heard managers say 'I don't
know where to start,' 'I don't have time,'
or 'I don't know how to do it,'" said
Ornberg. The national QA Committee
has already done the heavy lifting by
providing sample QA spreadsheets in
Excel for police, fire, and emergency
medical services (for both the call‑taking
and dispatching). Instructions have also
been included for agencies to customize
the forms to fit their own requirements,
along with sample scoring criteria. One
additional benefit is that the templates
are vendor‑neutral, so they can be used
in tandem with existing QA practices and
programs. Ornberg sums it up this way:
"The thing to keep in mind is that while
it might seem overwhelming, you have
to dig in and start somewhere, because
what you do today will pay dividends in
the future."  ●
Linda Haelsen is the Marketing
Communications Manager for NICE
Public Safety.

APCO/NENA ANS Standard for the
Establishment of a Quality Assurance and
Quality Improvement Program for Public
Safety Answering Points. APCO International
and NENA The 911 Association. Retrieved from

Read the digital edition at‑nxt/


Table of Contents for the Digital Edition of The Call - Winter 2016

President’s Message
From the CEO
Failing to Plan is Planning to Fail: The Gatlinburg Fires
Government Affairs
10 Best Practices to Improve Your 9-1-1 Quality Assurance Program
Tech Trends
NENA Helps Bring Disability Awareness Training to the Forefront
Educational and Operational Issues
Public Safety Product and Service Buyer’s Guide
Index to Advertisers/
The Call - Winter 2016 - cover1
The Call - Winter 2016 - cover2
The Call - Winter 2016 - 3
The Call - Winter 2016 - 4
The Call - Winter 2016 - 5
The Call - Winter 2016 - 6
The Call - Winter 2016 - 7
The Call - Winter 2016 - President’s Message
The Call - Winter 2016 - From the CEO
The Call - Winter 2016 - Failing to Plan is Planning to Fail: The Gatlinburg Fires
The Call - Winter 2016 - 11
The Call - Winter 2016 - 12
The Call - Winter 2016 - Government Affairs
The Call - Winter 2016 - 10 Best Practices to Improve Your 9-1-1 Quality Assurance Program
The Call - Winter 2016 - 15
The Call - Winter 2016 - 16
The Call - Winter 2016 - 17
The Call - Winter 2016 - 18
The Call - Winter 2016 - Tech Trends
The Call - Winter 2016 - NENA Helps Bring Disability Awareness Training to the Forefront
The Call - Winter 2016 - 21
The Call - Winter 2016 - 22
The Call - Winter 2016 - Operations
The Call - Winter 2016 - Educational and Operational Issues
The Call - Winter 2016 - 25
The Call - Winter 2016 - Public Safety Product and Service Buyer’s Guide
The Call - Winter 2016 - 27
The Call - Winter 2016 - 28
The Call - Winter 2016 - 29
The Call - Winter 2016 - 30
The Call - Winter 2016 - 31
The Call - Winter 2016 - 32
The Call - Winter 2016 - 33
The Call - Winter 2016 - Index to Advertisers/
The Call - Winter 2016 - cover3
The Call - Winter 2016 - cover4