The Call - Winter 2016 - 15

the Working Group to create the new
national standard and whose primary
contribution was the quality assurance
review spreadsheets.

Best Practice #1: Review the Whole
Call, Not Just the Intake Piece
Perhaps no story illustrates more
poignantly the critical need for a 9‑1‑1
QA/QI program than that of Denise
Amber Lee. On January 17, 2008, Denise
was abducted from her home in North
Port, Florida. In the hours that followed,
Denise managed to dial 9‑1‑1 using her
captor's cell phone. At least four other
calls to 9‑1‑1 were placed, one from her
distraught husband and three from
eyewitnesses. But despite all the calls,
no help was ever dispatched.
According to Parry, one of the things
that sets the new standard apart is that it
embraces QA for the entire call‑taking and
call‑dispatch process. "Typically quality
assurance programs that are out there
and commercially available only cover
the call intake piece," said Parry. "What
we decided to do here is cover the whole
thing-stem to stern. So, we not only
have processes in place for evaluating the
three types of disciplines that come into
our 9‑1‑1 centers, we also have evaluation
templates for the dispatch piece as well."
Sherrill Ornberg, who along with
Parry is a member of the Denise Amber
Lee Foundation's Board of Directors,
agrees this is critical. "It wasn't just the
call‑taking in the Denise Amber Lee
tragedy, it was the dispatch portion that
was missing," she said.

Best Practice #2: How Many
and Which Calls to QA
The standard recommends that PSAPs
review a minimum of 2 percent of all
calls (including both call‑taking and
dispatching components). Ornberg,
Parry, and Botz all agree that random
QA is the best approach. Don't just
pick out bad calls or zero in on specific
Everyone involved in handling
calls should be monitored, whether
full time, temporary or volunteer
telecommunicators. "While all personnel

should be evaluated weekly to ensure
timely feedback, new hires should be
evaluated more frequently," said Botz.
The standard also recommends that
all low‑frequency calls involving any
high acuity or catastrophic events be
reviewed, in addition to any other types
of calls the agency deems important.

Best Practice #3: How to Overcome
Resistance to Your QA Program
The key to overcoming fear and
resistance to QA and monitoring is
to involve telecommunicators in the
planning process from the get‑go, and
elicit their input. Explain the objectives
of the program; clarify exactly how
they'll be monitored, what criteria they'll
be measured on, how evaluations will
be conducted, how the data will be
used, and why it matters to them. Get
them involved in QA form design. Allow
them to listen to their own calls and
self‑evaluate. Phase the program in over
a period of time, soliciting feedback and
making adjustments along the way.
According to Ornberg, your QA
program will also have better odds
of success if supervisors start out by
selecting a couple of positive calls for each
telecommunicator, so they are less fearful
as they acclimate to the QA process.
Employees must believe that the
program will be conducted with absolute
objectivity and fairness, so make sure
your PSAP uses the same criteria to
measure everyone, all the time, and
calibrates evaluators for consistency.
Finally, consider using QA forms during
the hiring and onboarding process to
help new employees better understand
how they'll be measured and evaluated,
and what knowledge, skills, and
abilities are necessary to be a successful

Best Practice #4: Setting up
Forms and Scoring Calls
According to Botz, there is a science to
how PSAPs should set up their QA forms
and score calls.
First, your forms should align with your
PSAP's standard operating procedures

(SOPs). They should address the entire
intake and dispatch process and focus
on three key areas: Adherence to SOPs/
protocol compliance, call quality (customer
service), and required telecommunicator
knowledge and skills.
Law enforcement, fire/rescue, and EMS
calls each have different procedures,
flows, and protocol compliance
requirements. Therefore, you'll want to
create unique QA audit forms for each
of these call types and also for different
job responsibilities within each type
(e.g. call taking vs. dispatching).
QA scoring is most effective when
the questions are black or white. QAEs
should only be able to select "yes," "no,"
"refused" (if the caller refuses to provide
the information), or "N/A" if the question
is not applicable. Determine a desired
point value for each question, based
on its relative importance to overall job
performance. Include a checkbox for
"exceeds requirements" (in addition
to "meets requirements" and "needs
improvement") so top performers can be
recognized and telecommunicators who
need additional coaching or training can
be easily identified.
Once you set up your forms, you'll need
to get to the task of scoring calls.
"Ultimately the target benchmark for a
passing score for each evaluation should
be 90 percent," said Ornberg. "Although
I would very strongly recommend that
PSAPs start at a much lower score, say
75 percent, when they roll their QA
program out. As staff become acclimated to
what's expected from them, the acceptable
score can be ramped up to 80 percent, then
up to 85 percent, and so on."
If you don't know where to start to set
up your forms, the full QA/QI standard
including sample QA forms can be
downloaded from the NENA Website at:

Best Practice #5: Ensure Timely
Reviews and Employee Notification
The NENA/APCO American National
Standards Institute (ANSI) standard

Read the digital edition at‑nxt/


Table of Contents for the Digital Edition of The Call - Winter 2016

President’s Message
From the CEO
Failing to Plan is Planning to Fail: The Gatlinburg Fires
Government Affairs
10 Best Practices to Improve Your 9-1-1 Quality Assurance Program
Tech Trends
NENA Helps Bring Disability Awareness Training to the Forefront
Educational and Operational Issues
Public Safety Product and Service Buyer’s Guide
Index to Advertisers/
The Call - Winter 2016 - cover1
The Call - Winter 2016 - cover2
The Call - Winter 2016 - 3
The Call - Winter 2016 - 4
The Call - Winter 2016 - 5
The Call - Winter 2016 - 6
The Call - Winter 2016 - 7
The Call - Winter 2016 - President’s Message
The Call - Winter 2016 - From the CEO
The Call - Winter 2016 - Failing to Plan is Planning to Fail: The Gatlinburg Fires
The Call - Winter 2016 - 11
The Call - Winter 2016 - 12
The Call - Winter 2016 - Government Affairs
The Call - Winter 2016 - 10 Best Practices to Improve Your 9-1-1 Quality Assurance Program
The Call - Winter 2016 - 15
The Call - Winter 2016 - 16
The Call - Winter 2016 - 17
The Call - Winter 2016 - 18
The Call - Winter 2016 - Tech Trends
The Call - Winter 2016 - NENA Helps Bring Disability Awareness Training to the Forefront
The Call - Winter 2016 - 21
The Call - Winter 2016 - 22
The Call - Winter 2016 - Operations
The Call - Winter 2016 - Educational and Operational Issues
The Call - Winter 2016 - 25
The Call - Winter 2016 - Public Safety Product and Service Buyer’s Guide
The Call - Winter 2016 - 27
The Call - Winter 2016 - 28
The Call - Winter 2016 - 29
The Call - Winter 2016 - 30
The Call - Winter 2016 - 31
The Call - Winter 2016 - 32
The Call - Winter 2016 - 33
The Call - Winter 2016 - Index to Advertisers/
The Call - Winter 2016 - cover3
The Call - Winter 2016 - cover4