Instrumentation & Measurement Magazine 23-5 - 11

Automated Test Systems Test
Program Software - Making
the Invisible Visible
David R. Carey and Jerome Romania

n "Automated Test Systems Test Program Software -The
Invisible Technology" we indicated that the amount of
work associated to Test Program Set (TPS) development,
maintenance, and migration in the Department of Defense
(DOD) has grown significantly [1]. In the paper, we proposed
a model for a TPS developer's certification plan, training program, and Quality Assurance management plan. This paper is
a continuation of that original work. It will take a deeper look
into one area: TPS Quality Assurance. Development of successful test program sets for Automated Test Systems (ATS)
is enhanced by timely application of proven management
procedures such as quality control plans, design reviews,
change control procedures. It is also enhanced by controls
over cost, schedule, development, deployment, and end item
configuration. This paper is devoted to a general discussion
of management and technical procedures, plans, tests, and
reports that are applicable to assuring quality throughout development and delivery of the TPS.
The applications software programs, written to test the
individual parts or subsystems of the system, are critical elements of automatic test support. Units Under Test (UUTs)
are the individual parts or subsystems. All interaction between the ATS and the UUT it tests requires a TPS. A TPS
consists of a hardware Interface Device (ID) that physically
connects to the ATS. The UUT may connect directly to the
ID, as would be the case for a circuit card, or it may use cables that connect from the ID to the UUT. Another important
component of the TPS is the software computer program
that controls the ATS functions. This control is done by automatically selecting and routing stimuli to the UUT and
automatically measuring UUT response. TPSs are written in a variety of languages and dependent upon the ATS
for which it is intended. Some ATS can support a variety of
software languages. TPSs constitute more than 70% of ATS
expenditures in the DOD today and will continue to represent an increasingly larger percentage of the total ATS

expenditure [2]. Quality Assurance (QA) has a significant
role in ensuring their effectiveness in terms of both performance and maintainability.

Quality Assurance Role in TPS
Development
The role of QA in TPS development defines the acceptable level of quality, typically defined by the customer, and
describes how the team will ensure this level of quality is delivered. This definition is laid out in four principal development
and quality plans:
◗◗ Test Program Set Development/Control Plan
◗◗ Quality Assurance Program Plan
◗◗ Configuration Management Plan
◗◗ Acceptance Test Plan
The two main techniques used to ensure quality of TPSs
are design reviews and quality control functions. Design reviews are conducted by design engineering personnel. Quality
control functions involve management, engineering and manufacturing personnel.
Quality is a product characteristic that can be designed into
a TPS by means of deliberate control efforts in all phases of
conception, design, production, integration, usage, obsolescence and sustainment. The role of QA is to establish a plan
and procedures that assure compliance with best practices
and requirements of the TPS development contract [3]. Good
QA plans are a tool used to promptly detect and correct design practices that could result in a defective TPS, control the
configuration of documentation, preserve documentation,
define and schedule the TPS development process, and establish the general test guidelines for acceptance and verification
test procedures. Personnel performing the quality functions
should have a sufficiently well-defined responsibility, authority and organizational stature. Additionally, they must
be technically competent to identify and act on quality problems, maintain the current design policy, be able to adequately

This article is based on research originally presented at AUTOTESTCON 2019 [1]
and recognized as the Best Management Paper.
August 2020

IEEE Instrumentation & Measurement Magazine 11
1094-6969/20/$25.00©2020IEEE

Instrumentation & Measurement Magazine 23-5

Table of Contents for the Digital Edition of Instrumentation & Measurement Magazine 23-5