POWER April 2013 - 34

REGULATORY
expect are cases where there is no test system
or backup system to test the change on. In
those cases, auditors will expect to see specific
procedures documented that will minimize
any possible negative impacts.
Auditors will typically select a statistically
accurate sample of all significant changes
and review these changes in detail. This will
include looking at the before and after test
results and matching them to the baseline for
the device. If there were any changes that affected
the baseline, they will need to verify
that the baseline was updated as required by
the standards (within 30 days of the change).
Auditors will expect to see a documented
process for configuration management. They
expect to see baseline documentation for the
ports and services for all critical cyber assets
(CCAs) and cyber assets within an electronic
security perimeter (except those with
TFEs for CIP-007 R2). This documentation
should include the justification for each port
and service that is active on the asset. Typical
justification is a manufacturer's statement
showing what is required, but if that includes
a very large range of ports, an auditor will
likely question it. Auditors may also ask if
you have turned off any ports and then tested
the asset to see if it will function without
them. Simply doing a display of the current
ports and services is not considered adequate
justification.
Auditors will look for any cases where the
baseline should have been changed as a result
of a significant change to a CCA. All changes
like that typically receive a detailed review.
CIP-003 R6: Analyze Your Evidence.
At this point, review your change control
and configuration management process
documents. Are all the procedures accurately
described? A good check is to take the procedure
and see if someone who does not work
in that area can successfully execute it. Alternatively,
you can take someone who has
implemented changes and walk them through
everything they did to see if there are any
steps they took that are not documented. You
want to be confident that the procedures accurately
represent what people really do.
Also verify the interaction between processes.
Verify that if the ports and services
baseline changes, part of the change process
is to update the associated documents (this
can include asset lists when decommissioning
or adding an asset, configuration baselines,
and even recovery procedures).
The next step is to determine what evidence
you need to show that you are following
the process. For change management,
this will start with your change-tracking
system. Assuming you are using a commercial
change-tracking system (like Remedy,
for example), you will need to gather any
additional procedure documentation for the
system if that is not covered in the previous
documents you reviewed.
Export from the change-tracking system a
report of all significant changes. If this is not
easily done, it is a flag that you will need additional
resources available during the audit to
respond to data requests. The report should include
a description of each change. If possible,
determine which changes resulted in a change
to the baseline documents. If that is not readily
available from your tracking system, then find
an alternate way to get that data.
From this report, you can run RAT-STATS
(with confidence level of 95%, rate of occurrence
of 0.5%, and precision range of 10%)
to determine the sample size. For example,
if there were 1,000 significant changes, you
would need to sample 34 random changes.
RAT-STATS can then be used to generate the
34 random samples. You can expect the auditors
to do a similar calculation.
For the changes in this sample, pull the
required documents and populate them. This
will start with the change ticket, but it will
also include the before and after test results.
It may even include associated e-mail referenced
in the ticket; an e-mail containing the
justification for a change in ports and services
is a good example.
Another concern for evidence is the organization
of your Entity. Many Entities have
completely different procedures and documentation
for generation, transmission, and
control centers. Often an IT group maintains
the network connectivity and network devices,
but it typically has different processes as well.
Consider how to organize these different processes
for presentation to the auditors.
CIP-003 R6: Package Your Evidence.
When you analyzed your evidence, you
should have been thinking about how to
package it. As you documented each piece of
evidence in your document reference spreadsheet,
you should note how and where you
want to package it. You should have created
a folder for each CIP and then copied each
document to that folder as it was analyzed,
unless it was a large file that will be sampled
instead. At this point, you can combine individual
files into PDFs with bookmarks for
the auditors to use.
Prepare each folder so that it contains only
the files you will submit as evidence, the
Thousands of enclosures
Hundreds of sizes
Next-day shipping
34
www.powermag.com
POWER | April 2013
http://www.powermag.com
POWER April 2013

Table of Contents for the Digital Edition of POWER April 2013
