BISA Magazine - Quarter 4, 2017 - 25

ment actions has increased. There have
been instances where the CCO was
personally held liable by the regulators,
with fines imposed on them, making
it prudent to track these trends (listed
below) and adapt operations as appropriate to the bank.
Increased Examination Focus
Nearly all the banks we have spoken
with have found that the Model Validation process was part of their overall
AML examinations. This has been ongoing and appears to also now apply to
a wider range of small banks, and the
depth of review has increased.
Evolving Requirements
The manner in which the Model Validation process is reviewed varies substantially across regulators. To some extent,
this is related to the risk and activities
of the bank. However, much of the variance does not appear to be explained
by risk alone, making it difficult for a
bank to anticipate the level of review
and their expected performance.
Expected Rigor and
Quantitative Process
A consistent trend in recent examinations is the rigor and demand for
quantitative support for judgments
reached in a Model Validation. This is
not surprising given that a quantitative approach is an essential aspect of
overall Model Risk Management.
The CRAD
The Compliance Risk Analysis Division (CRAD) of the OCC comprises
highly skilled statistical analysis and
econometric models professionals. The
group is primarily staffed with PhDs in
economics and statistics and provides
broad support for the OCC's supervisory and regulatory initiatives. Many of
the banks we spoke with have had the
CRAD play a role in their AML examinations. The group has reviewed and
provided substantive challenges to the
AML models used by those banks. With
respect to those challenges, the bank is
expected to quantitatively justify chosen detections rules and the thresholds
applied to them.

A Question of Cost vs. Scope
The OCC provided guidance on the use
of consultants as part of an enforcement action. Although this does not
directly apply to Model Validations, it
highlights the OCC's interest in a bank's
due diligence over third party consultants and their associated contracts.
In our review, a bank mentioned the
OCC's concern with a project's cost,
where the OCC felt the cost provided
was too low to cover the full work for a
proper Model Validation. This attention
to detail suggests that banks should
carefully define the scope of work and
be diligent with third-party contracts.

Bank Challenges
With an increasing focus on a larger
range of banks, Model Validations need
to be considered a priority by all CCOs,
and it's important to understand the
associated challenges.
Validation Approach
Although the OCC guidelines apply
to all Model Validations, its implementation can vary greatly from bank
to bank. The variability is based on
several factors including AML specific
requirements, risk profile and the views
of the regulator. Without the benefit of
several refinements to the process, a
bank may face strong criticism that can
lead to regulatory actions.
System Limitations
Model Validations are inherently
system focused. Most AML systems are
acquired from a software vendor. The
vendor has a proprietary interest in
maintaining their intellectual property
rights and will often provide models
as a "black box." Without adequate
disclosure from the software vendor,
the correctness of model design and
implementation cannot be directly assessed. Validation is limited to a testing
and analysis methodology, which is less
comprehensive.
Required Skill Set
A proper Model Validation requires a
range of skill sets that may not be available or are not easily accessible to the
bank. A team approach is needed and

should include members with expertise
in the following areas: AML compliance
domain; AML system; econometric, statistical and mathematical; data analytics; and audit.
Needed Tools
Accessing and analyzing data, performing tests and interpreting results
are enhanced with the availability of
automated tools such as AML typology
models and data analytics platforms.
Reliance on Consultants
With the range of skills required for
a validation, banks may employ an
independent consulting firm to perform part or most of the required work.
Where the bank is lacking familiarity
of the overall requirements for an AML
Model Validation, the selection process
can be difficult, not to mention the
many small consulting firms that will
engage contractors for the work performed. The use of contractors by the
consulting firm can lead to inconsistent
processes and a lack of continuity over
subsequent engagements.
Communicating with Regulators
There is much latitude in the implementation of a Model Validation
process. When the bank takes an
approach that is not fully understood
by the regulator or where the regulator suggests an alternate approach, it
is required that the bank adequately
explain all aspects of the validation
approach. The explanation may need
to be made to the primary regulator
as well as to specialized teams such as
the CRAD. Each of these groups may
require a different level of detail and
explanation, and having the appropriate staff to provide these explanations
is often a challenge.
Budgeting
Senior management must fully understand the scope of work and time
commitment for a successful Model Validation. Whether the work is done internally or externally, adequate resources
must be made available to develop a
successful justification approach.

25
BISA Magazine

Table of Contents for the Digital Edition of BISA Magazine - Quarter 4, 2017