Looking for Banker’s Toolbox? You are in the Right Place!

Banker’s Toolbox is now Abrigo, giving you a single source for all your enterprise risk management needs. Use the login button here, or the link in the top navigation, to log in to Banker’s Toolbox Community Online.

Make yourself at home!

Looking for MainStreet Technologies? You are in the Right Place!

MainStreet Technologies is now Abrigo, giving you a single source for all your enterprise risk management needs. Use the contact us button here, or the link in the top navigation, to reach product support for your MST products.

Make yourself at home!

Looking for Sageworks? You are in the Right Place!

Sageworks is now Abrigo, giving you a single source for all your enterprise risk management needs. Use the login button here, or the link in the top navigation, to log in to your Sageworks products.

Make yourself at home!

What is Model Validation and Why Do You Need One?

By: Terri Luttrell, CAMS-Audit

Model validations are a vital component of monitoring a financial institution’s Bank Secrecy Act/anti-money laundering (BSA/AML) risk. However, many institutions don’t know what a model is, let alone when an independent third-party validation is required.   For BSA software, the question of what constitutes a model can be a bit gray. The term model refers to a quantitative method that applies statistical, economic, financial or mathematical theories, techniques, and assumption to process input data into quantitative estimates (OCC 2011-12; SR 11-7). Now that’s a legal mouthful. To simplify, this applies to a BSA department’s transaction monitoring system, customer risk rating system, sanctions and watchlist scanning, currency transaction reporting systems and the institution's CECL automation because they use quantitative data to produce outcomes. A tool, on the other hand, provides outputs which do not qualify as quantitative estimates. A tool may be a system that reorganizes data by aggregation, categorizing, or mapping, such as sophisticated spreadsheets.  Over the past two years, model risk management is one of the most commonly cited areas of regulatory AML criticism, particularly in the area of transaction monitoring and higher risk customer due diligence. If suspicious activity monitoring is the cornerstone of a strong BSA program, it must start with a strong model with reliable data and outputs.  At our recent Abrigo BAM+ User Group conference, Jason Chorlins of Kaufman Rossin spoke on model validations in easy to understand terms.  Here are five tips to ensure that your BSA program models are validated in accordance with regulatory expectations:  

  • Know the four components of a model. Each model must be validated by an independent third party if these components point to the AML system as a model:
    • Information Input Component: Delivers assumptions and data to the model 
    • Processing Component: Translates inputs into estimates 
    • Reporting Component: Translates estimates into useful business information 
    • Governance Component: Documents the roles and responsibility of the model  
  • Be sure the third party is qualified to conduct a full model validation. Ask for references that have already been through regulatory scrutiny. The third party cannot have any conflicts of interest with the model it is validating. For example, Abrigo cannot perform model validations on BAM+ or Sageworks ALLL.  
  • Understand the data. A data integrity review will ensure that all appropriate customers and transactions are flowing into an institution’s monitoring system. Be sure to verify all mapped source data, all transaction codes, and all wire and ACH fields. Are all source feeds showing accurately in the model? While a model validation is required periodically, a data integrity review may be warranted more frequently, at least annually, particularly if an institution has new transaction codes, a conversion of integrated systems, or mergers and acquisitions. 
  • Calibration testing of parameters should be part of a model validation. The evaluation of existing rules, parameters, and thresholds should be tested to ensure suspicious activity is being detected without having too many false positives. Statistical analysis of an institution’s customer activity should be used, along with above the line/below the line testing. A sampling of scenarios, preferably in a test environment, should be performed during a model validation.  
  • Ensure the AML scenarios work as intended. The third part of the model validation is to ensure that parameter algorithms work as designed. By back-testing logic through the recalculation of scenarios, one will know if its systems are working correctly. This is where the expertise and experience level of a third-party validator must be verified.

Model validations are critical to a sound BSA program and are expected by regulators. As the number of consent orders around this requirement increases, institutions should strongly consider establishing a Model Governance Committee, responsible for the oversight of the institution’s model risk management program. BSA professionals should perform data integrity reviews at least annually and consider a full third-party model validation bi-annually or anytime there has been a significant change in an institution’s model. This should ensure that all significant data from a BSA monitoring perspective is being properly fed into the system, the scenario algorithms are working as intended, and the model output is what one would expect to see. Those ingredients are the basis for a strong and sound BSA program.  

If you need help with a data integrity review, contact us today. Our team of experts can help ensure that your data is properly feeding into your systems. In order to get good information coming out, you need good data going in. 

About the Author

Terri Luttrell, CAMS-Audit

Terri Luttrell is a seasoned AML professional and former director and AML/OFAC officer with over 20 years in the banking industry, working both in medium and large community and commercial banks ranging from $2 billion to $330 billion in asset size. She has successfully worked with institutions in developing BSA/OFAC programs, optimizing various automated solutions, and streamlining processes while ensuring all regulatory requirements are met. As the Senior Manager of Strategy and Engagement at Abrigo, Terri provides insights that contribute and support long-term banking strategies based on analysis of market and industry trends, competitor developments, and financial and regulatory technology changes. She is an audit-certified anti-money laundering specialist and a board member of the Central Texas chapter of the Association of Certified Anti-Money Laundering Specialists (ACAMS). Terri earned her bachelor’s degree in business administration, specializing in business and finance, from the University of North Texas.

Full Bio

About Abrigo

Abrigo is a leading technology provider of compliance, credit risk, and lending solutions that community financial institutions use to manage risk and drive growth. Our software automates key processes — from anti-money laundering to fraud detection to lending solutions — empowering our customers by addressing their Enterprise Risk Management needs.

Make Big Things Happen.