Model validations are a vital component of monitoring a financial institution’s Bank Secrecy Act/anti-money laundering (BSA/AML) risk. However, many institutions don’t know what a model is, let alone when an independent third-party validation is required. For AML software, the question of what constitutes a model can be a bit gray. The term model refers to a quantitative method that applies statistical, economic, financial or mathematical theories, techniques, and assumption to process input data into quantitative estimates (OCC 2011-12; SR 11-7). Now that’s a legal mouthful. To simplify, this applies to a BSA department’s transaction monitoring system, customer risk rating system, sanctions and watchlist scanning, currency transaction reporting systems and the institution's CECL automation because they use quantitative data to produce outcomes. A tool, on the other hand, provides outputs which do not qualify as quantitative estimates. A tool may be a system that reorganizes data by aggregation, categorizing, or mapping, such as sophisticated spreadsheets. Over the past two years, model risk management is one of the most commonly cited areas of regulatory AML criticism, particularly in the area of transaction monitoring and higher risk customer due diligence. If suspicious activity monitoring is the cornerstone of a strong BSA program, it must start with a strong model with reliable data and outputs. At an Abrigo BAM+ User Group conference Jason Chorlins of Kaufman Rossin spoke on model validations in easy to understand terms. Here are five tips to ensure that your BSA program models are validated in accordance with regulatory expectations:
- Know the four components of a model. Each model must be validated by an independent third party if these components point to the AML system as a model:
- Information Input Component: Delivers assumptions and data to the model
- Processing Component: Translates inputs into estimates
- Reporting Component: Translates estimates into useful business information
- Governance Component: Documents the roles and responsibility of the model
- Be sure the third party is qualified to conduct a full model validation. Ask for references that have already been through regulatory scrutiny. The third party cannot have any conflicts of interest with the model it is validating. For example, Abrigo cannot perform model validations on BAM+ or Sageworks ALLL.
- Understand the data. A data integrity review will ensure that all appropriate customers and transactions are flowing into an institution’s monitoring system. Be sure to verify all mapped source data, all transaction codes, and all wire and ACH fields. Are all source feeds showing accurately in the model? While a model validation is required periodically, a data integrity review may be warranted more frequently, at least annually, particularly if an institution has new transaction codes, a conversion of integrated systems, or mergers and acquisitions.
- Calibration testing of parameters should be part of a model validation. The evaluation of existing rules, parameters, and thresholds should be tested to ensure suspicious activity is being detected without having too many false positives. Statistical analysis of an institution’s customer activity should be used, along with above the line/below the line testing. A sampling of scenarios, preferably in a test environment, should be performed during a model validation.
- Ensure the AML scenarios work as intended. The third part of the model validation is to ensure that parameter algorithms work as designed. By back-testing logic through the recalculation of scenarios, one will know if its systems are working correctly. This is where the expertise and experience level of a third-party validator must be verified.
Model validations are critical to a sound BSA program and are expected by regulators. As the number of consent orders around this requirement increases, institutions should strongly consider establishing a Model Governance Committee, responsible for the oversight of the institution’s model risk management program. BSA professionals should perform data integrity reviews at least annually and consider a full third-party model validation bi-annually or anytime there has been a significant change in an institution’s model. This should ensure that all significant data from a BSA monitoring perspective is being properly fed into the system, the scenario algorithms are working as intended, and the model output is what one would expect to see. Those ingredients are the basis for a strong and sound BSA program.
If you need help with a data integrity review, contact us today. Our team of experts can help ensure that your data is properly feeding into your systems. In order to get good information coming out, you need good data going in.
