On May 13, 2021 the ACAMS Carolinas Chapter hosted a virtual session on “Model Validation and Independent Testing.” Board member and SVP, Financial Crimes Governance at Truist, Megan Nelson moderated a panelist of experts to discuss:
Models and Their Governance
Validation & Testing Approach
BSA/AML Models and Regulatory Updates
Efficiencies & Synergies
Megan kicked the event off by discussing what a model is and whether or not it is tool or a model. The audience was asked to consider if the tool or model is or is not making decisions and whether or not those underlying decisions are automated or not. Panelists noted that there is no regulatory requirement to have an automated process for AML transaction monitoring but that even manual processes (e.g., spreadsheets) require some level of validation that the process is working. Maleka Ali, President of ARCH-SERV, LLC commented that a quality assurance (QA) or quality control (QC) may be appropriate for manual processes. Ultimately panelists agreed that a decision making tools must be validated and model validation should challenge the decision making process.
Next the panel discussed how to govern a model that is a proprietary tool or vendor provided service, and the importance of financial institutions (FIs) understanding the rules (the what) and how the model works. It was noted that system owners and users should challenge the vendor or service provider and ask questions, especially if transactional activity is not being picked up by the model.
The panelist dissected components of model validation and stressed that there is no one size fits all approach. Validation of models should be risk based. While change is inevitable, Maleka pointed out that FIs are ultimately responsible for their model and BSA Officers should be making decisions to approve or reject changes
made. Additionally, the design and workings of models should be documented, including changes, all of which should be supported by well written documentation. Financial Crimes Compliance Coach, Michael Schildow commented on recent enforcement actions and weaknesses in change control processes. He stated that depending on the level of risk (low, medium or high), the change might require escalation for approval. He noted that FIs should introduce changes to models in safe environments (e.g., test environments) first, rather than to live systems to address an immediate need such as staffing issues and unruly alert volumes. Michael stressed that model changes should be documented in a way to support why the change was risk based and should include sufficient detail to justify and support the change decision.
Dhritiman Bhattacharya, Director and Lead Expert for Financial Crimes Modeling at Promontory discussed risk based validation of financial crimes models. Having a pre-set, well defined methodology, an aptitude for “out of the box” thinking, and collaborative (instead of a siloed) testing approach are all key determinants of an effective validation exercise. These steps taken together maximize the chances of finding model issues early before they become materially important and costly to fix.
Dhritiman rounded out the discussion by summarizing and commenting on the recent Interagency Statement on Model Risk Management for Bank Systems Supporting Bank Secrecy Act/Anti-Money Laundering Compliance. He noted that the interagency statement should provide some assurance to banks that they can indeed adopt risk based, non-duplicative methods to BSA/AML validation and testing activities should be tailored for the unique risks, controls, and operating environment of each institution. Risk management efforts across all line of defense should be coordinated and focused on providing a comprehensive risk view rather than mechanically validating each other’s work. In addition, instead of engaging debating about which AML system component is or isn’t a model, banks should make sure testing of systems (models or tools) are commensurate with their risks.
The ACAMS Carolinas Chapter would like to thank our speakers and sponsor Promontory for making the event possible. The Chapter is already preparing for the next virtual event in June, where expert speakers will dive into Terrorist Financing. We hope you will join us!
Comments