Backtesting in ALLL calculations
Backtesting is an exercise that compares the actual outcome with model forecasts during a defined period – a period of time that was not used to develop the methodology.
A good starting point for any measure of efficacy is backtesting a reserve methodology on the portfolio.
At the simplest level, backtesting of the ALLL can be completed on global level, comparing the bank or credit union’s ALLL to its realized losses over a period of time. An example is shown below:
In this example, the institution is looking at its ALLL and comparing that amount with annualized Net Charge-offs Year to Date, an approximation of the whole year’s charge-offs. This comparison assesses the coverage of the ALLL relative to realized losses. Some basic questions to ask include:
1. How many years’ losses can be estimated to be covered by any period-end allowance when examining the current period allowance as compared to Net Charge-offs Year to Date (annualized)?
In this specific example, the bank’s calculated ALLL exceeds the realized net charge-offs consistently by at least 25 percent. In 2012, it would appear that, overall, the bank had approximately 150 percent coverage of their realized losses based on the ALLL that was reported in each quarter.
2. Does the current period’s Net Charge-offs Year to Date (annualized) exceed the prior period’s allowance? If so, are there any factors that could have been identified that would have caused the ALLL to increase?
This takes Question 1 a little farther. If there are periods in which annualized net charge-offs exceed the prior period’s allowance, then it is important to dig more into what reasons there were for charge-offs to increase and where it could have been identified in the ALLL. This requires a more in-depth review of specific FAS 5 (ASC 450-20) segments- specifically segment size and growth, loss rates and the qualitative and environmental factors.
3. Were there significant changes from one quarter to the next in the net charge-offs? If so, did the allowance increase accordingly? What were the factors that caused the increase in net charge-offs? If the ALLL did not increase, what factors could have been identified (that could be watched more closely in the future) to increase the ALLL accordingly?
This again requires a more in-depth review of the various FAS 5 segments to determine which segments had an increase in charge-offs that would ideally have led to an increase in required reserve.
Once these three questions have been addressed at a global level and their results documented for auditors and examiners, backtesting can also be exercised for individual loan portfolio segments.
For more information on ALLL calculations, download the Backtesting: Measuring the Effectiveness of ALLL Methodologies.