4 Questions to Ask When Determining Model Validation Scope

Model risk management is a necessary undertaking for which model owners must prepare on a regular basis. Model risk managers frequently struggle to strike an appropriate cost-benefit balance in determining whether a model requires validation, how frequently a model needs to be validated, and how detailed subsequent and interim model validations need to be. The extent to which a model must be validated is a decision that affects many stakeholders in terms of both time and dollars. Everyone has an interest in knowing that models are reliable, but bringing the time and expense of a full model validation to bear on every model, every year is seldom warranted. What are the circumstances under which a limited-scope validation will do and what should that validation look like? We have identified four considerations that can inform your decision on whether a full-scope model validation is necessary

Sample Size Requirements for CECL Modeling

With CECL implementation looming, many bankers are questioning whether they have enough internal loan data for CECL modeling. Ensuring your data is sufficient is a critical first step in meeting the CECL requirements, as you will need to find and obtain relevant third-party data if it isn’t. This article explains in plain English how to calculate statistically sufficient sample sizes to determine whether third-party data is required. More importantly, it shows modeling techniques that reduce the required sample size. Investing in the right modeling approach could ultimately save you the time and expense of obtaining third-party data.

What CECL Means To Investors

Recent updates to U.S. GAAP will dramatically change the way financial institutions incorporate credit risk into their financial statements. The new method is called the Current Expected Credit Loss (CECL) model and will take effect over the next few years. For many institutions, CECL will mean a one-time reduction in book equity and lower stated earnings during periods of portfolio growth. These reductions occur because CECL implicitly double-counts credit risk from the time of loan origination, as we will meticulously demonstrate. But for investors, will the accounting change alter the value of your shares?

Managing Model Risk and Model Validation

Over the course of several hundred model validations we have observed a number of recurring themes and challenges that appear to be common to almost every model risk management department. At one time or another, every model risk manager will puzzle over questions around whether an application is a model, whether a full-scope validation is necessary, how to deal with challenges surrounding “black box” third-party vendor models, and how to elicit assistance from model owners. This series of blog posts aims to address these and other related questions with what we’ve learned while helping our clients think through these issues.

CECL: Top 10 Organizational Impacts

  CECL is an acronym for Current Expected Credit Losses, and is used as shorthand for the new GAAP requirement to include expected life of loan (LOL) losses in the allowance for loan and lease losses (ALLL) for instruments held at amortized cost, versus the current incurred loss model. This is a very significant change for banks holding held to maturity (HTM) loans and securities.  Given the impact on financial institutions, roll-out of this standard has been slow, and the implementation time-frame is extended. For calendar year-end companies, implementation is required for the fiscal year 2020 for public companies that are SEC filers, and 2021 for everyone else. Early adoption is permitted starting in 2019. Upon implementation, the cumulative effect will be recorded as an adjustment to retained earnings (no income statement impact).  

What to Look for in a Current Expected Credit Loss (CECL) Partner

Financial institutions can partner with consulting firms to get ahead of the new CECL standard. A comprehensive CECL solution requires expertise from a wide range of disciplines, including data management, econometric and credit risk modeling, accounting, and model risk governance. Different financial institutions will need more outside resources in some of these areas than others. The ideal one-stop CECL partner, therefore, will have a breadth and depth of expertise such that your financial institution can trust substantial CECL-related work to them, but enough modularity in their offering so that you only pay for the services you need.


In 2012, economists from the FHFA published a research paper describing a countercyclical approach for estimating the capital level required for mortgage portfolios to withstand future shocks to the housing sector. This approach uses state-level countercyclical stressed housing price paths (CSPs) based on where a state’s housing price levels are relative to its long-run trends and on the historical downside volatility of the state’s housing prices. FHFA has published a set of 51 of these 30-year CSPs (one for each state plus DC) starting from 13 different launch dates (each of the past 7 quarters—Q4 2013 through Q2 2015—as well as 6 quarters from 2003 to 2010).[1] While the approach is not new, we believe it provides an interesting alternative to the Federal Reserve’s annual Dodd Frank Act Stress Test (DFAST) stressed housing price scenarios because it is more transparent and more granular. This paper compares FHFA’s CSPs to the DFAST stressed HPI scenarios and outlines an approach for applying the state-level granularity of the CSPs to national-level DFAST Severely Adverse scenario.

Preparing for Model Validation: Ideas for Model Owners

Though not its intent, model validation can be disruptive to model owners and others seeking to carry out their day-to-day work. We have performed enough model validations over the past decade to have learned how cumbersome the process can be to business unit model owners and others we inconvenience with what at times must feel like an endless barrage of touch-point meetings, documentation requests and other questions relating to modeling inputs, outputs, and procedures.

Raising the Bar on CCAR Reliability

The data collected through the Capital Assessments and Stress Testing (FR Y–14)[1] schedules provide the Federal Reserve with the information and perspective needed to help ensure that large bank holding companies (BHCs) have strong, firm-wide risk measurement and management processes supporting their internal assessments of capital adequacy. Information gathered in this data collection effort is used in the supervision and regulation of these financial institutions. Therefore, large BHCs should have internal controls that ensure the integrity of reported results.