Get Started
Category: Article

RiskSpan VQI: Current Underwriting Standards Q3 2020

RiskSpan’s Vintage Quality Index, which had declined sharply in the first half of the year, leveled off somewhat in the third quarter, falling just 2.8 points between June and September, in contrast to its 12 point drop in Q2.

This change, which reflects a relative slowdown in the tightening of underwriting standards reflects something of a return to stability in the Agency origination market.

Driven by a drop in cash-out refinances (down 2.3% in the quarter), the VQI’s gradual decline left the standard credit-related risk attributes (FICO, LTV, and DTI) largely unchanged.

The share of High-LTV loans (loans with loan-to-value ratios over 80%) which fell 1.3% in Q3, has fallen dramatically over the last year–1.7% in total. More than half of this drop (6.1%) occurred before the start of the COVID-19 crisis. This suggests that, even though the Q3 VQI reflects tightening underwriting standards, the stability of the credit-related components, coupled with huge volumes from the GSEs, reflects a measure of stability in credit availability.

Risk Layers – September 20 – All Issued Loans By Count

Risk Layers – September 20 – All Issued Loans By Count

Analytical And Data Assumptions

Population assumptions:

  • Monthly data for Fannie Mae and Freddie Mac.

  • Loans originated more than three months prior to issuance are excluded because the index is meant to reflect current market conditions.

  • Loans likely to have been originated through the HARP program, as identified by LTV, MI coverage percentage, and loan purpose are also excluded. These loans do not represent credit availability in the market as they likely would not have been originated today but for the existence of HARP.                                                                                                                          

Data assumptions:

  • Freddie Mac data goes back to 12/2005. Fannie Mae only back to 12/2014.

  • Certain fields for Freddie Mac data were missing prior to 6/2008.   

GSE historical loan performance data release in support of GSE Risk Transfer activities was used to help back-fill data where it was missing.

An outline of our approach to data imputation can be found in our VQI Blog Post from October 28, 2015.                                                


LIBOR Transition: Winning the Fourth Quarter

In July 2017, the United Kingdom’s Financial Conduct Authority (FCA) announced that financial institutions will no longer be required to publish LIBOR rates after December 2021, signaling the effective end of LIBOR. Given that the FCA provided a four-year transition period for market participants to identify and implement alternative reference rates, market participants are rapidly approaching the “fourth quarter” of the transition away from LIBOR. 

Winning in the fourth quarter is more difficult when you finish the third quarter down by 28 points. And so, it is critical that institutions assess their progress to date in preparing for the cessation of LIBOR and making plans to implement an alternative reference rate. At this stage of an institution’s transition plan, a number of milestones need to be completed in order for an institution to reasonably consider itself “on-track.”  

These include having the workstreams listed below and a detailed plan in place to complete the execution of these tasks over the next year: 

  • LIBOR Transition Project Team Established – Financial institutions should have established a dedicated project team responsible for managing the transition from LIBOR. For larger institutions with LIBOR exposure in multiple business units, business unit leaders should be identified and made responsible for LIBOR transition activities in their business unit. 
  • Identification of LIBOR Exposure – Legacy contracts should already have been evaluated and exposure to LIBOR products maturing beyond year-end 2021 should have been quantified. During the upcoming year, monthly and quarterly updates on LIBOR exposure should be communicated to management. 
  • Assessment of LIBOR Contracts – Contracts should be reviewed to determine whether clear fallback language has been incorporated. Contracts with a) clear fallback language, b) fallback language requiring legal interpretation, and c) no fallback language must be identified and inventoried. 
  • Remediate Contracts without Clear Fallback Language – For contracts without adequate fallback language, institutions need to identify and finalize options for alternative reference rates, remediation plans, and a communication strategy with stakeholders when LIBOR is terminated. 
  • Assess Financial Exposure to Alternative Reference Rates – Because institutions will likely be impacted by exposure to alternative reference rates beginning in January 2022, plans need to be in the works for performing analyses on how the new alternative reference rate is likely to impact income, funding, liquidity, and capital levels.  
  • Stop Use of LIBOR on New Products – It may not need to be said, but one of the most effective methods of mitigating LIBOR exposure is to stop creating new LIBOR products.  To the extent new LIBOR products need to be issued, institutions must ensure that clear, easy-to-follow fallback language has been incorporated. 
  • Update and Remediate Technology – LIBOR is likely embedded in many applications and systems that set pricing on products, determine contractual payments, and determine the fair value for instruments. Plans need to be developed and implemented to update and test technology applications with LIBOR exposure.  

Consider engaging with external data and technology vendors to ensure operational readiness to transition away from LIBOR. Each business line and core function such as Finance or Treasury needs to inventory technology, operations, and modeling tools to ensure every LIBOR touch point is properly accounted for. 

  • Validate Models With LIBOR Assumptions – As we discussed last month, many models rely on LIBOR as an assumption or as part of the cash flow discounting mechanism.  Validators of models transitioning from LIBOR to an alternative reference rate need to account for this. And unscheduled validations may become necessary for models that might not otherwise be up for review before the end of 2021. 

The cessation of LIBOR is a significant event impacting a broad set of financial products and market segments. Because it is intertwined in the products, technology, and models of a financial institution, LIBOR transition must be sufficiently planned, resources must be mobilized, and alternative reference rates must be implemented into every business and process.  

The “fourth quarter” of the LIBOR transition game is upon us and the stakes are too high to rely on the second string. Financial institutions cannot underestimate the operational, technical, legal, communication, and risk management work required to move existing transactions off LIBOR and prepare for alternative reference rates. Although these efforts to transition from LIBOR should already be in full swing, they will continue to require additional time and resources. Teams that seem to be in control of the game still need to finish strong.  

Financial institutions that have not begun a comprehensive LIBOR transition plan are running out of time and will need to mount a furious fourth-quarter comeback. It’s not too late, but with the last year of the LIBOR transition dawning, financial institutions that are behind in their planning need to hustle. No one can afford to lose this game. The costs of failing to prepare are simply too high. 


EDGE: Unexplained Behavior for Idaho HFA

People familiar with specified pool trading recognize pools serviced by the state housing finance authorities as an expanding sector with a rich set of behavior. The Idaho Housing Finance Authority leads all HFAs in servicing volume, with roughly $18B in Fannie, Freddie and Ginnie loans.[1]

In the October prepay report, an outsized acceleration in speeds on FNMA pools serviced by the Idaho HFA caught our attention because no similar acceleration was occurring in FHLMC or GNMA pools.

Speeds on Idaho HFA-serviced pools for GNMA (orange), FHLMC (blue), and FNMA (black)

Digging deeper, we analyzed a set of FNMA pools totaling around $3.5B current face that were serviced entirely by the Idaho HFA. These pools experienced a sharp dip in reported forbearance from factor dates August through October, dropping from nearly 6% in forbearance to zero before rebounding to 4.5% (black line). By comparison, FHLMC pools serviced by the Idaho HFA (blue line) show no such change.

Seeking to understand what was driving this mysterious dip/rebound, we noticed in the October report that 2.7% of the Fannie UPB serviced by the Idaho HFA was repurchased (involuntarily) on account of being 120 days delinquent, thus triggering a large involuntary prepayment which was borne by investors.

We suspect that in the September report, loans that were in COVID-forbearance were inadvertently reclassified as not in forbearance. In turn, this clerical error released these loans from the GSE’s moratorium on repurchasing forbearance-delinquent loans and triggered an automatic buyout of these 120+ day delinquent loans by FNMA.

We have asked FNMA for clarification on the matter and they have responded that they are looking into it. We will share information as soon as we are aware of it.


[1] Idaho HFA services other states’ housing finance authority loans, including Washington state and several others.

.

If you are interested in seeing variations on this theme, contact us. Using Edge, we can examine any loan characteristic and generate a S-curve, aging curve, or time series.


 


EDGE: An Update on Property Inspection Waivers

In June, we wrote about the significant prepay differences observed between loans with full inspection/appraisals and loans with property inspection waivers (PIW). In this short piece, we revisit these relationships to see if the speed differentials have persisted over the previous four months.

From an origination standpoint, PIWs continue to gain in popularity and are beginning to approach half of all new issuance (blue line). For refi loans this figure approaches 60% (green line).

Graph 1: Percent of loans with property inspection waivers, by balance. Source: RiskSpan Edge

Performance

Broadly speaking, PIW loans still pay significantly faster than loans with appraisals. In our June report, the differential was around 15 CPR for the wider cohort of borrowers. Since that time, the relationship has held steady. Loans with inspection waivers go up the S-curve faster than loans with appraisals, and top out around 13-18 CPR faster, depending on how deep in the money the borrower is.

Graph 2: S-curves for loans aged 6-48 months with balance >225k, waivers (black) vs inspection (blue). Source: RiskSpan Edge. 
 

The differential is smaller for purchase loans. The first chart, which reflects only purchase loans, shows PIW loans paying only 10-12 CPR faster than loans with full appraisals. In contrast, refi loans (second chart) continue to show a larger differential, ranging from 15 to 20 CPR, depending on how deep in the money the loan is.

Graph 3: Purchase loans with waivers (black) versus inspections (blue). Source: RiskSpan Edge.

Graph 4: Refi loans with waivers (black) versus inspections (blue). Source: RiskSpan Edge.

We also compared bank-serviced loans with non-bank serviced loans. The PIW speed difference was comparable between the two groups of servicers, although non-bank speeds were in general faster for both appraisal and PIW loans.

Inspection waivers have been around since 2017 but have only gained popularity in the last year. While investors disagree on what is driving the speed differential, it could be as simple as self-selection: a borrower who qualifies for an inspection waiver will also qualify upon refinancing, unless that borrower takes out a large cash-out refi which pushes the LTV above 70%[1]. In any event, the speed differential between loans with waivers and loans with full inspections continues to hold over the last four months of factor updates. Given this, appraisal loans still offer significantly better prepay profiles at all refi incentives, along with a slightly flatter S-curve, implying lower option cost, than loans with inspection waivers.

If you are interested in seeing variations on this theme, contact us. Using Edge, we can examine any loan characteristic and generate a S-curve, aging curve, or time series.


[1] No-cash-out refis qualify for waivers up to 90% LTV.


Why Model Validators Need to Care About the LIBOR Transition

The transition to the Secured Overnight Financing Rate (SOFR) as a LIBOR replacement after 2021 creates layers of risk for banks. Many of these risks are readily apparent, others less so. But the factors banks must consider while choosing replacement rates and correctly implementing contractual fallback language makes a seamless transition a daunting proposition. Though sometimes overlooked, model risk managers have an important role in ensuring this happens correctly and in a way that does not jeopardize the reliability of model outputs.   

LIBOR, SOFR and the need for transition

A quick refresher: The London Interbank Offered Rate (LIBOR) currently serves as the benchmark at which major global banks lend to one another on a short-term basis in the international interbank market. LIBOR is calculated by the Intercontinental Exchange (ICE) and is published daily. LIBOR is based on a combination of five currencies and seven maturities. The most common of these is the three-month U.S. Dollar rate.

Accusations of manipulation by major banks going back as early as 2008, however, raised concerns about the sustainability of LIBOR. A committee convened by the Federal Reserve Board and the Federal Reserve Bank of New York in 2017—the Alternative Reference Rates Committee (ARRC)—identified a broad Treasury repurchase agreement (repo) financing rate as its preferred alternative reference rate to replace LIBOR after 2021. This repo rate (now known as SOFR) was chosen for its ability to provide liquidity to underlying markets and because the volumes underlying SOFR are far larger than any other U.S. money market. This combination of size and liquidity contributes to SOFR’s transparency and protects market participants from attempts at manipulation.

What Does This Mean for MRM?

Because the transition has potential bearing on so many layers of risk—market risk, operational risk, strategic risk, reputation risk, compliance risk, not to mention the myriad risks associated with mispricing assets—any model in a bank’s existing inventory that is tasked with gauging or remediating these risks is liable to be impacted. Understanding how and the extent to which models are considering how LIBOR transition may affect pricing and other core processes are (or should be) of principal concern to model validators.

Ongoing Monitoring and Benchmarking

Regulatory guidance and model validation best practices require testing model inputs and benchmarking how the model performs with the selected inputs relative to alternatives. For this reason, the validation any model whose outputs are sensitive to variable interest rates should include an assessment of how a replacement index (such as SOFR) and adjustment methodology were selected.

Model validators should be able to ascertain whether the model developer has documented enough evidence relating to:

  • Available reference rates and the appropriateness of each to the bank’s specific products
  • System capabilities for using these replacement rates with the bank’s products.
  • Control risks associated with unavailable alternative rates


Fallback Language considerations:

Fallback language—contractual provisions that govern the process for selecting a replacement rate in the event of LIBOR termination—should also factor into a validator’s assessment of model inputs. While many existing fallback provisions can be frustratingly vague when it comes to dealing with a permanent cessation of LIBOR, validators of models that rely on reference rates as inputs have an obligation to determining compliance with fallback language containing clear and executable terms. These include:

  • Specific triggers to enact the replacement rate
  • Clarity regarding the replacement rate and spread adjustments
  • Permissible options under fallback language – and whether other options might be more appropriate than the one ultimately selected based on the potential for valuation changes, liquidity impact, hedging implications, system changes needed, and customer impact

In November 2019, the ARRC published the finalized fallback language for residential adjustable rate mortgages, bilateral business loans, floating rate notes, securitizations, and syndicated loans. It has also actively engaged with the International Swap Derivatives Association (ISDA) to finalize the fallback parameters for derivatives.

The ARRC also recommended benchmark replacement rates adjusted for spread that would replace the current benchmark due to circumstances that trigger the replacement. The recommendation included the following benchmark replacement waterfalls. Validators of models relying on these replacements may choose, as part of their best practices review, to determine the extent to which existing fallback provisions align with the recommendations.

Replacement Description
Term SOFR + spread adjustment Forward-looking term SOFR for the applicable corresponding tenor. Note: Loan recommendations allow use of the next longest tenor term SOFR rate if the corresponding tenor is unavailable  
Compounded SOFR + spread Adjustment Compounded average of daily SOFRs over the relevant period depending on the tenor of USD LIBOR being replaced
Relevant selected rate + spread adjustment   Rate selected by the Relevant Governmental Body, lender, or borrower & administrative agent
Relevant ISDA replacement rate + spread adjustment The applicable replacement rate (without spread adjustment) that is embedded in ISDA’s standard definitions  
Issuer, designated transaction representative or noteholder replacement + spread adjustment An identified party will select a replacement rate, in some cases considering any industry-accepted rate in the related market. Note: in certain circumstances this step could be omitted


Model risk managers can sometimes be lulled into believing that the validation of interest rate inputs consists solely of verifying their source and confirming that they have been faithfully brought into the model. Ultimately, however, model validators are responsible for verifying not only the provenance of model inputs but also their appropriateness. Consequently, ensuring a smooth transition to the most appropriate available reference rate replacement is of paramount importance to risk management efforts related to the models these rates feed.


RESOURCES:

https://www.insideafricalaw.com/blog/benchmark-reform-the-impact-of-libor-transition-on-the-african-project-finance-market

https://www.occ.treas.gov/news-issuances/bulletins/2020/bulletin-2020-68.html

https://www.isda.org/a/n6tME/Supplemental-Consultation-on-USD-LIBOR-CDOR-HIBOR-and-SOR.pdf

https://www.investopedia.com/terms/l/libor.asp

https://www.newyorkfed.org/medialibrary/Microsites/arrc/files/2020/ARRC-factsheet.pdf

https://www.newyorkfed.org/arrc/sofr-transition

https://www.newyorkfed.org/medialibrary/Microsites/arrc/files/2019/LIBOR_Fallback_Language_Summary

https://www.isda.org/a/n6tME/Supplemental-Consultation-on-USD-LIBOR-CDOR-HIBOR-and-SOR.pdf

http://assets.isda.org/media/50b3fed0/47be9435-pdf/


The information within this section has been taken directly from the https://www.occ.treas.gov/news-issuances/bulletins/2020/bulletin-2020-68.html [AR1]


Managing Machine Learning Model Risk

Though the terms are often used interchangeably in casual conversation, machine learning is a subset of artificial intelligence. Simply put, ML is the process of getting a computer to learn the properties of one dataset and generalizing this “knowledge” on other datasets.


ML Financial Models

ML models have crept into virtually every corner of banking and finance — from fraud and money-laundering prevention to credit and prepayment forecasting, trading, servicing, and even marketing. These models take various forms (see Table 1, below). Modelers base their selection of a particular ML technique on a model’s objective and data availability.   

Table 1. ML Models and Application in Finance

Model Application
Linear Regression Credit Risk; Forecasting
Logistic Regression Credit Risk
Monte Carlo Simulation Capital Market; (ALM)
Artificial Neutral Networks Score Card and AML
Decision Trees Regression Models (Random Forest, Bagging) Score Card
Multinomial Logistic Regression Prepayment Projection
Deep Learning Prepayment Projection
Time Series Model Capital Forecasting; Macroeconomics Forecasting Model
Linear Regression with ARIMA Errors Capital Forecasting
Factor Models Short Rate Evolution
Fuzzy Matching AML; OFAC
Linear Discriminant Analysis (LDA) AML; OFAC
K Means Clustering AML; OFAC

 

ML models require large datasets relative to conventional models as well as more sophisticated computer programing and econometric/statistical skills. ML model developers are required to have deep knowledge about the ML model they want to use, its assumptions and limitations, and alternative approaches.

 

ML Model Risk

ML models present many of the same risks that accompany conventional models. As with any model, errors in design or application can lead to performance issues resulting in financial losses, poor decisions, and damage to reputation.

ML is all about algorithms. Failing to understand the mathematical aspects of these algorithms can lead to adopting inefficient optimization algorithms without knowing the nature or the interpretation of the optimization being solved. Making decisions under these circumstances increases model risk and can lead to unreliable outputs.

As sometimes befalls conventional regression models, ML models may perform well on the training data but not on the test data. Their complexity and high dimensionality makes them especially susceptible to overfitting. The poor performance of some ML models when applied beyond the training dataset can translate into a huge source of risk.

Finally, ML models can give rise to unintended consequences when used inappropriately or incorrectly. Model risk is magnified when the goal of a ML model’s algorithm is not aligned with the business problem or doesn’t consider all relevant considerations of the business problem. Model risk also arises when an ML model is used outside the environment for which it was designed. These risks include overstated/understated model outputs and lack of fairness. Table 2, below, presents a more comprehensive list of these risks.

Table 2. Potential risk from ML models

Overfitting
Underfitting
Bias toward protected groups
Interpretability
Complexity
Use of poor-quality data
Job displacement
Models may produce socially unacceptable results
Automation may create model governance issues

 

Managing ML Model Risk

managing ML model risk

It may seem self-evident, but the first step in managing ML model risk consists of reliably  identifying every model in the inventory that relies on machine learning. This exercise is not always as straightforward as it might seem. Successfully identifying all ML models requires MRM departments to incorporate the right information requests into their model determination or model assessment forms. These should include questions designed to identify specific considerations of ML model techniques, algorithms, platforms and capabilities. MRM departments need to adopt a consistent but flexible definition about what constitutes an ML model across the institution. Models developers, owners and users should be trained in identifying ML models and those features that need to be reported in the model identification assessment form.

MRM’s next step involves risk assessing ML models in the inventory. As with traditional models, ML models should be risk assessed based on their complexity, materiality and frequency of use. Because of their complexity, however, ML models require an additional level of screening in order to account for data structure, level of algorithm sophistication, number of hyper-parameters, and how the models are calibrated. The questionnaire MRM uses to assess the risk of its conventional models often needs to be enhanced in order to adequately capture the additional risk dimensions introduced by ML models.

Managing ML model risk also involves not only ensuring that a clear model development and implementation process is in place but also that it is consistent with the business objective and the intended use of the models. Thorough documentation is important for any model, but the need to describe model theory, methodology, design and logic takes on added importance when it comes to ML models. This includes specifying the methodology (regression or classification), the type of model (linear regression, logistic regression natural language processing, etc.), the resampling method (cross-validation, bootstrap) and the subset selection method such as backward, forward or stepwise selection. Obviously, simply stating that the model “relies on a variety of machine learning techniques” is not going to pass muster.

As with traditional models, developers must document the data source, quality and any transformations that are performed. This includes listing the data sources, normalization and sampling techniques, training and test data size, the data dimension reduction technique (principal component, partial least squares, etc.) as well as controls around them. An assessment of the risk around the utilization of certain data should also be assessed.

A model implementation plan and controls around the model should be also be developed.

Finally, all model performance testing should be clearly stated, and the results documented. This helps assess whether the model is performing as intended and in line with its design and business objective. Limitations and calibrations around the models should also be documented.

Like traditional models, ML models require independent validation to ensure they are sound and performing as intended and to identify potential limitations. All components of ML models should be subject to validation, including conceptual soundness, outcomes analysis and ongoing monitoring.

Validators can assess the conceptional soundness of an ML model by evaluating its design and construction, focusing on the theory, methodology, assumptions and limitations, data quality and integrity, hyper-parameter calibration and overlays, bias and interpretability.

Validators can assess outcomes analysis by checking whether the model outputs are appropriate and in line with a priori expectations. Results of the performance metrics should also be assessed for accuracy and degree of precision. Performance metrics for ML models vary by model type. Similar to traditional predictive models, common performance metrics for ML models include the mean-squared-error (MSE), Gini coefficient, entropy, the confusion matrix, and the receiver operating characteristic (ROC) curve.

Outcomes analysis should also include out-of-sample testing, which can be conducted using cross-validation techniques. Finally, ongoing monitoring should be reviewed as a core element of the validation process. Validators should evaluate whether model use is appropriate given changes in products, exposures and market conditions. Validators should also ensure performance metrics are being monitored regularly based on the inherent risk of the model and frequency of use. Validators should ensure that a continuous performance monitoring plan exists and captures the most important metrics. Also, a change control document and access control document should be available.  

The principles outlined above will sound familiar to any experienced model validator—even one with no ML training or experience. ML models do not upend the framework of MRM best practices but rather add a layer of complexity to their implementation. This complexity requires MRM departments in many cases to adjust their existing procedures to property identify ML models and suitably capture the risk emerging from them. As is almost always the case, aggressive staff training to ensure that their well-considered process enhancements are faithfully executed and have their desired effect.       


EDGE: GNMA Delinquencies and Non-Bank Servicers

In the past two months, investors have seen outsized buyouts of delinquent loans from GNMA pools, leading to a significant uptick in prepayment speeds. Nearly all of these buyouts were driven by bank servicers, including Wells Fargo, US Bank, Truist, and Chase. GNMA buyout speeds in July’s report were the fastest, with Wells Fargo leading the charge on their seriously delinquent loans. The August report saw lower but still above-normal buyout activity. For September, we expect a further decline in bank buyout speeds, as the 60-day delinquent bucket for banks has declined from 6.6% just prior to the July report to 2.2% today.[1]

During that same time, buyouts from non-banks were nearly non-existent. We note that the roll rate from 60-day delinquent to 90-day delinquent (buyout-eligible) is comparable between banks and non-banks.[2] So buyout-eligible delinquencies for non-banks continue to build. That pipeline, coupled with the fact that non-banks service more than 75% of GNMA’s current balance, presents a substantial risk of future GNMA buyouts.

As discussed in previous posts, the differential in buyouts between banks and their non-bank counterparts is mainly due to bank servicers being able to warehouse delinquent loans until they reperform, modified or unmodified, or until they can otherwise dispose of the loan. Non-bank servicers typically do not have the balance sheet or funding to perform such buyouts in size. If these large non-bank servicers were to team with entities with access to cheap funding or were to set up funding facilities sponsored by investors, they could start to take advantage of the upside in re-securitization. The profits from securitizing reperforming loans is substantial, so non-bank servicers can afford to share the upside with yield-starved investors in return for access to funding. In this scenario, both parties could engage in a profitable trade.

Where do delinquencies stand for non-bank servicers? In the table below, we summarize the percentage of loans that have missed 3 or more payments for the top five non-bank servicers, by coupon and vintage.[3] In this table, we show 90-day+ delinquencies, which are already eligible for buyout, as opposed to the 60 day delinquency analysis we performed for banks, where 60 day delinquencies feed the buyout-eligible bucket via a 75% to 80% roll-rate from 60-day to 90-day delinquent.

30 yr GN2 Multi-lender pools

In this table, 2017-19 vintage GN2 3.5 through 4.5s show the largest overhang of non-bank delinquencies coupled with the largest percentage of non-bank servicing for the cohort.

We summarize delinquencies for the top five non-bank servicers because they presumably have a better chance at accessing liquidity from capital markets than smaller non-bank servicers. However, we observe significant build-up of 90-day+ delinquency across all non-bank servicers, which currently stands at 7.7% of non-bank UPB, much higher than the 6.6% bank-serviced 60-day delinquency in June.

Within the top five non-bank servicers, Penny Mac tended to have the largest buildup of 90-day+ delinquencies and Quicken tended to have the lowest but results varied from cohort to cohort. In the graph below, we show the 90+ delinquency pipeline for all GN2 30yr multi-lender pools.

90+ DQ in GN2 Multi-lender Pools

While we cannot say for certain when (or if) the market will see significant buyout activity from non-bank servicers, seriously delinquent loans continue to build. This overhang of delinquent loans, coupled with the significant profits to be made from securitizing reperforming loans, poses the risk for a significant uptick in involuntary speeds in GN2 multi-lender pools.[4]

If you interested in seeing variations on this theme, contact us. Using Edge, we can examine any loan characteristic and generate a S-curve, aging curve, or time series.


[1] For this analysis, we focused on the roll rate for loans in 30yr GN2 Multi-lender pools vintage 2010 onward. See RiskSpan for analysis of other GNMA cohorts.

[2] Over the past two months, 77% of bank-serviced loans that were 60-days delinquent rolled to a buyout-eligible delinquency state compared to 75% for non-banks.

[3] This analysis was performed for loans that are securitized in 30yr GN2 multi-lender pools issued 2010 onward. The top five servicers include Lakeview, Penny Mac, Freedom, Quicken, and Nationstar (Mr. Cooper).

[4] Reperforming loans could include modifications or cures without modification. Even with a six-month waiting period for securitizing non-modified reperforming loans, the time-value of borrowing at current rates should prove only a mild hinderance to repurchases given the substantial profits on pooling reperforming loans.


Consistent & Transparent Forbearance Reporting Needed in the PLS Market

There is justified concern within the investor community regarding the residential mortgage loans currently in forbearance and their ultimate resolution. Although most of the 4M loans in forbearance are in securities backed by the Federal Government (Fannie Mae, Freddie Mac or Ginnie Mae), approximately 400,000 loans currently in forbearance represent collateral that backs private-label residential mortgage-backed securities (PLS). The PLS market operates without clear, articulated standards for forbearance programs and lacks the reporting practices that exist in Agency markets. This leads to disparate practices for granting forbearance to borrowers and a broad range of investor reporting by different servicers. COVID-19 has highlighted the need for transparent, consistent reporting of forbearance data to investors to support a more efficient PLS market.

Inconsistent investor reporting leaves too much for interpretation. It creates investor angst while making it harder to understand the credit risk associated with underlying mortgage loans. RiskSpan performed an analysis of 2,542 PLS deals (U.S. only) for which loan-level foreclosure metrics are available. The data shows that approximately 78% of loans reported to be in forbearance were backing deals originated between 2005-2008 (“Legacy Bonds”).  As you would expect, new issue PLS has a smaller percentage of loans reported to be in forbearance.

% total forebearance UPB

Not all loans in forbearance will perform the same and it is critical for investors to receive transparent reporting of underlying collateral within their PLS portfolio in forbearance.  These are unchartered times and, unlike historic observations of borrowers requesting forbearance, many loans presently in forbearance are still current on their mortgage payments. In these cases, they have elected to join a forbearance program in case they need it at some future point. Improved forbearance reporting will help investors better understand if borrowers will eventually need to defer payments, modify loan terms, or default leading to foreclosure or sale of the property.

In practice, servicers have followed GSE guidance when conducting forbearance reviews and approval. However, without specific guidance, servicers are working with inconsistent policies and procedures developed on a company-by-company basis to support the COVID forbearance process. For example, borrowers can be forborne for 12-months according to FHFA guidance. Some servicers have elected to take a more conservative approach and are providing forbearance in 3-month increments with extensions possible once a borrower confirms they remain financially impacted by the COVID pandemic.

Servicers have the data that investors want to analyze. Inconsistent practices in the reporting of COVID forbearances by servicers and trustees has resulted in forbearance data being unavailable on certain transactions. This means investors are not able to get a clear picture of the financial health of borrowers in transactions. In some cases, trustees are not reporting forbearance information to investors which makes it nearly impossible to obtain a reliable credit assessment of the underlying collateral.  

The PLS market has attempted to identify best practices for monthly loan-level reporting to properly assess the risk of loans where forbearance has been granted.  Unfortunately, the current market crisis has highlighted that not all market participants have adopted the best practices and there are not clear advantages for issuers and servicers to provide clear, transparent forbearance reporting. At a minimum, RiskSpan recommends that the following forbearance data elements be reported by servicers for PLS transactions:

  • Last Payment Date: The last contractual payment date for a loan (i.e. the loan’s “paid- through date”).
  • Loss Mitigation Type: A code indicating the type of loss mitigation the servicer is pursuing with the borrower, loan, or property.
  • Forbearance Plan Start Date: The start date when either a) no payment or b) a payment amount less than the contractual obligation has been granted to the borrower.
  • Forbearance Plan Scheduled End Date: The date on which a Forbearance Plan is scheduled to end.
  • Forbearance Exit – Reason Code: The reason provided by the borrower for exiting a forbearance plan.
  • Forbearance Extension Requested: Flag indicating the borrower has requested one or more forbearance extensions.
  • Repayment Plan Start Date: The start date for when a borrower has agreed to make monthly mortgage payments greater than the contractual installment in an effort to repay amounts due during a Forbearance Plan.
  • Repayment Plan Scheduled End Date: The date at which a Repayment Plan is scheduled to end.
  • Repayment Plan Violation Date: The date when the borrower ceased complying with the terms of a defined repayment plan.

The COVID pandemic has highlighted monthly reporting weaknesses by servicers in PLS transactions. Based on investor discussions, additional information is needed to accurately assess the financial health of the underlying collateral. Market participants should take the lessons learned from the current crisis to re-examine prior attempts to define monthly reporting best practices. This includes working with industry groups and regulators to implement consistent, transparent reporting policies and procedures that provide investors with improved forbearance data.


Machine Learning Models: Benefits and Challenges

Having good Prepayment and Credit Models is critical in the analysis of Residential Mortgage-Backed Securities. Prepays and Defaults are the two biggest risk factors that traders, portfolio managers and originators have to deal with. Traditionally, regression-based Behavioral Models have been used to accurately predict human behavior. Since prepayments and defaults are not just complex human decisions but also competing risks, accurately modeling them has been challenging. With the exponential growth in computing power (GPUs, parallel processing), storage (Cloud), “Big Data” (tremendous amount of detailed historical data) and connectivity (high speed internet), Artificial Intelligence (AI) has gained significant importance over the last few years. Machine Learning (ML) is a subset of AI and Deep Learning (DL) is a further subset of ML. The diagram below illustrates this relationship:

Due to the technological advancements mentioned above, ML based prepayment and credit models are now a reality. They can achieve better predictive power than traditional models and can deal effectively with high-dimensionality (more input variables) and non-linear relationships. The major drawback which has kept them from being universally adopted is their “black box” nature which leads to validation and interpretation issues. Let’s do a quick comparison between traditional and ML models:

behavioral models versus machine learning models

Within ML Models are two ways to train them:

  • Supervised Learning  (used for ML Prepay and Credit Models)
    • Regression based
    • Classification based
  • Unsupervised Learning
    • Clustering
    • Association

Let’s compare the major differences between Supervised and Unsupervised Learning:

Supervised learning versus unsupervised learning

The large amounts of loan level time series data available for RMBS (agency and non-agency) lends itself well for the construction of ML models and early adopters have reported higher accuracy. Besides the obvious objections mentioned above (black box, lack of control, interpretation) ML models are also susceptible to overfitting (like all other models). Overfitting is when a model does very well on the training data but less well on unseen data (validation set). The model ends up “memorizing” the noise and outliers in the input data and is not able to generalize accurately. The non-parametric and non-linear nature of ML Models accentuates this problem. Several techniques have been developed to address this potential problem: reducing the complexity of decision trees, expanding the training dataset, adding weak learners, dropouts, regularization, reducing the training time, cross validation etc.. The interpretation problem is a bit more challenging since users demand both, predictive accuracy and some form of interpretability. Several interpretation methods are used currently, like PDP (Partial dependence plot), ALE (accumulated local effects), PFI (permutation feature importance) and ICE (individual conditional expectation) but each has its shortcomings. Some of the challenges with the interpretability methods are:

  • Isolating Cause and Effect – This is not often possible with supervised ML models since they only exploit associations and do not explicitly model cause/effect relationships.
  • Mistaking Correlation for Dependence – Independent variables have a correlation coefficient of zero but a zero correlation coefficient may not imply independence. The correlation coefficient only tracks linear correlations and the non-linear nature of the models makes this difficult.
  • Feature interaction and dependence – An incorrect conclusion can be drawn about the features influence on the target when there are interactions and dependencies between them.

While ML based prepay and credit models offer better predictive accuracy and automatically capture feature interactions and non-linear effects, they are still a few years away from gaining widespread acceptance. A good use for such models, at this stage, would be to use them in conjunction with traditional models. They would be a good benchmark to test traditional models with.


Note: Some of the information on this post was obtained from publicly available sources on the internet. The author wishes to thank  Lei Zhao and Du Tang of the modeling group for proofreading this post.


The Why and How of a Successful SAS-to-Python Model Migration

A growing number of financial institutions are migrating their modeling codebases from SAS to Python. There are many reasons for this, some of which may be unique to the organization in question, but many apply universally. Because of our familiarity not only with both coding languages but with the financial models they power, my colleagues and I have had occasion to help several clients with this transition.

Here are some things we’ve learned from this experience and what we believe is driving this change.

Python Popularity

The popularity of Python has skyrocketed in recent years. Its intuitive syntax and a wide array of packages available to aid in development make it one of the most user-friendly programming languages in use today. This accessibility allows users who may not have a coding background to use Python as a gateway into the world of software development and expand their toolbox of professional qualifications.

Companies appreciate this as well. As an open-source language with tons of resources and low overhead costs, Python is also attractive from an expense perspective. A cost-conscious option that resonates with developers and analysts is a win-win when deciding on a codebase.

Note: R is another popular and powerful open-source language for data analytics. Unlike R, however, which is specifically used for statistical analysis, Python can be used for a wider range of uses, including UI design, web development, business applications, and others. This flexibility makes Python attractive to companies seeking synchronicity — the ability for developers to transition seamlessly among teams. R remains popular in academic circles where a powerful, easy-to-understand tool is needed to perform statistical analysis, but additional flexibility is not necessarily required. Hence, we are limiting our discussion here to Python.

Python is not without its drawbacks. As an open-source language, less oversight governs newly added features and packages. Consequently, while updates may be quicker, they are also more prone to error than SAS’s, which are always thoroughly tested prior to release.

Visualization Capabilities

While both codebases support data visualization, Python’s packages are generally viewed more favorably than SAS’s, which tend to be on the more basic side. More advanced visuals are available from SAS, but they require the SAS Visual Analytics platform, which comes at an added cost.

Python’s popular visualization packages — matplotlib, plotly, and seaborn, among others — can be leveraged to create powerful and detailed visualizations by simply importing the libraries into the existing codebase.

Accessibility

SAS is a command-driven software package used for statistical analysis and data visualization. Though available only for Windows operating systems, it remains one of the most widely used statistical software packages in both industry and academia.

It’s not hard to see why. For financial institutions with large amounts of data, SAS has been an extremely valuable tool. It is a well-documented language, with many online resources and is relatively intuitive to pick up and understand – especially when users have prior experience with SQL. SAS is also one of the few tools with a customer support line.

SAS, however, is a paid service, and at a standalone level, the costs can be quite prohibitive, particularly for smaller companies and start-ups. Complete access to the full breadth of SAS and its supporting tools tends to be available only to larger and more established organizations. These costs are likely fueling its recent drop-off in popularity. New users simply cannot access it as easily as they can Python. While an academic/university version of the software is available free of charge for individual use, its feature set is limited. Therefore, for new users and start-up companies, SAS may not be the best choice, despite being a powerful tool. Additionally, with the expansion and maturity of the variety of packages that Python offers, many of the analytical abilities of Python now rival those of SAS, making it an attractive, cost-effective option even for very large firms.

Future of tech

Many of the expected advances in data analytics and tech in general are clearly pointing toward deep learning, machine learning, and artificial intelligence in general. These are especially attractive to companies dealing with large amounts of data.

While the technology to analyze data with complete independence is still emerging, Python is better situated to support companies that have begun laying the groundwork for these developments. Python’s rapidly expanding libraries for artificial intelligence and machine learning will likely make future transitions to deep learning algorithms more seamless.

While SAS has made some strides toward adding machine learning and deep learning functionalities to its repertoire, Python remains ahead and consistently ranks as the best language for deep learning and machine learning projects. This creates a symbiotic relationship between the language and its users. Developers use Python to develop ML projects since it is currently best suited for the job, which in turn expands Python’s ML capabilities — a cycle which practically cements Python’s position as the best language for future development in the AI sphere.

Overcoming the Challenges of a SAS-to-Python Migration

SAS-to-Python migrations bring a unique set of challenges that need to be considered. These include the following.

Memory overhead

Server space is getting cheaper but it’s not free. Although Python’s data analytics capabilities rival SAS’s, Python requires more memory overhead. Companies working with extremely large datasets will likely need to factor in the cost of extra server space. These costs are not likely to alter the decision to migrate, but they also should not be overlooked.

The SAS server

All SAS commands are run on SAS’s own server. This tightly controlled ecosystem makes SAS much faster than Python, which does not have the same infrastructure out of the box. Therefore, optimizing Python code can be a significant challenge during SAS-to-Python migrations, particularly when tackling it for the first time.

SAS packages vs Python packages

Calculations performed using SAS packages vs. Python packages can result in differences, which, while generally minuscule, cannot always be ignored. Depending on the type of data, this can pose an issue. And getting an exact match between values calculated in SAS and values calculated in Python may be difficult.

For example, the true value of “0” as a float datatype in SAS is approximated to 3.552714E-150, while in Python float “0” is approximated to 3602879701896397/255. These values do not create noticeable differences in most calculations. But some financial models demand more precision than others. And over the course of multiple calculations which build upon each other, they can create differences in fractional values. These differences must be reconciled and accounted for.

Comparing large datasets

One of the most common functions when working with large datasets involves evaluating how they change over time. SAS has a built-in function (proccompare) which compares datasets swiftly and easily as required. Python has packages for this as well; however, these packages are not as robust as their SAS counterparts. 

Conclusion

In most cases, the benefits of migrating from SAS to Python outweigh the challenges associated with going through the process. The envisioned savings can sometimes be attractive enough to cause firms to trivialize the transition costs. This should be avoided. A successful migration requires taking full account of the obstacles and making plans to mitigate them. Involving the right people from the outset — analysts well versed in both languages who have encountered and worked through the pitfalls — is key.


Get Started