Linkedin    Twitter   Facebook

Get Started
Log In

Linkedin

Blog Archives
Home / Blog Archives

Reviving the Private-Label RMBS Market with Improvements to the Securitization Process

Weaknesses in securitization processes for mortgage loans contributed to the financial crisis of 2007 – 2008 and have led to a decade-long stagnation in the private-label residential mortgage-backed securities (PLS) market.

Although market participants have attempted to improve known weaknesses, lack of demand for private-label RMBS reflects investors’ reluctance to re-enter the market and the need for continued improvements to securitization processes to re-establish market activity.  While significant issues still need to be addressed, promising advances have been made in the PLS market that improve information provided to investors as well as checks and balances designed to boost transaction performance.

Specifically, we are beginning to see significant improvements in the following securitization processes:

  • Due Diligence
  • Rating Agency Assessment
  • Representation and Warranty Framework and Enforcement
  • Loan Quality Standards
  • Risk Retention
  • Bondholder Communication

Enhancements to these processes in the post-crisis PLS market improve transparency; align incentives between issuers, sponsors, and investors; and may lead to increased investor trust in this market segment.

Due Diligence

The due diligence process is intended to provide the purchaser of an asset with an opportunity to assess the asset’s quality. Prior to the financial crisis, investors relied on the underwriter of the securitization (i.e., an investment bank) to perform loan-level due diligence on their behalf and assess the quality of the underlying loans. Limited information about these reviews was made available to investors. The process was opaque and did not provide investors a clear view of the quality of loans underlying a securitization.

Prior to the financial crisis, due diligence was performed on between 5% and 10% of the loans in a securitization. (Slightly larger samples were selected for Alt-A and subprime transactions.) The criteria for selecting the specific loans in the sample was generally not communicated to investors and rating agencies. Even more odd, the due diligence results were not communicated to key transaction parties (rating agencies and investors) and issuers did not disclose the results in disclosure documents.

Since the crisis, the following improvements to the due diligence process have made it more transparent:

  • While specific due diligence sample sizes have not been mandated, securitizations issued since the financial crisis have significantly increased the percentage of loans being reviewed—in many transactions, issuers have even included all loans. In two recent Prime Jumbo securitizations, Flagstar and JPMorgan Chase performed 100% due diligence on the underlying loans.
  • Rating Agencies have defined requirements for the firms that perform due diligence activities. Market participants have recommended standards for the scope of the due diligence performed. For example, the Structured Finance Industry Group (“SFIG”) has outlined general criteria for the review of credit, property valuation and regulatory compliance on loans reviewed during the due diligence.
  • Due diligence results are provided to all rating agencies under SEC Rule 17g-10. These reports detail the number of loans reviewed, due diligence findings, the number of loans dropped during the due diligence process, and the rationale behind dropping them. The reports summarize grades assigned to each loan based on rating agency criteria and are made available on the Securities and Exchange Commission (“SEC”)’s EDGAR site as well as in securitization disclosure documents.
  • If a transaction is rated, issuers are required to file detailed reports of due diligence results with the SEC (Rule 17-Ga2 filings) at least five business days prior to first sale of an offered security. Examples of summary reports for both the Flagstar and JPMorgan Chase securitizations show the additional information on due diligence results provided to investors. For those investors interested in more detail, loan-level reporting of the due diligence findings is also available on EDGAR.

This increased transparency enables investors to independently assess the quality of mortgage loans in a private-label RMBS transaction and factor the results of the due diligence process into their investment decision.

Rating Agency Assessment Process

Over-reliance on rating agencies and the conflict of interest caused by the “issuer pay” model for credit ratings is a frequently cited problem with pre-crisis private-label RMBS transactions. Passage of the Dodd-Frank Act is expected to help reduce the blind reliance by investors and regulators on the ratings process by eliminating the use of credit ratings within the regulatory framework and increasing independent due diligence by investors. Despite tremendous criticism of the “issuer pay” model, the system remains intact almost a decade after the financial crisis across multiple asset classes, including corporate bonds and municipal bonds. The Dodd-Frank Act, however, now requires rating agencies to establish “firewalls” between their business development processes and their ratings processes.

With the criticism levied on the performance and opacity of the rating agency assessment process, the SEC Rule 17g-7 requires public disclosures from rating agencies whenever they provide a credit rating.  With these new disclosures, rating agencies have increased the transparency of the ratings process by making public the following changes to their assessment process:

  • Assumptions, methodologies, and processes used to rate transactions
  • Pre-Sale Reports that outline how a rating agency reviews the specific transaction, including areas such as the capital structure, cash flow triggers, pool characteristics, loan underwriting criteria, representations and warranties, and origination and servicing practices

While many market participants criticize the pre-crisis methodologies used by rating agencies to establish credit enhancement levels, pre-sale reports detail reviews performed on each rated private-label RMBS transaction and the assessments made by rating agencies to compute the expected credit enhancement requirements to support the securitization ratings.

In response to a weak pre-crisis representation and warranty framework (discussed in greater detail in the following section), rating agencies now publish “market standard” representations and warranties for each asset class and compare the representations and warranties in each private-label RMBS transaction being evaluated against the standard. The rating agencies also assess a transaction’s processes for enforcing representations and warranties (including repurchases) when a breach occurs.

Rating agencies typically publish the pre-sale report and their assessment of the representations and warranties a few days before a new private-label RMBS issuance is priced. Together with the preliminary offering documents, these items provide post-crisis PLS market investors a comprehensive view of the transaction’s risk prior to making a pricing / investment decision.

Finally, in another step to reduce the risk of issuers “shopping” for favorable ratings, SEC Rule 17g-5 requires rating agencies to make information provided to them by an issuer available to all other rating agencies. This allows other rating agencies to assess transactions on an equal basis and reach independent conclusions – using the same data – on credit enhancement requirements.

One measure of whether the rating agency process has changed since the crisis is the credit enhancement levels themselves. Higher credit enhancement levels would tend to suggest more stringent ratings. Credit enhancement levels on prime jumbo private-label RMBS can be observed in the tables below.

Post-Crisis Transaction Summary:

Pre-Crisis Transaction Summary:

In general, post-crisis AAA credit enhancement levels are higher today compared to pre-crisis AAA credit enhancement levels, which generally ranged between 3.50% – 4.00%. The rating agency assessment process has become more transparent since the crisis, and credit enhancement levels have increased. The future performance of these transactions will determine whether these changes are sufficient.

Representation and Warranty Framework and Enforcement

Representations and warranties are designed to allocate risks associated with a securitization’s underlying loans between issuers and investors. Basic principles of an effective process for allocating risks associated with underwriting standards, collateral value, or regulatory compliance include:

  • Clear rules (i.e., representations and warranties) defining when loans must be repurchased out of the security
  • Transparent and robust methods for identifying loans that may cause losses
  • Financial stability of the entity responsible for funding required loan repurchases

One criticism of the pre-crisis PLS market was the lack of an independent party tasked with identifying rep and warrant breaches. In many cases, the issuers or sponsors themselves were the only transaction parties capable of conducting the type of forensic loan review necessary to discover breaches. However, because these very parties would be on the hook to fund any repurchases required by their analyses, investors had reason to question the thoroughness of these reviews.

In response, the post-crisis PLS market has generally adapted a rules-based approach that relies on delinquency and other objective “triggers” to review loans and identify potential representation and warranty breaches. Once triggered, reviews are often performed by either 1) an independent third-party with forensic review capabilities, or 2) the holder of the most subordinate outstanding security. Reviews are no longer performed or controlled by issuers whose incentive to identify a breach could be questioned.

These process improvements are meant to increase the likelihood that potential representation and warranty breaches are identified and their terms enforced. If a loan meets the contractual requirements for a repurchase, it is critical that the entity responsible for repurchasing it has the financial ability to do so. New SEC disclosure requirements (Rule 15-Ga1) help track and assess an issuer’s ability to comply with repurchase requests.

Changes in the representation and warranty framework have improved methods for breach identification, evaluation, and enforcement. These changes have increased transparency, clarified the allocation of risk, contractually established roles for identifying and evaluating potential breaches, and brought about more effective enforcement mechanisms.

Loan Quality Standards

The Dodd-Frank Act requires lenders to make a good faith effort to determine borrowers’ ability to repay (ATR) their mortgage obligations. The ATR rule seeks to discourage some of the practices used to originate pre-crisis mortgage loans and requires lenders to consider certain underwriting criteria, such as the borrower’s assets or income, debt load, and credit history, to determine whether a loan can be repaid.

Lenders are presumed to comply with the ATR rule when they originate a “qualified mortgage” (QM) which meets the requirements of the ATR rule and additional underwriting and pricing standards. These requirements generally include a limit on points and fees, along with various restrictions on loan terms and features.2

Risk Retention

The risk retention requirements added by Section 15G of the Securities Exchange Act of 1934 generally require the issuer of securities backed by non-QM loans to retain at least 5 percent of the credit risk of the mortgage loans collateralizing the securities. This rule change helps align the interests of issuers and sponsors with those of investors by requiring issuers and sponsors to retain an economic interest in the credit risk of the assets they securitize. The rule allows issuers and sponsors to retain risk as either a horizontal interest (i.e., retaining the most subordinate 5% of the securitization), a vertical interest (i.e., retaining a “slice” of each security issued), an “L-shaped” interest (i.e., a combination of horizontal and vertical), or a cash reserve account.

For most non-QM securitizations, the issuers and sponsors have migrated towards the vertical interest, which performs like whole loan exposure and avoids the comprehensive fair value disclosures required for retained horizontal interests. At the margin, this change will create “skin in the game” for non-QM issuers and sponsors and better align their incentives with those of investors.

Bondholder Communication

To address concerns expressed by investors in locating other investors to enforce contractual rights, recent private-label RMBS transactions have incorporated mechanisms for investors to communicate with each other. Many transactions have incorporated methods for investors who wish to communicate to be included in a transaction registry, which may allow them to reach the required percentage of security holders necessary to provide specific direction to the trustee.

Summary

The PLS market has experienced a decade of stagnation since the financial crisis of 2007 – 2008. Notwithstanding new entrants to this market, a persistent lack of investor trust in and demand for private-label RMBS remains a challenge. While opportunities for improvement remain, major improvements to the securitization process are beginning to take hold.  These changes in post-crisis private-label RMBS transactions improve transparency, align the incentives of issuers and sponsors with those of investors, and hold the key to attracting investors back to this once-thriving market segment.

[1] Include loans with original term less than 20 years.

[2] Unpermitted features include negative amortization, interest-only payments, loan terms of more than 30 years, and “back-end” debt-to-income ratios above 43%. (The back-end debt-to-income ratio limit does not apply to 1) loans guaranteed by the Federal Housing Administration and Veterans Administration, 2) loans eligible for purchase by Fannie Mae and Freddie Mac, and 3) portfolio loans made by “small creditors.”)

Machine Learning and Portfolio Performance Analysis

Attribution analysis of portfolios typically aims to discover the impact that a portfolio manager’s investment choices and strategies had on overall profitability. They can help determine whether success was the result of an educated choice or simply good luck. Usually a benchmark is chosen and the portfolio’s performance is assessed relative to it.

This post, however, considers the question of whether a non-referential assessment is possible. That is, can we deconstruct and assess a portfolio’s performance without employing a benchmark? Such an analysis would require access to historical return as well as the portfolio’s weights and perhaps the volatility of interest rates, if some of the components exhibit a dependence on them. This list of required variables is by no means exhaustive.

There are two prevalent approaches to attribution analysis—one based on factor models and the other on return decomposition. The factor model approach considers the equities in a portfolio at a single point in time and attributes performance to various macro- and micro-economic factors prevalent at that time. The effects of these factors are aggregated at the portfolio level and a qualitative assessment is done. Return decomposition, on the other hand, explores the manner in which positive portfolio returns are achieved across time. The principal drivers of performance are separated and further analyzed. In addition to a year’s worth of time series data for the variables listed in the previous paragraph, covariance, correlation, and cluster analyses and other mathematical methods would likely be required.

Normality Assumption

Is the normality assumption for stock returns fully justified? Are sample means and variances good proxies for population means and variances? This assumption is worth testing because Normality and the Central Limit Theorem are widely assumed when dealing with financial data. The Delta-Normal Value at Risk (VaR) method, which is widely used to compute portfolio VaR, assumes that stock returns and allied risk factors are normally distributed. Normality is also implicitly assumed in financial literature. Consider the distribution of S&P returns from May 1980 to May 2017 displayed in Figure 1.

Figure One: Distribution of S&P Returns

Panel (a) is a histogram of S&P daily returns from January 2001 to January 2017. The red curve is a Gaussian fit. Panel (b) shows the same data on a semi-log plot (logarithmic Y axis). The semi-log plot emphasizes the tail events.

The returns displayed in the left panel of figure 1 have a higher central peak and the “shoulders” are somewhat wider than what is predicted by the Gaussian fit. This mismatch in the tails is more visible in the semi-log plot shown in panel (b). This demonstrates that a normal distribution is probably not a very accurate assumption. Sigma, the standard deviation, is typically used as a measure of the relative magnitude of market moves and as a rough proxy for the occurrence of such events. The normal distribution places the odds of a minus-5 sigma swing at only 2.86×10-5 %. In other words, assuming 252 trading days per year, a drop of this magnitude should occur once in every 13,000 years! However, an examination of S&P returns over the 37-year period cited shows drops of 5 standard deviations or greater on 15 occasions. Assuming a normal distribution would consistently underestimate the occurrence of tail events.

We conducted a subsequent analysis focusing on the daily returns of SPY, a popular exchange-traded fund (ETF). This ETF tracks 503 component instruments. Using returns from July 01, 2016 through June 31, 2017, we tested each component instrument’s return vector for normality using the Chi-Square Test, the Kurtosis estimate, and a visual inspection of the Q-Q plot. Brief explanations of these methods are provided below.

Chi-Square Test

This is a goodness-of-fit test that assumes a specific data distribution (Null hypothesis) and then tests that assumption. The test evaluates the deviations of the model predictions (Normal distribution, in this instance) from empirical values. If the resulting computed test statistic is large, then the observed and expected values are not close and the model is deemed a poor fit to the data. Thus, the Null hypothesis assumption of a specific distribution is rejected.

Kurtosis

The kurtosis of any univariate standard-Normal distribution is 3. Any deviations from this value imply that the data distribution is correspondingly non-Normal. An example is illustrated in Figures 2, 3, and 4, below.

Q-Q Plot

Quantile-quantile (QQ) plots are graphs on which quantiles from two distributions are plotted relative to each other. If the distributions correspond, then the plot appears linear. This is a visual assessment rather than a quantitative estimation. A sample set of results is shown in Figures 2, 3, and 4, below.

Figure Two: Year’s Returns for Exxon

Figure 2. The left panel shows the histogram of a year’s returns for Exxon (XOM). The null hypothesis was rejected with the conclusion that the data is not normally distributed. The kurtosis was 6 which implies a deviation from normality. The Q-Q plot in the right panel reinforces these conclusions.

Figure Three: Year’s Returns for Boeing

Figure 3. The left panel shows the histogram of a year’s returns for Boeing (BA). The data is not normally distributed and shows a significant skewness also. The kurtosis was 12.83 and implies a significant deviation from normality. The Q-Q plot in the right panel confirms this.

For the sake of comparison, we also show returns that exhibit normality in the next figure.

Figure Four: Year’s Returns for Xerox

The left panel shows the histogram of a year’s returns for Xerox (XRX). The data is normally distributed, which is apparent from a visual inspection of both panels. The kurtosis was 3.23 which is very close to the value for a theoretical normal distribution.

Machine learning literature has several suggestions for addressing this problem, including Kernel Density Estimation and Mixture Density Networks. If the data exhibits multi-modal behavior, learning a multi-modal mixture model is a possible approach.

Stationarity Assumption

In addition to normality, we also make untested assumptions regarding stationarity. This critical assumption is implicit when computing covariances and correlations. We also tend to overlook insufficient sample sizes. As observed earlier, the SPY dataset we had at our disposal consisted of 503 instruments, with around 250 returns per instrument. The number of observations is much lower than the dimensionality of the data. This will produce a covariance matrix which is not full-rank and, consequently, its inverse will not exist. Singular covariance matrices are highly problematic when computing the risk-return efficiency loci in the analysis of portfolios. We tested the returns of all instruments for stationarity using the Augmented Dickey Fuller (ADF) test. Several return vectors were non-stationary. Non-stationarity and sample size issues can’t be wished away because the financial markets are fluid with new firms coming into existence and existing firms disappearing due bankruptcies or acquisitions. Consequently, limited financial histories will be encountered and must be dealt with.

This is a problem where machine learning can be profitably employed. Shrinkage methods, Latent factor models, Empirical Bayes estimators and Random matrix theory based models are widely published techniques that are applicable here.

Portfolio Performance Analysis

Once issues surrounding untested assumptions have addressed, we can focus on portfolio performance analysis–a subject with a vast collection of books and papers devoted to it. We limit our attention here to one aspect of portfolio performance analysis – an inquiry into the clustering behavior of stocks in a portfolio.

Books on portfolio theory devote substantial space to the discussion of asset diversification to achieve an optimum balance of risk and return. To properly diversify assets, we need to know if resources have been over-allocated to a specific sector and, consequently, under-allocated to others. Cluster analysis can help to answer this. A pertinent question is how to best measure the difference or similarity between stocks. One way would be to estimate correlations between stocks. This approach has its own weaknesses, some of which have been discussed in earlier sections. Even if we had a statistically significant set of observations, we are faced with the problem of changing correlations during the course of a year due to structural and regime shifts caused by intermittent periods of stress. Even in the absence of stress, correlations can break down or change due to factors that are endogenous to individual stocks.

We can estimate similarity and visualize clusters using histogram analysis. However, histograms eliminate temporal information. To overcome this constraint, we used Spectral Clustering, which is a machine learning technique that explores cluster formation without neglecting temporal information.

Figures 5 to 7 display preliminary results from our cluster analysis. Analyses like this will enable portfolio managers to realize clustering patterns and their strengths in their portfolios. They will also help guide decisions on reweighting portfolio components and diversification.

Figures 5-7: Cluster Analyses

Figure 5. Cluster analysis of a limited set of stocks is shown here. The labels indicate the names of the firms. Clusters are illustrated by various colored bullets, and increasing distances indicate decreasing similarities. Within clusters, stronger affinities are indicated by greater connecting line weights.

The following figures display magnified views of individual clusters.

Figure 6. We can see that Procter & Gamble, Kimberly Clark and Colgate Palmolive form a cluster (top left, dark green bullets). Likewise, Bank of America, Wells Fargo and Goldman Sachs form a cluster (top right, light green bullets). This is not surprising as these two clusters represent two sectors: consumer products and banking. Line weights are correlated to affinities within sectors.

Figure 7. The cluster on the left displays stocks in the technology sector, while the clusters on the right represent firms in the defense industry (top) and the energy sector (bottom).

In this post, we raised questions about standard assumptions that are made when analyzing portfolios. We also suggested possible solutions from machine learning literature. We subsequently analyzed one year’s worth of returns of SPY to identify clusters and their strengths and discussed the value of such an analysis to portfolio managers in evaluating risk and reweighting or diversifying their portfolios.

Mitigating EUC Risk Using Model Validation Principles

The challenge associated with simply gauging the risk associated with “end user computing” applications (EUCs)— let alone managing it—is both alarming and overwhelming. Scanning tools designed to detect EUCs can routinely turn up tens of thousands of potential files, even at not especially large financial institutions. Despite the risks inherent in using EUCs for mission-critical calculations, EUCs are prevalent in nearly any institution due to their ease of use and wide-ranging functionality.

This reality has spurred a growing number of operational risk managers to action. And even though EUCs, by definition, do not rise to the level of models, many of these managers are turning to their model risk departments for assistance. This is sensible in many cases because the skills associated with effectively validating a model translate well to reviewing an EUC for reasonableness and accuracy.  Certain model risk management tools can be tailored and scaled to manage burgeoning EUC inventories without breaking the bank.

Identifying an EUC

One risk of reviewing EUCs using personnel accustomed to validating models is the tendency of model validators to do more than is necessary. Subjecting an EUC to a full battery of effective challenges, conceptual soundness assessments, benchmarking, back-testing, and sensitivity analyses is not an efficient use of resources, nor is it typically necessary. To avoid this level of overkill, reviewers ought to be able to quickly recognize when they are looking an EUC and when they are looking at something else.

Sometimes the simplest definitions work best: an EUC is a spreadsheet.

While neither precise, comprehensive, nor 100 percent accurate, that definition is a reasonable approximation. Not every EUC is a spreadsheet (some are Access databases) but the overwhelming majority of EUCs we see are Excel files. And not every Excel file is an EUC—conference room schedules and other files in Excel that do not do any serious calculating do not pose EUC risk. Some Excel spreadsheets are models, of course, and if an EUC review discovers quantitative estimates in a spreadsheet used to compute forecasts, then analysts should be empowered to flag such applications for review and possible inclusion in the institution’s formal model inventory. Once the dust has settled, however, the final EUC inventory is likely to contain almost exclusively spreadsheets.

Building an EUC Inventory

EUCs are not models, but much of what goes into building a model inventory applies equally well to building an EUC inventory. Because the overwhelming majority of EUCs are Excel files, the search for latent EUCs typically begins with an automated search for files with .xls and .xlsx extensions. Many commercially available tools conduct these sorts of scans. The exercise typically returns an extensive list of files that must be sifted through.

Simple analytical tools, such as Excel’s “Inquire” add-in, are useful for identifying the number and types of unique calculations in a spreadsheet as well as a spreadsheet’s reliance on external data sources. Spreadsheets with no calculations can likely be excluded from further consideration from the EUC inventory. Likewise, spreadsheets with no data connections (i.e., links to or from other spreadsheets) are unlikely to qualify for the EUC inventory because such files do not typically have significant downstream impact. Spreadsheets with many tabs and hundreds of unique calculations are likely to qualify as EUCs (at least—if not as models) regardless of their specific use.

Most spreadsheets fall somewhere between these two extremes. In many cases, questioning the owners/users of identified spreadsheets is necessary to determine its use and help ascertain any potential institutional risks if the spreadsheet does not work as intended. When making inquiries of spreadsheet owners, open-ended questions may not always be as helpful as those designed to elicit a narrow band of responses. Instead of asking, “What is this spreadsheet used for?” A more effective request would be, “What other systems and files is this spreadsheet used to populate?”

Answers to these sorts of questions aid not only in determining whether a spreadsheet qualifies as an EUC but the risk-rating of the EUC as well.

Testing Requirements

For now, regulator interest in seeing that EUCs are adequately monitored and controlled appears to be outpacing any formal guidance on how to go about doing it.

Absent such guidance, many institutions have started approaching EUC testing like a limited-scope model validation. Effective reviews include a documentation review, a tie-out of input data to authorized, verified sources, an examination of formulas and coding, a form of benchmarking, and an overview of spreadsheet governance and controls.

Documentation Review

Not unlike a model, each EUC should be accompanied by documentation that explains its purpose and how it accomplishes what it intends to do. Documentation should describe the source of input data and what the EUC does with it. Sufficient information should be provided for a reasonably informed reviewer to re-create the EUC based solely on the documentation. If a reviewer must guess the purpose of any calculation, then the EUC’s documentation is likely deficient.

Input Review

The reviewer should be able to match input data in the EUC back to an authoritative source. This review can be performed manually; however, any automated lookups used to pull data in from other files should be thoroughly reviewed, as well.

Formula and Function Review

Each formula in the EUC should be independently reviewed to verify that it is consistent with its documented purposes. Reviewers do not need to test the functionality of Excel—e.g., they do not need to test arithmetic functions on a calculator—however, formulas and functions should be reviewed for reasonableness.

Benchmarking

A model validation benchmarking exercise generally consists of comparing the subject model’s forecasts with those of a challenger model designed to do the same thing, but perhaps in a different way. Benchmarking an EUC, in contrast, typically involves constructing an independent spreadsheet based on the EUC documentation and making sure it returns the same answers as the EUC.

Governance and Controls

An EUC should ideally be subjected to the same controls requirements as a model. Procedures designed to ensure process checks, access and change control management, output reconciliation, and tolerance levels should be adequately documented.

The extent to which these tools should be applied depends largely on how much risk an EUC poses. Properly classifying EUCs as high-, medium, or low-risk during the inventory process is critical to determining how much effort to invest in the review.

Other model validation elements, such as back-testing, stress testing, and sensitivity analysis, are typically not applicable to an EUC review. Because EUCs are not predictive by definition, these sorts of analyses are not likely to bring much value to an EUC review .

Striking an appropriate balance — leveraging effective model risk management principles without doing more than needs to be done — is the key to ensuring that EUCs are adequately accounted for, well controlled, and functioning properly without incurring unnecessary costs.

The Non-Agency MBS Market: Re-Assessing Securitization Market Conditions

Since the financial crisis began in 2007, the “Non-Agency” MBS market, i.e., securities neither issued nor guaranteed by Fannie Mae, Freddie Mac, or Ginnie Mae, has been sporadic and has not rebounded from pre-crisis levels. In recent months, however, activity by large financial institutions, such as AIG and Wells Fargo, has indicated a return to the issuance of Non-Agency MBS. What is contributing to the current state of the securitization market for high-quality mortgage loans? Does the recent, limited-scale return to issuance by these institutions signal an increase in private securitization activity in this sector of the securitization market? If so, what is sparking this renewed interest?

 

The MBS Securitization Market

Three entities – Ginnie Mae, Fannie Mae, and Freddie Mac – have been the dominant engine behind mortgage-backed securities (MBS) issuance since 2007. These entities, two of which remain in federal government conservatorship and the third a federal government corporation, have maintained the flow of capital from investors into guaranteed MBS and ensured that mortgage originators have adequate funds to originate certain types of single-family mortgage loans.

Virtually all mortgage loans backed by federal government insurance or guaranty programs, such as those offered by the Federal Housing Administration and the Department of Veterans Affairs, are issued in Ginnie Mae pools. Mortgage loans that are not eligible for these programs are referred to as “Conventional” mortgage loans. In the current market environment, most Conventional mortgage loans are sold to Fannie Mae and Freddie Mac (i.e. “Conforming” loans) and are securitized in Agency-guaranteed pass-through securities.

 

The Non-Agency MBS Market

Not all Conventional mortgage loans are eligible for purchase by Fannie Mae or Freddie Mac, however, due to collateral restrictions (i.e., their loan balances are too high or they do not meet certain underwriting requirements). These are referred to as “Non-Conforming” loans and, for most of the past decade, have been held in portfolio at large financial institutions, rather than placed in private, Non-Agency MBS. The Non-Agency MBS market is further divided into sectors for “Qualified Mortgage” (QM) loans, non-QM loans, re-performing loans and nonperforming loans. This post deals with the securitization of QM loans through Non-Agency MBS programs.

Since the crisis, Non-Agency MBS issuance has been the exclusive province of JP Morgan and Redwood Trust, both of which continue to issue a relatively small number of deals each year. The recent entry of AIG into the Non-Agency MBS market and, combined with Wells Fargo’s announcement that it intends to begin issuing as well, makes this a good time to discuss reasons why these institutions with other funding sources available to them are now moving back to this securitization market sector.

 

Considerations for Issuing QM Loans

Three potential considerations may lead financial institutions to investigate issuing QM Loans through Non-Agency MBS transactions:

  • “All-In” Economics
  • Portfolio Concentration or Limitations
  • Regulatory Pressures

Investigate “All-In” Economics

Over the long-term, mortgage originators gravitate to funding sources that provide the lowest cost to borrowers and profitability for their firms.  To improve the “all-in” economics of a Non-Agency MBS transaction, investment banks work closely with issuers to broaden the investor base for each level of the securitization capital structure.  Partly due to the success of the Fannie Mae and Freddie Mac Credit Risk Transfer transactions, there appears to be significant interest in higher-yielding mortgage-related securities at the lower-rated (i.e. higher risk) end of the securitization capital structure. This need for higher yielding assets has also increased demand for lower-rated securities in the Non-Agency MBS sector.

However, demand from investors at the higher-rated end of the securitization capital structure (i.e. ‘AAA’ and ‘AA’ securities) has not resulted in “all-in” economics for a Non-Agency MBS transaction that surpass the economics of balance sheet financing provided by portfolios funded with low deposit rates or low debt costs. If deposit rates and debt costs remain at historically low levels, the portfolio funding alternative will remain attractive. Notwithstanding the low interest rate environment, some institutions may develop operational capabilities for Non-Agency MBS programs as a risk mitigation process for future periods where balance sheet financing alternatives may not be as beneficial.

 

Portfolio Concentration or Limitations

Due to the lack of robust investor demand and unfavorable economics in Non-Agency MBS, many banks have increased their portfolio exposure to both fixed-rate and intermediate-adjustable-rate QM loans. The ability to hold these mortgage loans in portfolio has provided attractive pricing to a key customer demographic and earned an attractive net interest rate margin during the historical low-rate environment. While bank portfolios have provided an attractive funding source for Non-Agency QM loans, some financial institutions may attempt to develop diversified funding sources in response to regulatory pressure or self-imposed portfolio concentration limits. Selling existing mortgage portfolio assets into the Non-Agency MBS securitization market is one way in which financial institutions might choose to reduce concentrated mortgage risk exposure.

 

Regulatory Pressure

Some financial institutions may be under pressure from their regulators to demonstrate their ability to sell assets out of their mortgage portfolio as a contingency plan. The Non-Agency MBS market is one way of complying with these sorts of regulatory requests. Developing a contingency ability to tap Non-Agency MBS markets develops operational capabilities under less critical circumstances, while assessing the time needed by the institution to liquidate such assets through securitization. This early establishment of securitization functionalities is a prudent activity for those institutions who foresee the possibility of securitization as a future funding option.

While the Non-Agency MBS market has been dormant for most of the past decade, some financial institutions that have relied upon portfolio funding now appear to be testing the current viability of the Non-Agency MBS market. Other mortgage originators would be wise to take notice of these events, monitor activity in these markets, and assess the viability of this alternative funding source for their on-Conforming QM Loans. With the continued issuance by JP Morgan and Redwood Trust and new entrants such as AIG and Wells Fargo, -Non-Agency MBS market activity should be monitored by other mortgage originators to determine whether securitization has the potential to provide an alternative funding source for future lending activity.

In our next article on the Non-Agency MBS market, we will review the changes in due diligence practices, loan-level data disclosures, the representation and warranty framework, and the ratings process made by securitization market participants and the impact of these changes on the Non-Agency MBS market segment.

Advantages and Disadvantages of Open Source Data Modeling Tools

Using open source data modeling tools has been a topic of debate as large organizations, including government agencies and financial institutions, are under increasing pressure to keep up with technological innovation to maintain competitiveness. Organizations must be flexible in development and identify cost-efficient gains to reach their organizational goals, and using the right tools is crucial. Organizations must often choose between open source software, i.e., software whose source code can be modified by anyone, and closed software, i.e., proprietary software with no permissions to alter or distribute the underlying code.

Mature institutions often have employees, systems, and proprietary models entrenched in closed source platforms. For example, SAS Analytics is a popular provider of proprietary data analysis and statistical software for enterprise data operations among financial institutions. But several core computations SAS performs can also be carried out using open source data modeling tools, such as Python and R. The data wrangling and statistical calculations are often fungible and, given the proper resources, will yield the same result across platforms.

Open source is not always a viable replacement for proprietary software, however. Factors such as cost, security, control, and flexibility must all be taken into consideration. The challenge for institutions is picking the right mix of platforms to streamline software development.  This involves weighing benefits and drawbacks.

Advantages of Open Source Programs

The Cost of Open Source Software

The low cost of open source software is an obvious advantage. Compared to the upfront cost of purchasing a proprietary software license, using open source programs seems like a no-brainer. Open source programs can be distributed freely (with some possible restrictions to copyrighted work), resulting in virtually no direct costs. However, indirect costs can be difficult to quantify. Downloading open source programs and installing the necessary packages is easy and adopting this process can expedite development and lower costs. On the other hand, a proprietary software license may bundle setup and maintenance fees for the operational capacity of daily use, the support needed to solve unexpected issues, and a guarantee of full implementation of the promised capabilities. Enterprise applications, while accompanied by a high price tag, provide ongoing and in-depth support of their products. The comparable cost of managing and servicing open source programs that often have no dedicated support is difficult to determine.

Open Source Talent Considerations

Another advantage of open source is that it attracts talent who are drawn to the idea of sharable and communitive code. Students and developers outside of large institutions are more likely to have experience with open source applications since access is widespread and easily available. Open source developers are free to experiment and innovate, gain experience, and create value outside of the conventional industry focus. This flexibility naturally leads to more broadly skilled inter-disciplinarians. The chart below from Indeed’s Job Trend Analytics tool reflects strong growth in open source talent, especially Python developers.

From an organizational perspective, the pool of potential applicants with relevant programming experience widens significantly compared to the limited pool of developers with closed source experience. For example, one may be hard-pressed to find a new applicant with development experience in SAS since comparatively few have had the ability to work with the application. Key-person dependencies become increasingly problematic as the talent or knowledge of the proprietary software erodes down to a shrinking handful of developers.

Job Seekers Interests via Indeed

*Indeed searches millions of jobs from thousands of job sites. The jobseeker interest graph shows the percentage of jobseekers who have searched for SAS, R, and python jobs.

*Indeed searches millions of jobs from thousands of job sites. The jobseeker interest graph shows the percentage of jobseekers who have searched for SAS, R, and python jobs.

Support and Collaboration

The collaborative nature of open source facilitates learning and adapting to new programming languages. While open source programs are usually not accompanied by the extensive documentation and user guides typical of proprietary software, the constant peer review from the contributions of other developers can be more valuable than a user guide. In this regard, adopters of open source may have the talent to learn, experiment with, and become knowledgeable in the software without formal training.

Still, the lack of support can pose a challenge. In some cases, the documentation accompanying open source packages and the paucity of usage examples in forums do not offer a full picture. For example, RiskSpan built a model in R that was driven by the available packages for data infrastructure – a precursor to performing statistical analysis – and their functionality. R does not have an active support solutions line and the probability of receiving a response from the author of the package is highly unlikely. This required RiskSpan to thoroughly vet packages.

Flexibility and Innovation

Another attractive feature of open source is its inherent flexibility. Python allows users to use different integrated development environments (IDEs) that have multiple different characteristics or functions, as compared to SAS Analytics, which only provides SAS EG or Base SAS. R makes possible web-based interfaces for server-based deployments. These functionalities grant more access to users at a lower cost. Thus, there can be more firm-wide development and participation in development. The ability to change the underlying structure of open source makes it possible to mold it per the organization’s goals and improve efficiency.

Another advantage of open source is the sheer number of developers trying to improve the software by creating many functionalities not found in their closed source equivalent. For example, R and Python can usually perform many functions like those available in SAS, but also have many capabilities not found in SAS: downloading specific packages for industry specific tasks, scraping the internet for data, or web development (Python). These specialized packages are built by programmers seeking to address the inefficiencies of common problems. A proprietary software vendor does not have the expertise nor the incentive to build equivalent specialized packages since their product aims to be broad enough to suit uses across multiple industries.

RiskSpan uses open source data modeling tools and operating systems for data management, modeling, and enterprise applications. R and Python have proven to be particularly cost effective in modeling. R provides several packages that serve specialized techniques. These include an archive of packages devoted to estimating the statistical relationship among variables using an array of techniques, which cuts down on development time. The ease of searching for these packages, downloading them, and researching their use incurs nearly no cost.

Open source makes it possible for RiskSpan to expand on the tools available in the financial services space. For example, a leading cash flow analytics software firm that offers several proprietary solutions in modeling structured finance transactions lacks the full functionality RiskSpan was seeking.  Seeking to reduce licensing fees and gain flexibility in structuring deals, RiskSpan developed deal cashflow programs in Python for STACR, CAS, CIRT, and other consumer lending deals. The flexibility of Python allowed us to choose our own formatted cashflows and build different functionalities into the software. Python, unlike closed source applications, allowed us to focus on innovating ways to interact with the cash flow waterfall.

Disadvantages of Open Source Programs

Deploying open source solutions also carries intrinsic challenges. While users may have a conceptual understanding of the task at hand, knowing which tools yield correct results, whether derived from open or closed source, is another dimension to consider. Different parameters may be set as default, new limitations may arise during development, or code structures may be entirely different. Different challenges may arise from translating a closed source program to an open source platform. Introducing open source requires new controls, requirements, and development methods.

Redundant code is an issue that might arise if a firm does not strategically use open source. Across different departments, functionally equivalent tools may be derived from distinct packages or code libraries. There are several packages offering the ability to run a linear regression, for example. However, there may be nuanced differences in the initial setup or syntax of the function that can propagate problems down the line. In addition to the redundant code, users must be wary of “forking” where the development community splits on an open source application. For example, R develops multiple packages performing the same task/calculations, sometimes derived from the same code base, but users must be cognizant that the package is not abandoned by developers.

Users must also take care to track the changes and evolution of open source programs. The core calculations of commonly used functions or those specific to regular tasks can change. Maintaining a working understanding of these functions in the face of continual modification is crucial to ensure consistent output. Open source documentation is frequently lacking. In financial services, this can be problematic when seeking to demonstrate a clear audit trail for regulators. Tracking that the right function is being sourced from a specific package or repository of authored functions, as opposed to another function, which may have an identical name, sets up blocks on unfettered usage of these functions within code. Proprietary software, on the other hand, provides a static set of tools, which allows analysts to more easily determine how legacy code has worked over time.

Using Open Source Data Modeling Tools

Deciding on whether to go with open source programs directly impacts financial services firms as they compete to deliver applications to the market. Open source data modeling tools are attractive because of their natural tendency to spur innovation, ingrain adaptability, and propagate flexibility throughout a firm. But proprietary software solutions are also attractive because they provide the support and hard-line uses that may neatly fit within an organization’s goals. The considerations offered here should be weighed appropriately when deciding between open source and proprietary data modeling tools.

Questions to consider before switching platforms include:

  • How does one quantify the management and service costs for using open source programs? Who would work on servicing it, and, once all-in expenses are considered, is it still more cost-effective than a vendor solution?
  • When might it be prudent to move away from proprietary software? In a scenario where moving to a newer open source technology appears to yield significant efficiency gains, when would it make sense to end terms with a vendor?
  • Does the institution have the resources to institute new controls, requirements, and development methods when introducing open source applications?
  • Does the open source application or function have the necessary documentation required for regulatory and audit purposes?

Open source is certainly on the rise as more professionals enter the space with the necessary technical skills and a new perspective on the goals financial institutions want to pursue. As competitive pressures mount, financial institutions are faced with a difficult yet critical decision of whether open source is appropriate for them. Open source may not be a viable solution for everyone—the considerations discussed above may block the adoption of open source for some organizations. However, often the pros outweigh the cons, and there are strategic precautions that can be taken to mitigate any potential risks.

References

 https://www.redhat.com/en/open-source/open-source-way

http://www.stackoverflow.blog/code-for-a-living/how-i-open-sourced-my-way-to-my-dream-job-mohamed-said

https://www.redhat.com/f/pdf/whitepapers/WHITEpapr2.pdf

http://www.forbes.com/sites/benkepes/2013/10/02/open-source-is-good-and-all-but-proprietary-is-still-winning/#7d4d544059e9

https://www.indeed.com/jobtrends/q-SAS-q-R-q-python.html

Open Source Software for Mortgage Data Analysis

While open source has been around for decades, using open source software for mortgage data analysis is a recent trend. Financial institutions have traditionally been slow to adopt the latest data and technology innovations due to the strict regulatory and risk-averse nature of the industry, and open source has been no exception. As open source becomes more mainstream, however, many of our clients have come to us with questions regarding its viability within the mortgage industry.

The short answer is simple: open source has a lot of potential for the financial services and mortgage industries, particularly for data modeling and data analysis. Within our own organization, we frequently use open source data modeling tools for our proprietary models as well as models built for clients. While a degree of risk is inherent, prudent steps can be taken to mitigate them and profit from the many worthwhile benefits of open source.

Open source has a lot of potential for the mortgage industry, particularly for data modeling & analysis @RiskSpan (Click to Tweet)

To address the common concerns that arise with open source, we’ll be publishing a series of blog posts aimed at alleviating these concerns and providing guidelines for utilizing open source software for data analysis within your organization. Some of the questions we’ll address include:

  • Can open source programming languages be applied to mortgage data modeling and data analysis?
  • What risks does open source expose me to and what can I do to mitigate them?
  • What are the pitfalls of open source and do the benefits outweigh them?
  • How does using open source software for mortgage data analysis affect the control and governance of my models?
  • What factors do I need to consider when deciding whether to use open source at my institution?

Throughout the series, we’ll also include examples of how RiskSpan has used open source software for mortgage data analysis, why we chose to use it, and what factors were considered. Before we dive in on the considerations for open source, we thought it would be helpful to offer an introduction to open source and provide some context around its birth and development within the financial industry.

What Is Open Source Software?

Software has conventionally been considered open source when the original code is made publicly available so that anyone can edit, enhance, or modify it freely. This original concept has recently been expanded to incorporate a larger movement built on values of collaboration, transparency, and community.

Open Source Software Vs Proprietary Software

Proprietary software refers to applications for which the source code is only accessible to those who created it. Thus, only the original author(s) has control over any updates or modifications. Outside players are barred from even viewing the code to protect the owners from copying and theft. To use proprietary software, users agree to a licensing agreement and typically pay a fee. The agreement legally binds the user to the owners’ terms and prevents the user from any actions the owners have not expressly permitted.

Open source software, on the other hand, gives any user free rein to view, copy, or modify it. The idea is to foster a community built on collaboration, allowing users to learn from each other and build on each other’s work. Like with proprietary software, open source users must still agree to a licensing agreement, but the terms are very differ significantly from those of a proprietary license.1

History of Open Source Software

The idea of open source software first developed in the 1950s, when much of software development was done by computer scientists in higher education. In line with the value of sharing knowledge among academics, source code was openly accessible. By the 1960s, however, as the cost of software development increased, hardware companies were charging additional fees for software that used to be bundled with their products.

Change came again in the 1980s. At this point, it was clear that technology and software were important factors of the growing business economy. Technology leaders were frustrated with the increasing costs of software. In 1984, Richard Stallman launched the GNU Project with the purpose of creating a complete computer operating system with no limitations on the use of its source code. In 1991, the operating system now referred to as Linux was released.

The final tipping point came in 1997, when Eric Raymond published his book, The Cathedral and the Bazaar, in which he articulated the underlying principles behind open source software. His book was a driving factor in Netscape’s decision to release its source code to the public, inspired by the idea that allowing more people to find and fix bugs will improve the system for everyone. Following Netscape’s release, the term “open source software” was introduced in 1998.

In the data-driven economy of the past two decades, open source has played an ever-increasing role. The field of software development has evolved to embrace the values of open source. Open source has made it not only possible but easy for anyone to access and manipulate source code, improving our ability to create and share valuable software.2

Adoption of Open Source Software in Business

The growing relevance of open source software has also changed the way large organizations approach their software solutions. While open source software was at one point rare in an enterprise’s system, it’s now the norm. A survey conducted by Black Duck Software revealed that fewer than 3% of companies don’t rely on open source at all. Even the most conservative organizations are hopping on board the open source trend.3
Even the most conservative organizations are hopping on board the open source trend.

In a blog post from June 2016, TechCrunch writes:

“Open software has already rooted itself deep within today’s Fortune 500, with many contributing back to the projects they adopt. We’re not just talking stalwarts like Google and Facebook; big companies like Walmart, GE, Merck, Goldman Sachs — even the federal government — are fleeing the safety of established tech vendors for the promises of greater control and capability with open software. These are real customers with real budgets demanding a new model of software.”4

The expected benefits of open source software are alluring all types of institutions, from small businesses, to technology giants, to governments. This shift away from proprietary software in favor of open source is streamlining business operations. As more companies make the switch, those who don’t will fall behind the times and likely be at a serious competitive disadvantage.

Open Source Software for Mortgage Data Analysis

Open source software is slowly finding its way into the financial services industry as well. We’ve observed that smaller entities that don’t have the budgets to buy expensive proprietary software have been turning to open source as a viable substitute. Smaller companies are either building software in house or turning to companies like RiskSpan to achieve a cost-effective solution. On the other hand, bigger companies with the resources to spare are also dabbling in open source. These companies have the technical expertise in house and give their skilled workers the freedom to experiment with open source software.

Within our own work, we see tremendous potential for open source software for mortgage data analysis. Open source data modeling tools like Python, R, and Julia are useful for analyzing mortgage loan and securitization data and identifying historical trends. We’ve used R to build models for our clients and we’re not the only ones: several of our clients are now building their DFAST challenger models using R.

Open source has grown enough in the past few years that more and more financial institutions will make the switch. While the risks associated with open source software will continue to give some organizations pause, the benefits of open source will soon outweigh those concerns. It seems open source is a trend that is here to stay, and luckily, it is a trend ripe with opportunity.

[1] https://opensource.com/resources/what-open-source

[2] https://www.longsight.com/learning-center/history-open-source

[3] https://techcrunch.com/2016/06/19/the-next-wave-in-software-is-open-adoption-software/

[4] https://techcrunch.com/2016/06/19/the-next-wave-in-software-is-open-adoption-software/

Balancing Internal and External Model Validation Resources

The question of “build versus buy” is every bit as applicable and challenging to model validation departments as it is to other areas of a financial institution. With no “one-size-fits-all” solution, banks are frequently faced with a balancing act between the use of internal and external model validation resources. This article is a guide for deciding between staffing a fully independent internal model validation department, outsourcing the entire operation, or a combination of the two.

Striking the appropriate balance is a function of at least five factors:

  1. control and independence
  2. hiring constraints
  3. cost
  4. financial risk
  5. external (regulatory, market, and other) considerations

Control and Independence

Internal validations bring a measure of control to the operation. Institutions understand the specific skill sets of their internal validation team beyond their resumes and can select the proper team for the needs of each model. Control also extends to the final report, its contents, and how findings are described and rated.

Further, the outcome and quality of internal validations may be more reliable. Because a bank must present and defend validation work to its regulators, low-quality work submitted by an external validator may need to be redone by yet another external validator, often on short notice, in order to bring the initial external model validation up to spec.

Elements of control, however, must sometimes be sacrificed in order to achieve independence. Institutions must be able to prove that the validator’s interests are independent from the model validation outcomes. While larger banks frequently have large, freestanding internal model validation departments whose organizational independence is clear and distinct, quantitative experts at smaller institutions must often wear multiple hats by necessity.

Ultimately the question of balancing control and independence can only be suitably addressed by determining whether internal personnel qualified to perform model validations are capable of operating without any stake in the outcome (and persuading examiners that this is, in fact, the case).

Hiring Constraints

Practically speaking, hiring constraints represent a major consideration. Hiring limitations may result from budgetary or other less obvious factors. Organizational limits aside, it is not always possible to hire employees with a needed skill set at a workable salary range at the time when they are needed. For smaller banks with limited bandwidth or larger banks that need to further expand, external model validation resources may be sought out of sheer necessity.

Cost

Cost is an important factor that can be tricky to quantify. Model validators tend to be highly specialized; many typically work on one type of model, for example, Basel models. If your bank is large enough and has enough Basel models to keep a Basel model validator busy with internal model validations all year, then it may be cost effective to have a Basel model validator on staff. But if your Basel model validator is only busy for six months of the year, then a full-time Basel validator is only efficient if you have other projects that are suited to that validator’s experience and cost. To complicate things further, an employee’s cost is typically housed in one department, making it difficult from a budget perspective to balance an employee’s time and cost across departments.

If we were building a cost model to determine how many internal validators we should hire, the input variables would include:

  1. the number of models in our inventory
  2. the skills required to validate each model
  3. the risk classification of each model (i.e., how often validations must be completed)
  4. the average fully loaded salary expense for a model validator with those specific skills

Only by comparing the cost of external validations to the year-round costs associated with hiring personnel with the specialized knowledge required to validate a given type of model (e.g., credit models, market risk models, operational risk models, ALM models, Basel models, and BSA/AML models) can a bank arrive at a true apples-to-apples comparison.

Financial Risk

While cost is the upfront expense of internal or external model validations, financial risk accounts for the probability of unforeseen circumstances. Assume that your bank is staffed with internal validators and your team of internal validators can handle the schedule of model validations (validation projects are equally spaced throughout the year). However, operations may need to deploy a new version of a model or a new model on a schedule that requires a validation at a previously unscheduled time with no flexibility. In this case, your bank may need to perform an external validation in addition to managing and paying a fully-staffed team of internal validators.

A cost model for determining whether to hire additional internal validators should include a factor for the probability that models will need to be validated off-schedule, resulting in unforeseen external validation costs. On the other hand, a cost model might also consider the probability that an external validator’s product will be inferior and incur costs associated with required remediation.

External Risks

External risks are typically financial risks caused by regulatory, market, and other factors outside an institution’s direct control. The risk of a changing regulatory environment under a new presidential administration is always real and uncertainty clearly abounds as market participants (and others) attempt to predict President Trump’s priorities. Changes may include exemptions for regional banks from certain Dodd-Frank requirements; the administration has clearly signaled its intent to loosen regulations generally. Even though model validation will always be a best practice, these possibilities may influence a bank’s decision to staff an internal model validation team.

Recent regulatory trends can also influence validator hiring decisions. For example, our work with various banks over the past 12-18 months has revealed that regulators are trending toward requiring larger sample sizes for benchmarking and back-testing. Given the significant effort already associated with these activities, larger sample sizes could ultimately lower the number of model validations internal resources can complete per year. Funding external validations may become more expensive, as well.

Another industry trend is the growing acceptance of limited-scope validations. If only minimal model changes have occurred since a prior validation, the scope of a scheduled validation may be limited to the impact of these changes. If remediation activities were required by a prior validation, the scope may be limited to confirming that these changes were effectively implemented. This seemingly common-sense approach to model validations by regulators is a welcome trend and could reduce the number of internal and external validations required.

Joint Validations

In addition to reduced-scope validations, some of our clients have sought to reduce costs by combining internal and external resources. This enables institutions to limit hiring to validators without model-specific or highly quantitative skills. Such internal validators can typically validate a large number of lower-risk, less technical models independently.

For higher-risk, more technical models, such as ALM models, the internal validator may review the controls and documentation sufficiently, leaving the more technical portions of the validation—conceptual soundness, process verification, benchmarking, and back-testing, for example—to an external validator with specific expertise. In these cases, reports are produced jointly with internal and external validators each contributing the sections pertaining to procedures that they performed.

The resulting report often has the dual benefit of being more economical than a report generated externally and more defensible than one that relies solely on internal resources who may lack the specific domain knowledge necessary.

Conclusion

Model risk managers have limited time, resources, and budgets and face unending pressure from management and regulators. Striking an efficient resource-balancing strategy is critically important to consistently producing high-quality model validation reports on schedule and within budgets. The question of using internal vs. external model validation resources is not an either/or proposition. In considering it, we recommend that model risk management (MRM) professionals

  • consider the points above and initiate risk tolerance and budget conversations within the MRM framework.
  • reach out to vendors who have the skills to assist with your high-risk models, even if there is not an immediate need. Some institutions like to try out a model validation provider on a few low- or moderate-risk models to get a sense of their capabilities.
  • consider internal staffing to meet basic model validation needs and external vendors (either for full validations or outsourced staff) to fill gaps in expertise.

Credit Risk Transfer: Front End Execution – Why Does It Matter?

This article was originally published on the GoRion blog.

Last month I described an overview of the activities of Credit Risk Transfer (CRT) as outlined from the Federal Finance Housing Agency (FHFA) guidance to Fannie Mae and Freddie Mac (the GSEs). This three-year-old program has shown great promise and success in creating a deeper residential credit investor segment and has enabled risk increments to be shifted from the GSEs and taxpayer to the private sector.

The FHFA issued an RFI to solicit feedback from stakeholders on proposals from the GSEs to adopt additional front-end credit risk transfer structures and to consider additional credit risk transfer policy issues. There is firm interest in this new and growing execution for risk transfer by investors who have confidence in the underwriting and servicing of mortgage loans through new and improved GSE standards.

In addition to the back-end industry appetite for CRT, there is also a growing focus to increase risk share at the front-end of the origination transaction. In particular, the mortgage industry and insurers (MIs) are interested in exploring risk sharing more actively on the front-end of the mortgage process. The MIs desire to participate in this new and growing market opportunity would increase their traditional coverage to much deeper levels than the standard 30% coverage.

 

Front-End Credit Risk Transfer

In 2016 FHFA expanded the GSE scorecards to include broadening the types of loans and risk transfer which included expanding to the front-end CRT. In addition to many prescriptive outlines on CRT, they also included wording such as “…Work with FHFA to conduct an analysis and assessment of front-end credit risk transfer transactions, including work to support a forthcoming FHFA Request for Input. Work with FHFA to engage key stakeholders and solicit their feedback. After conducting the necessary analysis and assessment, work with FHFA to take appropriate steps to continue front-end credit risk transfer transactions.”

Two additional ways to work with risk sharing on the front-end are using 1) recourse transactions and 2) deeper mortgage insurance.

 

Recourse Transaction

Recourse as a form of credit enhancement is not a new concept. In years past, some institutions would sell loans with recourse to the GSEs but it was usually determined to be capital intensive and not an efficient way of selling loans to the secondary markets. However, some of the non-depositories have found recourse to be an attractive way to sell loans to the GSEs.

To date from 2013 through December 2015, the GSE’s have executed 12 deals with recourse on $12.6 billion in UPB. The pricing and structures are very different and the transactions are not transparent. While this can be attractive to both parties if structured adequately, the transactions are not as scalable and each deal requires significant review and assessment. Arguments against recourse note this diminishes opportunities for the small to medium sized player who would like to participate in this new form of reduced g-fee structure and front-end CRT transaction.

Penny Mac shared their perspective on this activity at a recent CRT conference. They use the recourse structure with Fannie Mae and it leverages their capital structure and allows flexibility. Importantly, Penny Mac reminds us that both parties’ interests are aligned as there is skin in the game for quality originations.

 

Deeper Mortgage Insurance

The GSE model has a significant amount of counter-party risk with MIs through their standard business offerings. Through their charter, they require credit enhancements on loans of 80% or higher Loan to Value (LTV). This traditionally plays out to be 30% first loss coverage of such loans. For example, a 95% LTV loan is insured down to 65%. The mortgage insurers are integral to most of the GSE’ higher LTV books of business. Per the RFI, as of December 2015, the MI industry collectively has counter-party exposure of $185.5 billion, covering $724.5 billion of loans. So as a general course of business, this is already a risk they share of higher LTV lending without adding any additional exposure.

Through the crisis, the MIs were unable to pay dollar for dollar initial claims. This has caused hesitation on embracing a more robust model with more counter-party risk than the model of today. It is well documented that the MIs did pay a great deal of claims and buffered the GSEs by taking the first loss on billions of dollars before any losses were incurred by the GSEs. While much of that has been paid back, memories are long and this has generated pause as to how to value the insurance which is different than the back-end transactions. Today the MI industry is in much better shape through capital raises and increased standards directed from the GSEs and state regulators. (Our recent blog post on mortgage insurance haircuts explores this phenomenon in greater detail.)

FHFA instituted the PMIERs which required higher capital for the MI business transacted with the GSEs. The state regulators also increased the regulatory capital for the residential insurance sector and today the industry has strengthened their hand as a partner to the GSEs. In fact, the industry has new entrants who do not have the legacy books of losses which also adds new opportunities for the GSEs to expand the counter-party pools.

The MI companies can be a front-end model and play a more significant role in the risk share business by having deeper MI on the front-end (to 50% coverage) as a way of de-levering the GSE’s and ultimately, the taxpayers. And, like the GSE’s, MI’s may also participate in reinsurance markets to shed risk and balance out their own portfolios. Other market participants may also participate in this type of transaction and we will observe what opportunities avail themselves in the longer term. While nothing is ever black and white, there appear to be benefits to expanding the risk share efforts to the front-end of the business.

 

Benefits

1) Strong execution: Pricing and executing on mortgage risk, at the front of the origination will allow for options in a counter-cyclical volatile market.

2) Transparency: Moving risk metrics and pricing to the front-end will drive more front-end price transparency for mortgage credit risk.

3) Inclusive institutional partnering: Smaller entities may participate in a front-end risk share effort thereby creating opportunities outside of the largest financial institutions.

4) Inclusive borrower process: Front-end CRT may reach more borrowers and provide options as more institutions can take part of this opportunity.

5) Expands options for CRT in pilot phase: By driving the risk share to the front-end, the GSE’s reach their goals in de-risking their credit guarantee while providing a timely trade off of G-fee and MI pricing on the front-end of the transaction.

As part of the RFI response, the trade representing the MIs summarized principal benefits of front-end CRT as follows:

  1. Increased CRT availability and market stability
  2. Reduced first-loss holding risk
  3. Beneficial stakeholder familiarity and equitable access
  4. Increased transparency.

The full letter may be found at usmi.org.

In summary, whether it is recourse to a lending institution or participation in the front-end MI cost structure, pricing this risk at origination will continue to bring forward price discovery and transparency. This means the consumer and lender will be closer to the true credit costs of origination. With experience pricing and executing on CRT, it may become clearer where the differential cost of credit lies. The additional impact of driving more front-end CRT will be scalability and less process on the back-end for the GSEs. By leveraging the front-end model, GSEs will reach more borrowers and utilize a wider array of lending partners through this process.

As of November 8, we experienced a historic election which may take us in new directions. However, credit risk transfer is an option that may be used in the future regardless of GSE status, even if they 1) revert back to the old model with recap and release; 2) re-emerge after housing reform post legislation; or 3) remain in conservatorship and continue to be led by FHFA down this path.

**Footnote: All data was retrieved from the Federal Housing Finance Agency, FHFA, Single Family Credit Risk Transfer request for input, June 2016. More information may be found at FHFA.gov.

This is the second installment in a monthly Credit Risk Transfer (CRT) series on the GoRion blog. CRT is a significant accomplishment in bringing back private capital to the housing sector. This young effort, three years strong, has already shown promising investor appetite while discussions are underway to expand offerings to front-end risk share executions. My goal in this series is to share insights around CRT as it evolves with the private sector.

Validating Model Inputs: How Much Is Enough?

In some respects, the OCC 2011-12/SR 11-7 mandate to verify model inputs could not be any more straightforward: “Process verification … includes verifying that internal and external data inputs continue to be accurate, complete, consistent with model purpose and design, and of the highest quality available.” From a logical perspective, this requirement is unambiguous and non-controversial. After all, the reliability of a model’s outputs cannot be any better than the quality of its inputs. From a functional perspective, however, it raises practical questions around the amount of work that needs to be done in order to consider a particular input “verified.” Take the example of a Housing Price Index (HPI) input assumption. It could be that the modeler obtains the HPI assumption from the bank’s finance department, which purchases it from an analytics firm. What is the model validator’s responsibility? Is it sufficient to verify that the HPI input matches the data of the finance department that supplied it? If not, is it enough to verify that the finance department’s HPI data matches the data provided by its analytics vendor? If not, is it necessary to validate the analytics firm’s model for generating HPI assumptions? It depends. Just as model risk increases with greater model complexity, higher uncertainty about inputs and assumptions, broader use, and larger potential impact, input risk increases with increases in input complexity and uncertainty. The risk of any specific input also rises as model outputs become increasingly sensitive to it.

Validating Model Inputs Best Practices

So how much validation of model inputs is enough? As with the management of other risks, the level of validation or control should be dictated by the magnitude or impact of the risk. Like so much else in model validation, no ‘one size fits all’ approach applies to determining the appropriate level of validation of model inputs and assumptions. In addition to cost/benefit considerations, model validators should consider at least four factors for mitigating the risk of input and assumption errors leading to inaccurate outputs.

  • Complexity of inputs
  • Manual manipulation of inputs from source system prior to input into model
  • Reliability of source system
  • Relative importance of the input to the model’s outputs (i.e., sensitivity)

Consideration 1: Complexity of Inputs

The greater the complexity of the model’s inputs and assumptions, the greater the risk of errors. For example, complex yield curves with multiple data points will be inherently subject to greater risk of inaccuracy than binary inputs such as “yes” and “no.” In general, the more complex an input is, the more scrutiny it requires and the “further back” a validator should look to verify its origin and reasonability.

Consideration 2: Manual Manipulation of Inputs from Source System Prior to Input into Model

Input data often requires modification from the source system to facilitate input into the model. More handling and manual modifications increase the likelihood of error. For example, if a position input is manually copied from Bloomberg and then subjected to a manual process of modification of format to enable uploading to the model, there is a greater likelihood of error than if the position input is extracted automatically via an API. The accuracy of the input should be verified in either case, but the more manual handling and manipulation of data that occurs, the more comprehensive the testing should be. In this example, more comprehensive testing would likely take the form of a larger sample size.

In addition, the controls over the processes to extract, transform, and load data from a source system into the model will impact the risk of error. More mature and effective controls, including automation and reconciliation, will decrease the likelihood of error and therefore likely require a lighter verification procedure.

Consideration 3: Reliability of Source Systems

More mature and stable source systems generally produce more consistently reliable results. Conversely, newer systems and those that have produced erroneous results increase the risk of error. The results of previous validation of inputs, from prior model validations or from third parties, including internal audit and compliance, can be used as an indicator of the reliability of information from source systems and the magnitude of input risk. The greater the number of issues identified, the greater the risk, and the more likely it is that a validator should seek to drill deeper into the fundamental sources of source data.

Consideration 4: Output Sensitivity to Inputs

No matter how reliable an input data’s source system is deemed to be, or the amount of manual manipulation to which an input is subjected, perhaps the most important consideration is the individual input’s power to affect the model’s outputs. Returning to our original example, if a 50 percent change in the HPI assumption has only a negligible impact on the model’s outputs, then a quick verification against the report supplied by the finance department may be sufficient. If, however, the model’s outputs are extremely sensitive to even small shifts in the HPI assumption, then additional testing is likely warranted—perhaps even to include a validation of the analytics vendor’s HPI model (along with all of its inputs).

A Cost-Effective Model Input Validation Strategy

When it comes to verifying model inputs, there is no theoretical limitation to the lengths to which a model validator can go. Model risk managers, who do not have unlimited time or budgets, would benefit from applying practical limits to validation procedures using a risk-based approach to determine the most cost-effective strategies to ensure that models are sufficiently validated. Applying the considerations listed above on a case-by-case basis will help validators appropriately define and scope model input reviews in a manner commensurate with appropriate risk management principles.

Performance Testing: Benchmarking Vs. Back-Testing

When someone asks you what a model validation is, what is the first thing you think of? If you are like most, then you would immediately think of performance metrics— those quantitative indicators that tell you not only if the model is working as intended, but also its performance and accuracy over time and compared to others. Performance testing is the core of any model validation and generally consists of the following components:

  • Benchmarking
  • Back-testing
  • Sensitivity Analysis
  • Stress Testing

Sensitivity analysis and stress testing, while critical to any model validation’s performance testing, will be covered by a future article. This post will focus on the relative virtues of benchmarking versus back-testing—seeking to define what each is, when and how each should be used, and how to make the best use of the results of each.

Benchmarking

Benchmarking is when the validator is providing a comparison of the model being validated to some other model or metric. The type of benchmark utilized will vary, like all model validation performance testing does, with the nature, use, and type of model being validated. Due to the performance information it provides, benchmarking should always be utilized in some form when a suitable benchmark can be found.

Choosing a Benchmark

Choosing what kind of benchmark to use within a model validation can sometimes be a very daunting task. Like all testing within a model validation, the kind of benchmark to use depends on the type of model being tested. Benchmarking takes many forms and may entail comparing the model’s outputs to:

  • The model’s previous version
  • An externally produced model
  • A model built by the validator
  • Other models and methodologies considered by the model developers, but not chosen
  • Industry best practice
  • Thresholds and expectations of the model’s performance

One of the most used benchmarking approaches is to compare a new model’s outputs to those of the version of the model it is replacing. It remains very common throughout the industry for models to be replaced due to a deterioration of performance, change in risk appetite, new regulatory guidance, need to capture new variables, or the availability of new sets of information. In these cases, it is important to not only document but also prove that the new model performs better and does not have the same issues that triggered the old model’s replacement.

Another common benchmarking approach compares the model’s outputs to those of an external “challenger” model (or one built by the validator) which functions with the same objective and data. This approach is likely to return more apt output comparisons than those generated by benchmarking against older versions that are likely to be out of date since the challenger model is developed and updated with the same data as the champion model.

Another benchmark set which could be used for model validation includes other models or methodologies reviewed by the model developers as possibilities for the model being validated but ultimately not used. Model developers as best practice should always list any alternative methodologies, theories, or data which were omitted from the model’s final version. Additionally, model validators should always leverage their experience and understanding of the current best practices throughout the industry, along with any analysis previously completed on similar models. Model validation should then take these alternatives and use them as benchmarks to the model being validated.

Model validators have multiple, distinct ways to incorporate benchmarking into their analysis. The use of the different types of benchmarking discussed here should be based on the type of model, its objective, and the validator’s best judgment. If a model cannot be reasonably benchmarked, then the validator should record why not and discuss the resulting limitations of the validation.

Back-Testing

Back-testing is used to measure model outcomes. Here, instead of measuring performance with a comparison, the validator is specifically measuring whether the model is both working as intended and is accurate. Back-testing can take many forms based on the model’s objective. As with benchmarking, back-testing should be a part of every full-scope model validation to the extent possible.

What Back-Tests to Perform

As a form of outcomes analysis, back-testing provides quantitative metrics which measure the performance of a model’s forecast, the accuracy of its estimates, or its ability to rank-order risk. For instance, if a model produces forecasts for a given variable, back-testing would involve comparing the model’s forecast values against actual outcomes, thus indicating its accuracy.

A related function of model back-testing evaluates the ability of a given model to adequately measure risk. This risk could take any of several forms, from the probability of a given borrower to default to the likelihood of a large loss during a given trading day. To back-test a model’s ability to capture risk exposure, it is important first to collect the right data. In order to back-test a probability of default model, for example, data would need to be collected containing cases where borrowers have actually defaulted in order to test the model’s predictions.

Back-testing models that assign borrowers to various risk levels necessitate some special considerations. Back-testing these and other models that seek to rank-order risk involves looking at the model’s performance history and examining its accuracy through its ability to rank and order the risk. This can involve analyzing both Type 1 (false positive) and Type 2 (false negative) statistical errors against the true positive and true negative rates for a given model.  Common statistical tests used for this type of back-testing analysis include, but are not limited to, a Kolmogorov-Smirnov score (KS), a Brier score, or a Receiver Operating Characteristic (ROC).

Benchmarking vs Backtesting

Back-testing measures a model’s outcome and accuracy against real-world observations, while benchmarking measures those outcomes against those of other models or metrics. Some overlap exists when the benchmarking includes comparing how well different models’ outputs back-test against real-world observations and the chosen benchmark. This overlap sometimes leads people to mistakenly conclude that model validations can rely on just one method. In reality, however, back-testing and benchmarking should ideally be performed together in order to bring their individual benefits to bear in evaluating the model’s overall performance. The decision, optimally, should not be whether to create a benchmark or to perform back-testing. Rather, the decision should be what form both benchmarking and back-testing should take.

While benchmarking and back-testing are complementary exercises that should not be viewed as mutually exclusive, their outcomes sometimes appear to produce conflicting results. What should a model validator do, for example, if the model appears to back-test well against real-world observations but do not benchmark particularly well against similar model outputs? What about a model that returns results similar to those of other benchmark models but does not back-test well? In the first” scenario, the model owner can derive a measure of comfort from the knowledge that the model performs well in hindsight. But the owner also runs the very real risk of being “out on an island” if the model turns out to be wrong. The second scenario affords the comfort of company in the model’s projections. But what if the models are all wrong together?

Scenarios where benchmarking and back-testing do not produce complementary results are not common, but they do happen. In these situations, it becomes incumbent on model validators to determine whether back-testing results should trump benchmarking results (or vice-versa) or if they should simply temper one another. The course to take may be dictated by circumstances. For example, a model validator may conclude that macro-economic indicators are changing to the point that a model which back-tests favorably is not an advisable tool because it is not tuned to the expected forward-looking conditions. This could explain why a model that back-tests favorably remains a benchmarking outlier if the benchmark models are taking into account what the subject model is missing. On the other hand, there are scenarios where it is reasonable to conclude that back-testing results trump benchmarking results. After all, most firms would rather have an accurate model than one that lines up with all the others.

As seen in our discussion here, benchmarking and back-testing can sometimes produce distinct or similar metrics depending on the model being validated. While those differences or similarities can sometimes be significant, both benchmarking and back-testing provide critical complementary information about a model’s overall performance. So when approaching a model validation and determining its scope, your choice should be what form of benchmarking and back-testing needs to be done, rather than whether one needs to be performed versus the other.

Logo

Company

Products

Security & Compliance

Linkedin

SOC2 Type II Certified

Get Started
Log in

Linkedin   

risktech2024