Article Archives

From Main Street to King Abdullah Financial District: Lessons Learned in International Mortgage Finance

In December 2016, I was asked to consult on a start-up real estate refinance company located in the Saudi Arabia. I wasn’t sure I understood what he was saying. As someone who has worked in the U.S. mortgage business since college, the word “refinance” has very strong connotations, but its use seemed wrong in this context. As it turned out in overseas mortgage markets, the phrase real estate refinance refers to “providing funding” or “purchasing mortgage assets.” And that started my quick introduction into the world of international mortgage finance where, “everything is different but in the end it’s all the same.”

By early January 2017 I found myself in Riyadh, Saudi Arabia, working as an adviser to a consulting firm contracted to manage the start-up of the new enterprise. Riyadh in January is nice—cool temperatures and low humidity. In the summer it’s another story. Our client was the Ministry of Housing and the Saudi Sovereign Wealth fund. One of the goals of Saudi Arabia’s ambitious Vision 2030 is the creation of its own secondary mortgage company. Saudi Arabia has 18 banks and finance companies originating Islamic mortgages, but the future growth of the economy and population is expected to create demand for mortgages that far exceeds the current financial system’s capacity. The travel and hotel accommodations were delightful. The jet lag and working hours were not.

My foremost motivation for taking the project was to check off “worked overseas” from my career bucket list. Having spent my entire career in the U.S. mortgage business, this had always seemed too distant an opportunity. The project was supposed to last three months, but seventeen months later I’m writing this article in a hotel room overlooking downtown Riyadh. The cultural experience living and working in Saudi Arabia is something I have spent hours discussing with family and friends.

But the goal of this article is not to describe my cultural experiences but to write about the lessons I’ve learned about the U.S. mortgage business sitting 7,000 miles away. Below, I’ve laid out some of my observations.

Underwriting is underwriting

As simple as that. Facts, practices and circumstances may be local, but the principles of sound mortgage underwriting are universal: 1) develop your risk criteria, 2) validate and verify the supporting documentation, 3) underwrite the file and 4) capture performance data to confirm your risk criteria. Although mortgage lending is only 10 years old in Saudi Arabia, underwriting criteria and methodologies here strongly resemble those in the USA. Loan-to-value ratios, use of appraisals, asset verification, and debt-to-income (DTI) determination—it’s basically the same. All mortgages are fully documented.

But it is different. In Saudi Arabia, where macro-economic issues—i.e., oil prices and lack of economic diversification—dominate the economy, lenders need to find alternatives in underwriting. For example, the use of credit scores takes a second seat to employment stability. To lenders, a borrower’s employer—i.e., government or the military—is more important than a high credit score. Why? Lower oil prices can crush economic growth, leading to higher unemployment with little opportunity for displaced workers to find new jobs. The lack of a diversified economy makes lenders wary of lending to employees of private-sector companies, hence their focus on lending to government employees. This impact leads to whole segments of potential borrowers being left out of the mortgage market.

The cold reality in emerging economic countries like Saudi Arabia is that only the best borrowers can get loans. Even then, lenders may require a “salary assignment,” in which a borrower’s employer pays the lender directly. The lesson is that the primary credit risk strategy in Saudi Arabia is to avoid credit losses by all means—the best way to manage credit risk is to avoid it.

Finance is finance

Finance is the same everywhere and concepts of cash flow and return analysis are universal, whether the transaction is Islamic or conventional. There’s lots of confusion about what Islamic finance is and how it works. Many people misunderstand shariah law and its rules on paying interest. Not all banks in Saudi Arabia are Islamic, and although many are, while paying interest on debt is non-sharia, leases and equity returns are sharia compliant. The key to Islamic finance is selecting appropriate finance products that comply with shariah but also meet the needs of lenders.

In Saudi Arabia, most lenders originate Islamic mortgages called Ijarah. With an Ijarah mortgage the borrower selects a property to purchase and then goes to the lender. At closing the lender accepts a down payment from the borrower and the lender purchases the property directly from the seller. The lender then executes an agreement to lease the property to the borrower for the life of the mortgage. This looks a lot like a long-term lease. Instead of paying an interest rate, the borrower pays an APR on a stated equity return or “profit rate” to the lender on the lease arrangement.

Similarly, Islamic warehouse lending on mortgage collateral resembles a traditional repo transaction—an agreed upon sale price and repurchase price and a bunch of commodity trades linked to the transaction. In Islamic finance, the art relies on a sound understanding of the cash flows, the collateral limitations, the needs of all parties, and Islamic law. Over the past decade, the needs of the lenders, investors and intermediaries has evolved into set of standardized transactions that meet the financing needs of the market.

People are people

People are the same everywhere—good, bad and otherwise—and it’s no different overseas. And there is a lot of great talent out there. The people I have worked with are talented, motivated and educated. I have had the opportunity to work with Saudis and people from at least 15 other countries. Fortunately for me, English is the operating business language in Saudi Arabia and no one is any wiser to whether my explanations of the U.S. mortgage market are accurate or not. The international consulting and accounting firms have done a tremendous job creating strong business models to identify, hire, train and manage employees, cultivating a rich talent pool of consultants and future employees. A rich country like Saudi Arabia is a magnet for expats—it has both the money and vision to afford talent. In addition, Saudi Arabia’s rapid population growth and strong education system has added to a homegrown pool of talented employees.

Standardization is a benefit worth fighting for

One of the primary goals of any international refinance or secondary market company is standardization. The benefits of standardization extend to all market participants—borrowers, lenders and investors. Secondary market companies thrive where transactions are cheaper, faster and better, making it an easy choice for government policymakers to support. For consumers, rates are lower, the choices of lenders and products are better, and the origination process is more transparent. For investors, the standardization of structures, cash flows and obligations improves liquidity, increases the number of active market participants and ultimately lowers the transactional bid/ask spreads and yields.

However, the benefits of standardization are less clear for the primary customer they are meant to help—the lenders. While standardization can lower operating expenses or improve business processes, it does little to increase the comparative advantages of each lender.

Saudi lenders are focused on customer service and product design, leaving price aside. This focus has led lenders to design mortgage products with unique interest rate adjustment periods, payment options and one-of-a-kind mortgage notes and customized purchase and sale agreements.[1] This degree of customization can be a recipe for disaster, leading to endless negotiations, misunderstandings of rate reset mechanisms, extended deal timelines, and differences of opinion among shariah advisers. When negotiations are culturally a zero-sum game, trying to persuade lenders of the rationale for advancing monthly payments by the 10^th of each month is exhausting.

Saudi lenders see the long-term benefits of increased volume, selling credit exposure and servicing income. But they haven’t figured out that strong secondary markets lead to the development of tertiary markets like forward trading in MBS, trading of Mortgage Servicing Rights (MSRs) or better terms for warehouse lending.

Mortgages are sold, not purchased

It’s a universal tenet throughout the world: buying real estate and financing it with a mortgage is a complex transaction. It requires experienced and well-trained loan officers to aid and walk the consumers through the process. A loan officer’s skill at persuading a potential customer to submit a loan application is every bit as important as his knowledge of mortgages. It’s no different in Saudi Arabia. While building relationships with realtors is important, the Saudi market is more of a construction-to-permanent market than a resale market. Individuals builders are simply too small to be able to channel consumers to lenders.

What to do? The Saudi mortgage origination market has quickly evolved to using alternatives like social media to capture consumer traffic. Saudi citizens are some of the most active users of social media in world.[2] (How active? From my experience, 9 out of 10 drivers on the road are reading their smart phones instead on looking at the road—it’s downright scary.) Lenders have developed sophisticated media campaigns using Twitter, You Tube and other platforms to drive traffic to their call centers where loan officers can sell mortgages to potential borrowers.

Whatever the language, closing lines are the same everywhere.

Regulation – A necessary evil

Saudi Arabia’s is a highly regulated financial market. Its primary financial regulator is the Saudi Arabia Monetary Authority, better known as SAMA. Regulation and oversight is centrally controlled and has been in place for almost 70 years. SAMA has placed a premium on well-capitalized financial institutions and closely monitors transactions and the liquidity of its institutions. The approval process is detailed and time consuming, but it has resulted in well-capitalized institutions. The minimum capital of the country’s five non-bank mortgage lenders exceeds $100MM USD.

A secondary role of SAMA has been to maintain stability within the financial markets—protecting consumers against bad actors and minimizing the market’s systematic risks. Financial literacy among Saudi citizens is low and comprehensive consumer protections akin to the Real Estate Settlement Procedures Act (RESPA) in the U.S. don’t exist here. SAMA fills this role, resulting in an ad hoc mix of consumer protections with mixed enforcement actions. Sometimes the cost of the protection is greater than evil it’s ostensibly protecting against.

As examples, SAMA regulates the maximum LTVs for the mortgage market and limits the consumer’s out-of-pocket cash fees to $1,250 USD. Managing LTV limits for the market goes a long way toward preventing over-lending when the markets are speculative. This was extremely beneficial in cooling down a hot Saudi real estate market in 2013.

Capping a borrower’s out-of-pocket expenses makes sense to limit unscrupulous market players from hustling borrowers. But the downside is the inability of lenders to monetize their transactions—i.e., to get cash from borrowers, sell mortgages at premium prices or sell servicing rights. This results in higher mortgage rates as lenders push up their mortgage coupons to generate cash to reimburse them for the higher costs associated with originating the mortgage. It is also a factor in the lenders’ use of prepayment penalties.

External constraints affect the design of local mortgage products

Ultimately, mortgage financing products available to consumers in any country are a function of the maturity level and the previous legacy development of its financial and capital markets. In Saudi Arabia, where large banks dominate, the deposit funding strategies determine mortgage product design. Capital markets are relatively new in the Kingdom. Only in the past several years has the Saudi government issued enough Sukuks to fill the Saudi Arabian yield curve out to ten years. While the government has plenty of buyers for its debt, the primary mortgage lenders do not. The concept of amortizing debt products is anathema to the market’s debt investors. Without access to longer-term debt buyers, the mortgage market products are primarily linked to 1-year SAIBOR (the Saudi version of LIBOR). This inability to secure long-term funding impacts amortization periods the lenders can offer, with most mortgages limited to a maximum amortization period of 20 years. The high mortgage rates, short-fixed payment tenors and short amortization periods all contribute to affordability issues for the average Saudi citizen.

Affordable Housing is an issue everywhere

Over the past 50 years Saudi Arabia’s vast oil wealth has enabled it to become an educated, middle-class society. The trillions of dollars in oil revenues have enabled the country to transform from a nomadic culture to a modern economy with growth centered in its primary cities. But its population growth rate and urban migration has created a mismatch of affordable housing in the growth centers of the country. The lack of affordable urban housing, outdated government housing policies and restrictive mortgage lending policies has stifled both the demand and supply of affordable housing units.

While well-functioning capital markets can help to lower mortgage rates and improve credit terms, it is only a small part of the solution for helping people afford and remain in housing. In this regard, Saudi Arabia looks a lot like the United States. With entities like the Real Estate Development Fund (REDF), Saudia Arabia is trying to manage the challenges of creating housing programs that solve housing issues for all, as opposed to subsidy programs that only help a small minority of people, operating with the high cost of program administration and with nominal benefits to its participants.

Concluding Thoughts

The past year and half have been both personally and professionally rewarding. The opportunity to live and work abroad and to become immersed in another culture has been gratifying. Professionally, it’s been eye-opening to see the limits of my previous experiences and need to recalibrate my core assumptions and thinking.

I maintain that the United States absolutely has the best mortgage finance system in the world. The ability of our secondary markets to provide consumers with low mortgage rates and a 30-yr fixed rate mortgage has no match in the world. The modern U.S. mortgage market, with its century of history and supportive policy decisions, has the luxury of scale, government guarantees and depth of investor classes.

Saudi Arabia’s own mortgage solutions are mostly a result of necessity. For the country, it has been more important to build a stable and well-capitalized banking system—and then to provide affordable mortgage products and terms. Think of it in terms of airline safely instructions—secure your own oxygen mask first, and then take care of your children.

Housing finance systems aren’t like building smart phone networks. You can’t just import the technology and billing systems and flip a switch. It’s a long-cycle development that requires the legal systems, regulatory framework and entities and a mature finance industry before you can start contemplating and building a secondary market.

As I reflect on my experiences in Saudi Arabia, I would describe the role I have played as that of an intermediary—applying proven “best in class” secondary market and risk management approaches I learned at home to Saudi Arabia. And then trying to understand their limits and coming up with Plan B. And sometimes Plan C…

[1] Competition has not prompted an expansion of the credit box, as lenders are generally risk averse and their regulators are hyper diligent on credit standards.

[2] https://www.go-gulf.com/blog/social-media-saudi-arabia/

MDM to the Rescue for Financial Institutions

Data becomes an asset only when it is efficiently harnessed and managed. Because firms tend to evolve into silos, their data often gets organized that way as well, resulting in multiple references and unnecessary duplication of data that dilute its value. Master Data Management (MDM) architecture helps to avoid these and other pitfalls by applying best practices to maximize data efficiency, controls, and insights.

MDM has particular appeal to banks and other financial institutions where non-integrated systems often make it difficult to maintain a comprehensive, 360-degree view of a customer who simultaneously has, for example, multiple deposit accounts, a mortgage, and a credit card. MDM provides a single, common data reference across systems that traditionally have not communicated well with each other. Customer-level reports can point to one central database instead of searching for data across multiple sources.

Financial institutions also derive considerable benefit from MDM when seeking to comply with regulatory reporting requirements and when generating reports for auditors and other examiners. Mobile banking and the growing number of new payment mechanisms make it increasingly important for financial institutions to have a central source of data intelligence. An MDM strategy enables financial institutions to harness their data and generate more meaningful insights from it by:

Eliminating data redundancy and providing one central repository for common data;
Cutting across data “silos” (and different versions of the same data) by providing a single source of truth;
Streamlining compliance reporting (through the use of a common data source);
Increasing operational and business efficiency;
Providing robust tools to secure and encrypt sensitive data;
Providing a comprehensive 360-degree view of customer data;
Fostering data quality and reducing the risks associated with stale or inaccurate data, and;
Reducing operating costs associated with data management.

Not surprisingly, there’s a lot to think about when contemplating and implementing a new MDM solution. In this post, we lay out some of the most important things for financial institutions to keep in mind.

MDM Choice and Implementation Priorities

MDM is only as good as the data it can see. To this end, the first step is to ensure that all of the institution’s data owners are on board. Obtaining management buy-in to the process and involving all relevant stakeholders is critical to developing a viable solution. This includes ensuring that everyone is “speaking the same language”—that everyone understands the benefits related to MDM in the same way—and establishing shared goals across the different business units.

Once all the relevant parties are on board, it’s important to identify the scope of the business process within the organization that needs data refinement through MDM. Assess the current state of data quality (including any known data issues) within the process area. Then, identify all master data assets related to the process improvement. This generally involves identifying all necessary data integration for systems of record and the respective subscribing systems that would benefit from MDM’s consistent data. The selected MDM solution should be sufficiently flexible and versatile that it can govern and link any sharable enterprise data and connect to any business domain, including reference data, metadata and any hierarchies.

An MDM “stewardship team” can add value to the process by taking ownership of the various areas within the MDM implementation plan. MDM is just not about technology itself but also involves business and analytical thinking around grouping data for efficient usage. Members of this team need to have the requisite business and technical acumen in order for MDM implementation to be successful. Ideally this team would be responsible for identifying data commonalities across groups and laying out a plan for consolidating them. Understanding the extent of these commonalities helps to optimize architecture-related decisions.

Architecture-related decisions are also a function of how the data is currently stored. Data stored in heterogeneous legacy systems calls for a different sort of MDM solution than does a modern data lake architecture housing big data. The solutions should be sufficiently flexible and scalable to support future growth. Many tools in the marketplace offer MDM solutions. Landing on the right tool requires a fair amount of due diligence and analysis. The following evaluation criteria are often helpful:

Enterprise Integration: Seamless integration into the existing enterprise set of tools and workflows is an important consideration for an MDM solution. Solutions that require large-scale customization efforts tend to carry additional hidden costs.
Support for Multiple Devices: Because modern enterprise data must by consumable by a variety of devices (e.g., desktop, tablet and mobile) the selected MDM architecture must support each of these platforms and have multi-device access capability.
Cloud and Scalability: With most of today’s technology moving to the cloud, an MDM solution must be able to support a hybrid environment (cloud as well as on-premise). The architecture should be sufficiently scalable to accommodate seasonal and future growth.
Security and Compliance: With cyber-attacks becoming more prevalent and compliance and regulatory requirements continuing to proliferate, the MDM architecture must demonstrate capabilities in these areas.

Start Small; Build Gradually; Measure Success

MDM implementation can be segmented into small, logical projects based on business units or departments within an organization. Ideally, these projects should be prioritized in such a way that quick wins (with obvious ROI) can be achieved in problem areas first and then scaling outward to other parts of the organization. This sort of stepwise approach may take longer overall but is ultimately more likely to be successful because it demonstrates success early and gives stakeholders confidence about MDM’s benefits.

The success of smaller implementations is easier to measure and see. A small-scale implementation also provides immediate feedback on the technology tool used for MDM—whether it’s fulfilling the needs as envisioned. The larger the implementation, the longer it takes to know whether the process is succeeding or failing and whether alternative tools should be pursued and adopted. The success of the implementation can be measured using the following criteria:

Savings on data storage—a result of eliminating data redundancy.
Increased ease of data access/search by downstream data consumers.
Enhanced data quality—a result of common data centralization.
More compact data lineage across the enterprise—a result of standardizing data in one place.

Practical Case Studies

RiskSpan has helped several large banks consolidate multiple data stores across different lines of business. Our MDM professionals work across heterogeneous data sets and teams to create a common reference data architecture that eliminates data duplication, thereby improving data efficiency and reducing redundant data. These professionals have accomplished this using a variety of technologies, including Informatica, Collibra and IBM Infosphere.

Any successful project begins with a survey of the current data landscape and an assessment of existing solutions. Working collaboratively to use this information to form the basis of an approach for implementing a best-practice MDM strategy is the most likely path to success.

Making Data Dictionaries Beautiful Using Graph Databases

Most analysts estimate that for a given project well over half of the time is spent on collecting, transforming, and cleaning data in preparation for analysis. This task is generally regarded as one of the least appetizing portions of the data analysis process and yet it is the most crucial, as trustworthy analyses are borne out of clean, reliable data. Gathering and preparing data for analysis can be either enhanced or hindered based on the data management practices in place at a firm. When data are readily available, clearly defined, and well documented it will lead to faster and higher-quality insights. As the size and variability of data grows, however, so too does the challenge of storing and managing it. Like many firms, RiskSpan manages a multitude of large, complex datasets with varying degrees of similarity and connectedness. To streamline the analysis process and improve the quantity and quality of our insights, we have made our datasets, their attributes, and relationships transparent and quickly accessible using graph database technology. Graph databases differ significantly from traditional relational databases because data are not stored in tables. Instead, data are stored in either a node or a relationship (also called an edge), which is a connection between two nodes. The image below contains a grey node labeled as a dataset and a blue node labeled as a column. The line connecting these two nodes is a relationship which, in this instance, signifies that the dataset contains the column. Graph 1 There are many advantages to this data structure including decreased redundancy. Rather than storing the same “Column1” in multiple tables for each dataset that contain it (as you would in a relational database), you can simply create more relationships between the datasets demonstrated below: Graph 2 With this flexible structure it is possible to create complex schema that remain visually intuitive. In the image below the same grey (dataset) -contains-> blue (column) format is displayed for a large collection of datasets and columns. Even at such a high level, the relationships between datasets and columns reveal patterns about the data. Here are three quick observations:

In the top right corner there is a dataset with many unique columns.
There are two datasets that share many columns between them and have limited connectivity to the other datasets.
Many ubiquitous columns have been pulled to the center of the star pattern via the relationships to the multiple datasets on the outer rim.

Graph 3 In addition to containing labels, nodes can store data as key-value pairs. The image below displays the column “orig_upb” from dataset “FNMA_LLP”, which is one of Fannie Mae’s public datasets that is available on RiskSpan’s Edge Platform. Hovering over the column node displays some information about it, including the name of the field in the RiskSpan Edge platform, its column type, format, and data type. Graph 4 Relationships can also store data in the same key-value format. This is an incredibly useful property which, for the database in this example, can be used to store information specific to a dataset and its relationship to a column. One of the ways in which RiskSpan has utilized this capability is to hold information pertinent to data normalization in the relationships. To make our datasets easier to analyze and combine, we have normalized the formats and values of columns found in multiple datasets. For example, the field “loan_channel” has been mapped from many unique inputs across datasets to a set of standardized values. In the images below, the relationships between two datasets and loan_channel are highlighted. The relationship key-value pairs contain a list of “mapped_values” identifying the initial values from the raw data that have been transformed. The dataset on the left contains the list: [“BROKER”, “CORRESPONDENT”, “RETAIL”] Graph 5 While the dataset on the right contains: [“R”, “B”, “C”, “T”, “9”] Graph 6 We can easily merge these lists with a node containing a map of all the recognized enumerations for the field. This central repository of truth allows us to deploy easy and robust changes to the ETL processes for all datasets. It also allows analysts to easily query information related to data availability, formats, and values. Graph 7 In addition to queries specific to a column, this structure allows an analyst to answer questions about data availability across datasets with ease. Normally, comparing pdf data dictionaries, excel worksheets, or database tables can be a painstaking process. Using the graph database, however, a simple query can return the intersection of three datasets as shown below. The resulting graph is easy to analyze and use to define the steps required to obtain and manipulate the data. Graph 8 In addition to these benefits for analysts and end users, utilizing graph database technology for data management comes with benefits from a data governance perspective. Within the realm of data stewardship, ownership and accountability of datasets can be assigned and managed within a graph database like the one in this blog. The ability to store any attribute in a node and create any desired relationship makes it simple to add nodes representing data owners and curators connected to their respective datasets. Graph 9 The ease and transparency with which any data related information can be stored makes graph databases very attractive. Graph databases can also support a nearly infinite number of nodes and relationships while also remaining fast. While every technology has a learning curve, the intuitive nature of graphs combined with their flexibility makes them an intriguing and viable option for data management.

Augmenting Internal Loan Data to Comply with CECL and Boost Profit

The importance of sound internal data gathering practices cannot be understated. However, in light of the new CECL standard, many lending institutions have found themselves unable to meet the data requirements. This may have served as a wake-up call for organizations at all levels to look at their internal data warehousing systems and identify and remedy the gaps in their strategies. For some institutions, it may be difficult to consolidate data siloed within various stand-alone systems. Other institutions, even after consolidating all available data, may lack sufficient loan count, timespan, or data elements to meet the CECL standard with internal data alone. This post will discuss some of the strategies to make up for shortfalls while data gathering systems and procedures are built and implemented for the future.

Identify Your Data

The first step is to identify the data that is available. As many tasks go, this is easier said than done. Often, organizations without formal data gathering practices and without a centralized data warehouse find themselves looking at multiple data storage systems across various departments and a multitude of ad-hoc processes implemented in time of need and not upgraded to a standardized solution. However, it is important to begin this process now, if it is not already underway. As part of the data identification phase, it is important to keep track of not only the available variables, but also the length of time for which the data exists, and whether any time periods have missing or unreliable information. In most circumstances, to meet the CECL standard, institutions should have loan performance data that will cover a full economic cycle by the time of CECL adoption. Such data enables an institution to form grounded expectations of how assets will perform over their full contractual lives, across a variety of potential economic climates. Some data points are required regardless of the CECL methodology, while others are necessary only for certain approaches. At this part of the data preparation process, it is more important to understand the big picture than it is to confirm only some of the required fields—it is wise to see what information is available, even if it may not appear relevant at this time. This will prove very useful for drafting the data warehousing procedures, and will allow for a more transparent understanding of requirements should the bank decide to use a different methodology in the future.

Choose Your CECL Methodology

There are many intricacies involved in choosing a CECL Methodology. Each organization should determine both its capabilities and its needs. For example, the Vintage method has relatively simple calculations and limited data requirements, but provides little insight and control for management, and does not yield early model performance indicators. On the other hand, the Discounted Cash Flow method provides many insights and controls, and identifies model performance indicators preemptively, but requires more complex calculations and a very robust data history. It is acceptable to implement a relatively simple methodology at first and begin utilizing more advanced methodologies in the future. Banks with limited historical data, but with procedures in place to ramp up data gathering and data warehousing capabilities, would be well served to implement a method for which all data needs are met. They can then work toward the goal of implementing a more robust methodology once enough historical data is available. However, if insufficient data exists to effectively implement a satisfactory methodology, it may be necessary to augment existing historical data with proxy data as a bridge solution while your data collections mature.

Augment Your Internal Data

Choose Proxy Data

Search for cost-effective datasets that give historical loan performance information about portfolios that are reasonably similar to your go-forward portfolio. Note that proxy portfolios do not need to perfectly resemble your portfolio, so long as either a) the data provider offers filtering capability that enables you to find the subset of proxy loans that matches your portfolio’s characteristics, or b) you employ segment- or loan-level modeling techniques that apply the observations from the proxy dataset in the proportions that are relevant to your portfolio. RiskSpan’s Edge platform contains a Data Library that offers historical loan performance datasets from a variety of industry sources covering multiple asset classes:

For commercial real estate (CRE) portfolios, we host loan-level data on all CRE loans guaranteed by the Small Business Administration (SBA) dating back to 1990. Data on loans underlying CMBS securitizations dating back to 1998, compiled by Trepp, is also available on the RiskSpan platform.
For commercial and industrial (C&I) portfolios, we also host loan-level on all C&I loans guaranteed by the SBA dating back to 1990.
For residential mortgage loan portfolios, we offer large agency datasets (excellent, low-cost options for portfolios that share many characteristics with GSE portfolios) and non-agency datasets (for portfolios with unique characteristics or risks).
By Q3 2018, we will also offer data for auto loan portfolios and reverse mortgage portfolios (Home Equity Conversion Mortgages).

Note that for audit purposes, limitations of proxy data and consequent assumptions for a given portfolio need to be clearly outlined, and all severe limitations addressed. In some cases, multiple proxy datasets may be required. At this stage, it is important to ensure that the proxy data contains all the data required by the chosen CECL methodology. If such proxy data is not available, a different CECL model may be best.

Prepare Your Data

The next step is to prepare internal data for augmentation. This includes standard data-keeping practices, such as accurate and consistent data headers, unique keys such as loan numbers and reporting dates, and confirmation that no duplicates exist. Depending on the quality of internal data, additional analysis may also be required. For example, all data fields need to be displayed in a consistent format according to the data type, and invalid data points, such as FICO scores outside the acceptable range, need to be cleansed. If the data is assembled manually, it is prudent to automate the process to minimize the possibility of user error. If automation is not possible, it is important to implement data quality controls that verify that the dataset is generated according to the metadata rules. This stage provides the final opportunity to identify any data quality issues that may have been missed. For example, if, after cleansing the data for invalid FICO scores, it appears that the dataset has many invalid entries, further analysis may be required, especially if borrower credit score is one of the risk metrics used for CECL modeling. Once internal data preparation is complete, proxy metadata may need to be modified to be consistent with internal standards. This includes data labels and field formats, as well as data quality checks to ensure that consistent criteria are used across all datasets.

Identify Your Augmentation Strategy

Once the internal data is ready and its limitations identified, analysts need to confirm that the proxy data addresses these gaps. Note that it is assumed at this stage that the proxy data chosen contains information for loans that are consistent with the internal portfolio, and that all proxy metadata items are consistent with internal metadata. For example, if internal data is robust, but has a short history, proxy data needs to cover the additional time periods for the life of the asset. In such cases, augmenting internal data is relatively simple: the datasets are joined, and tested to ensure that the join was successful. Testing should also cover the known limitations of the proxy data, such as missing non-required fields or other data quality issues deemed acceptable during the research and analysis phase. More often, however, there is a combination of data shortfalls that lead to proxy data needs, which can include either time-related gaps, data element gaps, or both. In such cases, the augmentation strategy is more complex. In the cases of optional data elements, a decision to exclude certain data columns is acceptable. However, when incorporating required elements that are inputs for the allowance calculation, the data must be used in a way that complies with regulatory requirements. If internal data has incomplete information for a given variable, statistical methods and machine learning tools are useful to incorporate the proxy data with the internal data, and approximate the missing variable fields. Statistical testing is then used to verify that the relationships between actual and approximated figures are consistent with expectation, which are then verified by management or expert analysis. External research on economic or agency data, where applicable, can further be used to justify the estimated data assumptions. While rigorous statistical analysis is integral for the most accurate metrics, the qualitative analysis that follows is imperative for CECL model documentation and review.

Justify Your Proxy Data

Overlaps in time periods between internal loan performance datasets and proxy loan performance datasets are critical in establishing the applicability of the proxy dataset. A variety of similarity metrics can be calculated that compare the performance of the proxy loans with the internal loan during the period of overlap. Such similarity metrics can be put forward to justify the use of the proxy dataset. The proxy dataset can be useful for predictions even if the performance of the proxy loans is not identical to the performance of the institutions’ loans. As long as there is a reliable pattern linking the performance of the two datasets, and no reason to think that pattern will discontinue, a risk-adjusting calibration can be justified and applied to the proxy data, or to results of models built thereon.

Why Augment Internal Data?

While the task of choosing the augmentation strategy may seem daunting, there are concrete benefits to supplementing internal data with a proxy, rather than using simply the proxy data on its own. Most importantly, for the purpose of calculating the allowance for a given portfolio, incorporating some of the actual values will in most cases produce the most accurate estimate. For example, your institution may underwrite loans conservatively relative to the rest of the industry—incorporating at least some of the actual data associated with the lending practices will make it easier to understand how the proxy data differs from characteristics unique to your business. More broadly, proxy data is useful beyond CECL reporting, and has other applications that can boost bank profits. For example, lending institutions can build better predictive models based on richer datasets to calibrate loan screening and loan pricing decisions. These datasets can also be built into existing models to provide better insight on risk metrics and other asset characteristics, and to allow for more fine-tuned management decisions.

Applying Machine Learning to Conventional Model Validations

In addition to transforming the way in which financial institutions approach predictive modeling, machine learning techniques are beginning to find their way into how model validators assess conventional, non-machine-learning predictive models. While the array of standard statistical techniques available for validating predictive models remains impressive, the advent of machine learning technology has opened new avenues of possibility for expanding the rigor and depth of insight that can be gained in the course of model validation. In this blog post, we explore how machine learning, in some circumstances, can supplement a model validator’s efforts related to:

Outlier detection on model estimation data
Clustering of data to better understand model accuracy
Feature selection methods to determine the appropriateness of independent variables
The use of machine learning algorithms for benchmarking
Machine learning techniques for sensitivity analysis and stress testing

Outlier Detection

Conventional model validations include, when practical, an assessment of the dataset from which the model is derived. (This is not always practical—or even possible—when it comes to proprietary, third-party vendor models.) Regardless of a model’s design and purpose, virtually every validation concerns itself with at least a cursory review of where these data are coming from, whether their source is reliable, how they are aggregated, and how they figure into the analysis.

Conventional model validation techniques sometimes overlook (or fail to look deeply enough at) the question of whether the data population used to estimate the model is problematic. Outliers—and the effect they may be having on model estimation—can be difficult to detect using conventional means. Developing descriptive statistics and identifying data points that are one, two, or three standard deviations from the mean (i.e., extreme value analysis) is a straightforward enough exercise, but this does not necessarily tell a modeler (or a model validator) which data points should be excluded.

Machine learning modelers use a variety of proximity and projection methods for filtering outliers from their training data. One proximity method employs the K-means algorithm, which groups data into clusters centered around defined “centroids,” and then identifies data points that do not appear to belong to any particular cluster. Common projection methods include multi-dimensional scaling, which allows analysts to view multi-dimensional relationships among multiple data points in just two or three dimensions. Sophisticated model validators can apply these techniques to identify dataset problems that modelers may have overlooked.

Data Clustering

The tendency of data to cluster presents another opportunity for model validators. Machine learning techniques can be applied to determine the relative compactness of individual clusters and how distinct individual clusters are from one another. Clusters that do not appear well defined and blur into one another are evidence of a potentially problematic dataset—one that may result in non-existent patterns being identified in random data. Such clustering could be the basis of any number of model validation findings.

Feature (Variable) Selection

What conventional predictive modelers typically refer to as variables are commonly referred to by machine learning modelers as features. Features and variables serve essentially the same function, but the way in which they are selected can differ. Conventional modelers tend to select variables using a combination of expert judgment and statistical techniques. Machine learning modelers tend to take a more systematic approach that includes stepwise procedures, criterion-based procedures, lasso and ridge regresssion and dimensionality reduction. These methods are designed to ensure that machine learning models achieve their objectives in the simplest way possible, using the fewest possible number of features, and avoiding redundancy. Because model validators frequently encounter black-box applications, directing applying these techniques is not always possible. In some limited circumstances, however, model validators can add to the robustness of their validations by applying machine learning feature selection methods to determine whether conventionally selected model variables resemble those selected by these more advanced means (and if not, why not).

Benchmarking Applications

Identifying and applying an appropriate benchmarking model can be challenging for model validators. Commercially available alternatives are often difficult to (cost effectively) obtain, and building challenger models from scratch can be time-consuming and problematic—particularly when all they do is replicate what the model in question is doing.

While not always feasible, building a machine learning model using the same data that was used to build a conventionally designed predictive model presents a “gold standard” benchmarking opportunity for assessing the conventionally developed model’s outputs. Where significant differences are noted, model validators can investigate the extent to which differences are driven by data/outlier omission, feature/variable selection, or other factors.

Sensitivity Analysis and Stress Testing

The sheer quantity of high-dimensional data very large banks need to process in order to develop their stress testing models makes conventional statistical analysis both computationally expensive and problematic. (This is sometimes referred to as the “curse of dimensionality.”) Machine learning feature selection techniques, described above, are frequently useful in determining whether variables selected for stress testing models are justifiable.

Similarly, machine learning techniques can be employed to isolate, in a systematic way, those variables to which any predictive model is most and least sensitive. Model validators can use this information to quickly ascertain whether these sensitivities are appropriate. A validator, for example, may want to take a closer look at a credit model that is revealed to be more sensitive to, say, zip code, than it is to credit score, debt-to-income ratio, loan-to-value ratio, or any other individual variable or combination of variables. Machine learning techniques make it possible for a model validator to assess a model’s relative sensitivity to virtually any combination of features and make appropriate judgments.

————————–

Model validators have many tools at their disposal for assessing the conceptual soundness, theory, and reliability of conventionally developed predictive models. Machine learning is not a substitute for these, but its techniques offer a variety of ways of supplementing traditional model validation approaches and can provide validators with additional tools for ensuring that models are adequately supported by the data that underlies them.

Applying Model Validation Principles to Machine Learning Models

Machine learning models pose a unique set of challenges to model validators. While exponential increases in the availability of data, computational power, and algorithmic sophistication in recent years has enabled banks and other firms to increasingly derive actionable insights from machine learning methods, the significant complexity of these systems introduces new dimensions of risk.

When appropriately implemented, machine learning models greatly improve the accuracy of predictions that are vital to the risk management decisions financial institutions make. The price of this accuracy, however, is complexity and, at times, a lack of transparency. Consequently, machine learning models must be particularly well maintained and their assumptions thoroughly understood and vetted in order to prevent wildly inaccurate predictions. While maintenance remains primarily the responsibility of the model owner and the first line of defense, second-line model validators increasingly must be able to understand machine learning principles well enough to devise effective challenge that includes:

Analysis of model estimation data to determine the suitability of the machine learning algorithm
Assessment of space and time complexity constraints that inform model training time and scalability
Review of model training/testing procedure
Determination of whether model hyperparameters are appropriate
Calculation of metrics for determining model accuracy and robustness

More than one way exists of organizing these considerations along the three pillars of model validation. Here is how we have come to think about it.

Conceptual Soundness

Many of the concepts of reviewing model theory that govern conventional model validations apply equally well to machine learning models. The question of “business fit” and whether the variables the model lands on are reasonable is just as valid when the variables are selected by a machine as it is when they are selected by a human analyst. Assessing the variable selection process “qualitatively” (does it make sense?) as well as quantitatively (measuring goodness of fit by calculating residual errors, among other tests) takes on particular importance when it comes to machine learning models.

Machine learning does not relieve validators of their responsibility assess the statistical soundness of a model’s data. Machine learning models are not immune to data issues. Validators protect against these by running routine distribution, collinearity, and related tests on model datasets. They must also ensure that the population has been appropriately and reasonably divided into training and holdout/test datasets.

Supplementing these statistical tests should be a thorough assessment of the modeler’s data preparation procedures. In addition to evaluating the ETL process—a common component of all model validations—effective validations of machine learning models take particular notice of variable “scaling” methods. Scaling is important to machine learning algorithms because they generally do not take units into account. Consequently, a machine learning model that relies on borrower income (generally ranging between tens of thousands and hundreds of thousands of dollars), borrower credit score (which generally falls within a range of a few hundred points) and loan-to-value ratio (expressed as a percentage), needs to apply scaling factors to normalize these ranges in order for the model to correctly process each variable’s relative importance. Validators should ensure that scaling and normalizations are reasonable.

Model assumptions, when it comes to machine learning validation, are most frequently addressed by looking at the selection, optimization, and tuning of the model’s hyperparameters. Validators must determine whether the selection/identification process undertaken by the modeler (be it grid search, random search, Bayesian Optimization, or another method—see this blog post for a concise summary of these) is conceptually sound.

Process Verification

Machine learning models are no more immune to overfitting and underfitting (the bias-variance dilemma) than are conventionally developed predictive models. An overfitted model may perform well on the in-sample data, but predict poorly on the out-of-sample data. Complex nonparametric and nonlinear methods used in machine learning algorithms combined with high computing power are likely to contribute to an overfitted machine learning model. An underfitted model, on the other hand, performs poorly in general, mainly due to an overly simplified model algorithm that does a poor job at interpreting the information contained within data.

Cross-validation is a popular technique for detecting and preventing the fitting or “generalization capability” issues in machine learning. In K-Fold cross-validation, the training data is partitioned into K subsets. The model is trained on all training data except the Kth subset, and the Kth subset is used to validate the performance. The model’s generalization capability is low if the accuracy ratios are consistently low (underfitted) or higher on the training set but lower on the validation set (overfitted). Conventional models, such as regression analysis, can be used to benchmark performance.

Outcomes Analysis

Outcomes analysis enables validators to verify the appropriateness of the model’s performance measure methods. Performance measures (or “scoring methods”) are typically specialized to the algorithm type, such as classification and clustering. Validators can try different scoring methods to test and understand the model’s performance. Sensitivity analyses can be performed on the algorithms, hyperparameters, and seed parameters. Since there is no right or wrong answer, validators should focus on the dispersion of the sensitivity results.

Many statistical tactics commonly used to validate conventional models apply equally well to machine learning models. One notable omission is the ability to precisely replicate the model’s outputs. Unlike with an OLS or ARIMA model, for which a validator can reasonably expect to be able to match the model’s coefficients exactly if given the same data, machine learning models can be tested only indirectly—by testing the conceptual soundness of the selected features and assumptions (hyperparameters) and by evaluating the process and outputs. Applying model validation tactics specially tailored to machine learning models allows financial institutions to deploy these powerful tools with greater confidence by demonstrating that they are of sound conceptual design and perform as expected.

Choosing a CECL Methodology

CECL presents institutions with a vast array of choices when it comes to CECL loss estimation methodologies. It can seem a daunting challenge to winnow down the list of possible methods. Institutions must consider considering competing concerns – including soundness and auditability, cost and feasibility, and the value of model reusability. Institutions must convince not only themselves but also external stakeholders that their methodology choices are reasonable, and often on a segment by segment basis, as methodology can vary by segment. It benefits banks, however, to narrow the field of CECL methodology choices soon so that they can finalize data preparation and begin parallel testing (generating CECL results alongside incurred-loss allowance estimates). Parallel testing generates advance signals of CECL impact and may itself play a role in the final choice of allowance methodology. In this post, we provide an overview of some of the most common loss estimation methodologies that banks and credit unions are considering for CECL, and outline the requirements, advantages and challenges of each.

Methods to Estimate Lifetime Losses

The CECL standard explicitly mentions five loss estimation methodologies, and these are the methodologies most commonly considered by practitioners. Different practitioners define them differently. Additionally, many sound approaches combine elements of each method. For this analysis, we will discuss them as separate methods, and use the definitions that most institutions have in mind when referring to them:

Vintage,
Loss Rate,
PDxLGD,
Roll Rate, and
Discount Cash Flow (DCF).

While CECL allows the use of other methods—for example, for estimating losses on individual collateral-dependent loans—these five methodologies are the most applicable to the largest subset of assets and institutions. For most loans, the allowance estimation process entails grouping loans into segments, and for each segment, choosing and applying one of the methodologies above. A common theme in FASB’s language regarding CECL methods is flexibility: rather than prescribing a formula, FASB expects that the banks consider historical patterns and the macroeconomic and credit policy drivers thereof, and then extrapolate based on those patterns, as well as each individual institution’s macroeconomic outlook. The discussion that follows demonstrates some of this flexibility within each methodology but focuses on the approach chosen by RiskSpan based on our view of CECL and our industry experience. We will first outline the basics of each methodology, followed by their data requirements, and end with the advantages and challenges of each approach.

Vintage Method

Using the Vintage method, historical losses are tabulated by vintage and by loan age, as a percentage of origination balances by vintage year. In the example below, known historical values appear in the white cells, and forecasted values appear in shaded cells. We will refer to the entire shaded region as the “forecast triangle” and the cells within the forecast triangle as “forecast cells.”[/vc_column_text][/vc_column][/vc_row]

A simple way to populate the forecast cells is with the simple average of the known values from the same column. In other words, we calculate the average marginal loss rate for loans of each age and extrapolate that forward. The limitation of this approach is that it does not differentiate loss forecasts based on the bank’s macroeconomic outlook, which is a core requirement of CECL, so a bank using this method will need to incorporate its macroeconomic outlook via management adjustments and qualitative factors (Q-factors).

As an alternative methodology, RiskSpan has developed an approach to forecast the loss triangle using statistical regression, developing a regression model that estimates the historical loss rates in the vintage matrix as a function of loan age, a credit indicator, and a macroeconomic variable, and then applies that regression equation along with a forecast for the macroeconomic variable (and a mean-reversion process) to populate the forecast triangle. The forecast cells can still be adjusted by management as desired, and/or Q-factors can be used. We caution, however, that management should take care not to double-count the influence of macroeconomics on allowance estimates (i.e., once via models, and again via Q-factors)

Once the results of the regression are ready and adjustments are applied where needed, the final allowance can be derived as follows:

Loss Rate Method

Using the Loss Rate method, the average lifetime loss rate is calculated for historical static pools within a segment. This average lifetime loss rate of a is used as the basis to predict the lifetime loss rate of the current static pool—that is, the loans on the reporting-date balance sheet.

In this context, a static pool refers to a group of loans that were on the balance sheet as of a particular date, regardless of when they were originated. For example, within an institutions’ owner-occupied commercial real estate portfolio, the 12/31/06 static pool would refer to all such loans that were on the institution’s balance sheet as of December 31, 2006. We would measure the lifetime losses of such a static pool beginning on the static pool date (December 31, 2006, in this example) and express those losses as a percentage of the balance that existed on the static pool date. This premise is consistent with what CECL asks us to do, i.e., estimate all future credit losses on the loans on the reporting-date balance sheet.

A historical static pool fully aged if all loans that made up the pool are either paid in full or charged off, where payments in full include renewals that satisfy the original contract. We should be wary of including partially aged static pools in the development of average lifetime loss estimates, because the cumulative loss rates of partially aged pools constitute life-to-date loss rates rather than complete lifetime loss rates, and inherently understates the lifetime loss rate that is required by CECL.

To generate the most complete picture of historical losses, RiskSpan constructs multiple overlapping static pools within the historical dataset of a given segment and calculates the average of the lifetime loss rates of all fully aged static pools. This provides an average lifetime loss rate over a business cycle as the soundest basis for a long-term forecast. This technique also allows, but does not require, the use of statistical techniques to estimate lifetime loss rate as a function of the credit mix of a static pool.

After the average lifetime loss rate has been determined, we can incorporate management’s view of how the forward-looking environment will differ from the lookback period over which the lifetime loss rates were calculated, via Q-Factors.

The final allowance can be derived as follows:

PDxLGD Method

Methods ranging from very simple to very sophisticated go by the name “PD×LGD.” At the most sophisticated end of the spectrum are models that calculate loan-by-loan, month-by-month, macro-conditioned probabilities of default and corresponding loss given default estimates. Such estimates can be used in a discounted cash flow context. These estimates can also be used outside of a cash flow context; we can summarize these monthly estimates into a cumulative default probability and corresponding exposure-at-default and loss-given-default estimates, which yield a single lifetime loss rate estimate. At the simpler end of the spectrum are calculations of the lifetime default rates and corresponding loss given default rates of static pools (not marginal monthly or annual default rates). This simpler calculation is the method that most institutions have in mind when referring to “PD×LGD methods,” so it is the definition we will use here.

Using this PDxLGD method, the loss rate is calculated based on the same static pool concept as that of the Loss Rate method. As with the Loss Rate method, we can use the default rates and loss given default rates of different static pools to quantify the relationship between those rates and the credit mix of the segment, and to use that relationship going forward based on the credit mix of today’s portfolio. However, under PDxLGD, the loss rate is a function of two components: the lifetime default rate (PD), and the loss given default (LGD). The final allowance can be derived as follows:

Because the PDxLGD and Loss Rate methods derive the Expected Loss Rate for the segment using different but related approaches, one of the important quality controls is to verify that the final calculated rates are equal under both methodologies, and that the cause of any discrepancies is investigated.

Roll Rate Method

Using the Roll Rate method, ultimate losses are predicted based on historical roll rates and the historical loss given default estimate. Roll rates are either (a) the frequency with which loans transition from one delinquency status to another, or (b) the frequency with which loans “migrate” or “transition” from one risk grade to another. While the former is preferred due to its transparency and objectivity, for institutions with established risk grades, the latter is an appropriate metric.

Under this method, management can apply adjustments for macroeconomic and other factors at the individual roll rate level, as well as on-top adjustments as needed. Roll rate matrices can included prepayment as a possible transition, thereby incorporating prepayment probabilities. Roll rates can be used in a cash flow engine that incorporates contractual loan features and generates probabilistic (expected) cash flows, or outside of a cash flow engine to generate expected chargeoffs of amortized cost. Finally, it is possible to use statistical regression techniques to express roll rates as a function of macroeconomic variables, and thus, to condition future roll rates on macroeconomic expectations.

The final allowance can be derived as follows:

Discounted Cash Flow (DCF) Method

Discounting cash flows is a way of translating expected future cash flows into a present value. DCF is a loan-level method (even for loans grouped into segments), and thus requires loan-by-loan, month-by-month forecasts of prepayment, default, and loss-given-default forecasts to translate contractual cash flows into prepay-, default-, and loss-given-default-adjusted cash flows. Although such loan-level, monthly forecasts could be derived using any method, most institutions have statistical forecasting techniques in mind when thinking about a DCF approach. Thus, even though statistical forecasting techniques and cash flow discounting are not inextricably linked, we will treat them as a pair here.

The most complex, and the most robust, of the five methodologies, DCF (paired with statistical forecasting techniques) is generally used by larger institutions that have the capacity and the need for the greatest amount of insight and control. Critically, DCF capabilities give institutions the ability (when substituting the effective interest rate for a market-observed discount rate) to generate fair value estimates that can serve a host of accounting and strategic purposes.

To estimate future cash flows, RiskSpan uses statistical models, which comprise:

Prepayment sub-models
Probability-of-default or roll rate sub-models
Loss-given-default sub-models

Allowance is then determined based on the expected cash flows, which, similarly to the Roll Rate method, are generated based on the rates predicted by the statistical models, contractual loan terms, and the loan status at the reporting date.

Some argue that an advantage of the discounted cash flow approach is lower Day 1 losses. Whether DCF or non-DCF methods produce a lower Day 1 allowance, all else equal, depends upon the length of the assumed liquidation timeline, the discount rate, and the recovery rate. This is an underdiscussed topic that merits its own blog post. We will cover this fully in a future post.

The statistical models often used with DCF methods use historical data to express the likelihood of default or prepayment as a mathematical function of loan-level credit factors and macroeconomic variables.

For example, the probability of transitioning from “Current” status to “Delinquent” at month_t can be calculated as a function of that loan’s loan age at multiplied by a sensitivity factor β₁ on the loan age variable derived based on the data in the historical dataset, the loan’s FICO multiplied by a sensitivity factor β₂, and the projected unemployment rate based on management’s macroeconomic assumptions at month_t multiplied by a sensitivity factor β₃. Mathematically,

Because macroeconomic and loan-level credit factors are explicitly and transparently incorporated into the forecast, such statistical techniques reduce reliance on Q-Factors. This is one of the reasons why such methods are the most scientific.

Historical Data Requirements

The table below summarizes the historical data requirements for each methodology, including the dataset type, the minimum required data fields, and the timespan.

In conclusion, having the most robust data allows the most options; for institutions with moderately complex historical datasets, Loss Rate, PDxLGD, and Vintage are excellent options. With limited historical data, the Vintage method can produce a sound allowance under CECL.

While the data requirements may be daunting, it is important to keep in mind that proxy data can be used in place of, or alongside, institutional historical data, and RiskSpan can help identify and fill your data needs. Some of the proxy data options are summarized below:

Advantages and Challenges of CECL Methodologies

Each methodology has advantages, and each carries its own set of challenges. While the Vintage method, for example, is forgiving to limited historical data, it also provides limited insight and control for further analysis. On the other hand, the DCF method provides significant insight and control, as well as early model performance indicators, but requires a robust dataset and advanced statistical expertise.

We have summarized some of the advantages and challenges for each method below.

Advantages and Challenges of CECL Methodologies

In addition to the considerations summarized in the table, it is important to consider audit and regulatory requirements. Generally, institutions facing higher audit and regulatory scrutiny will be steered toward more complex methods. Also, bankers who intend to leverage the loan forecasting model they use for CECL for strategic decision-making (for example, loan screening and pricing decisions), and who desire granular insight and dials around their allowance numbers, will gravitate toward methodologies that afford more precision. At the other end of the spectrum, the methods that provide less precision and insight generally come with lighter operational burden.

Choosing Your CECL Methodology

Choosing the method that’s right for you depends on many factors, from historical data availability to management objectives and associated operational costs.

In many cases, management can gain a better understanding of the institutional allowance requirements after analyzing the results determined by multiple complementary approaches.

RiskSpan is willing to talk further with individual institutions about their circumstances, as well as generate sample results using a set of various methodologies.

The Surging Reverse Mortgage Market

Momentum continues to build around reverse mortgages and related products. Persistent growth in both home prices and the senior population has stoked renewed interest and discussion about the most appropriate uses of accumulated home equity in financial planning strategies. A common and superficial way to think of reverse mortgages is as a “last-resort” means of covering expenses when more conventional planning tools prove insufficient. But experts increasingly are not thinking of reverse mortgages in this way. Last week, the American College of Financial Services and the Bipartisan Policy Center hosted the 2018 Housing Wealth in Retirement Symposium. Speakers represented policy research think tanks, institutional asset managers, large banks, and AARP. Notwithstanding the diversity of viewpoints, virtually every speaker reiterated a position that financial planners have posited for years: financial products that leverage home equity should, in many cases, be integrated into comprehensive retirement planning strategies, rather than being reserved as a product of last resort.

Senior Home Equity Continues Trending Upward

The National Reverse Mortgage Lenders Association (NRMLA) and RiskSpan have published the Reverse Mortgage Market Index (RMMI) since the beginning of 2000. The RMMI provides a trending measure of home equity of U.S. homeowners age 62 and older. The RMMI defines senior home equity as the difference between the aggregate value of homes owned and occupied by seniors and the aggregate mortgage balance secured by those homes. This measure enables the RMMI to help gauge the potential market size of those who may be qualified for a reverse mortgage product. The chart below illustrates the steady increase in this index since the end of the 2008 recession. It reached its latest all-time high in the most recent quarter (Q4 2017). Increasing house prices drive this trend, mitigated to some extent by a corresponding modest increase in mortgage debt held by seniors. The most recent RMMI report is published on NRMLA’s website. As summarized below by the Urban Institute, home equity can be extracted through many mechanisms, primarily Federal Housing Administration (FHA)–insured Home Equity Conversion Mortgages (HECMs), closed-end home equity loans, home equity lines of credit (HELOCs), and cash-out refinancing.

Share of Homeowners Who Extracted Home Equity by Strategy

The Urban Institute research goes on to point out that although few seniors have extracted home equity to date, the market is potentially very large (as reflected by the RMMI index) and more extraction is likely in the years ahead as the senior population both grows and ages. The data in the following chart confirm what one might reasonably expect—that younger seniors are more likely to have existing mortgages than older seniors.

Reverse Mortgage as Retirement Planning Tool

Looking at senior home equity in the context of overall net worth lends support to financial planners’ view of products like reverse mortgages as more than something on which to fall back as a last resort. The first three rows of data in the table below contains the median net worth by age cohort in 2013 and 2016, respectively, from Federal Reserve Board’s Survey of Consumer Finances. The bottom row, highlighted in yellow, is the estimated average senior home equity (total senior home equity as computed by the RMMI divided by senior population) for the same years. We acknowledge the imprecision inherent in this comparison due to the statistical method used (median vs. average) and certain data limitations on RMMI (addressed below). Additionally, the net worth figures may include non-homeowners. Nonetheless, home equity is an unignorably important component of senior net worth.

Following the release of the Federal Reserve’s 2016 Survey of Consumer Finances https://www.federalreserve.gov/econres/scfindex.htm, the Urban Institute published a summary research paper “What the 2016 Survey of Consumer Finances Tells Us about Senior Homeowners” https://www.urban.org/sites/default/files/publication/94526/what-the-2016-survey-of-consumer-finances-tells-us-about-senior-homeowners.pdf in November 2017. The paper notes that “Worries about retirement security are rooted in several factors, such as Social Security changes that shrink the share of preretirement earnings replaced by the program (Munnell and Sundén 2005), rising medical and long-term care costs (Johnson and Mommaerts 2009, 2010), student loan burdens, and the shift from employer-sponsored defined-benefit pension plans that guarantee lifetime income to 401(k)-type defined-contribution plans whose account balances depend on employee contributions and uncertain investment returns (Munnell 2014; Munnell and Sundén 2005). In addition, increased life expectancies require retirement savings to last longer.”

The financial position of seniors is evolving. Forty-one percent of homeowners age 65 and older now have a mortgage on their primary residence, compared with just 21 percent in 1989, and the median outstanding debt has risen from $16,793 to $72,000, according to the Urban Institute. As more households enter retirement with more debt, a growing number will likely tap into their home as a source of income. Hurdles and challenges remain, however, and education will play an important role in fostering responsible use of reverse mortgage products.

Note on the Limitations of RMMI

To calculate the RMMI, an econometric tool is developed to estimate senior housing value, senior mortgage level, and senior equity using data gathered from various public resources such as American Community Survey (ACS), Federal Reserve Flow of Funds (Z.1), and FHFA housing price indexes (HPI). The RMMI is simply the senior equity level at time of measure relative to that of the base quarter in 2000.[1] The main limitation of RMMI is non-consecutive data, such as census population. We use a smoothing approach to estimate data in between the observable periods and continue to look for ways to improve our methodology and find more robust data to improve the precision of the results. Until then, the RMMI and its relative metrics (values, mortgages, home equities) are best analyzed at a trending macro level, rather than at more granular levels, such as MSA.

[1] There was a change in RMMI methodology in Q3 2015 mainly to calibrate senior homeowner population and senior housing values observed in 2013 American Community Survey (ACS).

Machine Learning Detects Model Validation Blind Spots

Machine learning represents the next frontier in model validation—particularly in the credit and prepayment modeling arena. Financial institutions employ numerous models to make predictions relating to MBS performance. Validating these models by assessing their predictions is of paramount importance, but even models that appear to perform well based upon summary statistics can have subsets of input (input subspaces) for which they tend to perform poorly. Isolating these “blind spots” can be challenging using conventional model validation techniques, but recently developed machine learning algorithms are making the job easier and the results more reliable.

High-Error Subspace Visualization

RiskSpan’s modeling team has developed a statistical algorithm which identifies high-error subspaces and flags model outputs corresponding to inputs originating from these subspaces, indicating to model users that the results might be unreliable. An extension to this problem that we also address is whether migration of data points to more error-prone subspaces of the input space over time can be indicative of macroeconomic regime shifts and signal a need to re-estimate the model. This will aid in the prevention of declining model efficacy over time.

Due to the high-dimensional nature of the input spaces of many financial models, traditional statistical methods of partitioning data may prove inadequate. Using machine learning techniques, we have developed a more robust method of high-error subspace identification. We develop the algorithm using loan performance model data, but the method is adaptable to generic models.

Data Selection and Preparation

The dataset we use for our analysis is a random sample of the publicly available Freddie Mac Loan-Level Dataset. The entire dataset covers the monthly loan performance for loans originated from 1999 to 2016 (25.4 million fixed-rate mortgages). From this set, one million loans were randomly sampled. Features of this dataset include loan-to-value ratio, borrower debt-to-income ratio, borrower credit score, interest rate, and loan status, among others. We aggregate the monthly status vectors for each loan into a single vector which contains a loan status time series over the life of the loan within the historical period. This aggregated status vector is mapped to a value of 1 if the time series indicates the loan was ever 90 days delinquent within the first three years after its origination, representing a default, and 0 otherwise. This procedure results in 914,802 total records.

Algorithm Framework

Using the prepared loan dataset, we estimate a logistic regression loan performance model. The data is sampled and partitioned into training and test datasets for clustering analysis. The model estimation and training data is taken from loans originating in the period from 1999 to 2007, while loans originating in the period from 2008 to 2016 are used for testing. Once the data has been partitioned into training and test sets, a clustering algorithm is run on the training data.

Two-Dimensional Visualization of Select Clusters

The clustering is evaluated based upon its ability to stratify the loan data into clusters that meaningfully identify regions of the input for which the model performs poorly. This requires the average model performance error associated with certain clusters to be substantially higher than the mean. After the training data is assigned to clusters, cluster-level error is computed for each cluster using the logistic regression model. Clusters with high error are flagged based upon a scoring scheme. Each loan in the test set is assigned to a cluster based upon its proximity to the training cluster centers. Loans in the test set that are assigned to flagged clusters are flagged, indicating that the loan comes from a region for which loan performance model predictions exhibit lower accuracy.

Algorithm Performance Analysis

The clustering algorithm successfully flagged high-error regions of the input space, with flagged test clusters exhibiting accuracy more than one standard deviation below the mean. The high errors associated with clusters flagged during model training were persistent over time, with flagged clusters in the test set having a model accuracy of just 38.7%, compared to an accuracy of 92.1% for unflagged clusters. Failure to address observed high-error clusters in the training set and migration of data to high-error subspaces led to substantially diminished model accuracy, with overall model accuracy dropping from 93.9% in the earlier period to 84.1% in the later period.

Training/Test Cluster Error Comparison

Additionally, the nature of default misclassifications and variables with greatest impact on misclassification were also determined. Cluster FICO scores proved to be a strong indicator of cluster model prediction accuracy. While a relatively large proportion of loans in low-FICO clusters defaulted, the logistic regression model substantially overpredicted the number of defaults for these clusters, leading to a large number of Type I errors (inaccurate default predictions) for these clusters. Type II (inaccurate non-default predictions) errors constituted a smaller proportion of overall model error, and their impact was diminished even further when considering their magnitude relative to the number of true negative predictions (accurate non-default predictions), which are far fewer in number than true positive predictions (accurate default predictions).

FICO vs. Cluster Accuracy

Conclusion

Our application of the subspace error identification algorithm to a loan performance model illustrates the dangers of using high-level summary statistics as the sole determinant of model efficacy and failure to consistently monitor the statistical profile of model input data over time. Often, more advanced statistical analysis is required to comprehensively understand model performance. The algorithm identified sets of loans for which the model was systematically misclassifying default status. These large-scale errors come at a high cost to financial institutions employing such models.

As an extension to this research into high error subspace detection, RiskSpan is currently developing machine learning analytics tools that can detect the root cause of systematic model errors and suggest ways to enhance predictive model performance by alleviating these errors.

Permissioned Blockchains–A Quest for Consensus

Conspicuously absent from all the chatter around blockchain’s potential place in structured finance has been much discussion around the thorny matter of consensus. Consensus is at the heart of all distributed ledger networks and is what enables them to function without a trusted central authority. Consensus algorithms are designed to prevent fraud and error. With large, public blockchains, achieving consensus—ensuring that all new information has been examined before is universally accepted—is relatively straightforward. It is achieved either by performing large amounts of work or simply by members who collectively hold a majority stake in the blockchain.

However, when it comes to private (or “permissioned”) blockchains with a relatively small number of interested parties—the kind of blockchains that are currently poised for adoption in the structured finance space—the question of how to obtain consensus takes on an added layer of complexity. Restricting membership greatly reduces the need for elaborate algorithms to prevent fraud on permissioned blockchains. Instead, these applications must ensure that complex workflows and transactions are implemented correctly. They must provide a framework for having members agree to the very structure of the transaction itself. Consensus algorithms complement this by ensuring that the steps performed in verifying transaction data is agreed upon and verified.

With widespread adoption of blockchain in structured finance appearing more and more to be a question of when rather than if, SmartLink Labs, a RiskSpan fintech affiliate, recently embarked on a proof of concept designed to identify and measure the impact of the technology across the structured finance life cycle. The project took a holistic approach, looking at everything from deal issuance to bondholder payments. We sought to understand the benefits, how various roles would change, and the extent to which certain functions might be eliminated altogether. At the heart of virtually every lesson we learned along the way was a common, overriding principle: consensus is hard.

Why is Consensus Hard?

Much of blockchain’s appeal to those of us in the structured finance arena has to do with its potential to lend visibility and transparency to complicated payment rules that govern deals along with dynamic borrower- and collateral-level details that evolve over the lives of the underlying loans. Distributed ledgers facilitate the real-time sharing of these details across all relevant parties—including loan originators, asset servicers, and bond administrators—from deal issuance through the final payment on the transaction. The ledger transactions are synchronized to ensure that ledgers only update when the appropriate participants approve transactions. This is the essence of consensus, and it seems like it ought to be straightforward.

Imagine our surprise when one of the most significant challenges our test implementation encountered was designing the consensus algorithm. Unlike with public blockchains, consensus in a private, or “permissioned,” blockchain is designed for a specific business purpose where the counterparties are known. However, to achieve consensus, the data posted to the blockchain must be verified in an automated manner by the relevant parties to the transaction. One of the challenges with the data and rules that govern most structured transactions is that it is (at best) only partially digital. We approached our project with the premise that most business terms can be translated into a series of logical statements in the form of computer code. Translating unstructured data into structured data in a fully transparent way is problematic, however, and limitations to transparency represent a significant barrier to achieving consensus. In order for a distributed ledger to work in this context, all transaction parties need to reach consensus around how the cash will flow and numerous other business rules throughout the process.

A Potential Solution for Structured Finance

To this end, our initial prototype seeks to test our consensus algorithm on the deal waterfall model. If the industry can move to a process where consensus of the deal waterfall model is achieved at deal issuance, the model posted to the blockchain can then serve as an agreed-upon source of truth and perpetuate through the life of the security—from loan administration to master servicer aggregation and bondholder payments. This business function alone could save the industry countless hours and effectively eliminate all of today’s costs associated with having to model and remodel each transaction multiple times.

Those of us who have been in the structured finance business for 25 years or more know how little the fundamental business processes have evolved. They remain manual, governed largely by paper documents, and prone to human error.

The mortgage industry has proven to be particularly problematic. Little to no transparency in the process has fostered a culture of information asymmetry and general mistrust which has predictably given rise to the need to have multiple unrelated parties double-checking data, performing due diligence reviews on virtually all loan files, validating and re-validating cash flow models, and requiring costly layers of legal payment verification. Ten or more parties might contribute in one way or another to verifying and validating data, documents, or cash flow models for a single deal. Upfront consensus via blockchain holds the potential to dramatically reduce or even eliminate almost all of this redundancy.

Transparency and Real-Time Investor Reporting

The issuance process, of course, is only the beginning. The need for consensus does not end when the cash flow model is agreed to and the deal is finalized. Once we complete a verified deal, the focus of our proof of concept will shift to the monthly process of investor reporting and corresponding payments to the bond holders.

The immutability of transactions posted to the ledger is particularly valuable because of the unmistakable audit trail it creates. Rather than compelling master servicers to rely on a monthly servicing snapshot “tape” to try and figure out what happened to a severely delinquent loan with four instances of non-sufficient funds, a partial payment in suspense, and an interest rate change somewhere in the middle. Putting all these transactions on a blockchain creates a relatively straightforward sequence of transactions that everyone can decipher.

Posting borrower payments to a blockchain in real time will also require consensus among transaction parties. Once this is achieved, the antiquated notion of monthly investor reporting will become obsolete. The potential ramifications of this extend to timing of payments to bond holders. No longer needing to wait until the next month to find out what borrowers did the month before means that payments to investors might be accelerated and, in the private-label security markets, perhaps even more often than monthly. With real-time consensus comes the possibility of far more flexibility for issuers and investors in designing the timing of cash flows should they elect to pursue it.

This envisioned future state is not without its detractors. Some ask why servicers would opt for more transparency when they already encounter more scrutiny and criticism than they would like. In many cases, however, it is the lack of transparency, more than a servicer’s actions themselves, that invite the unwanted scrutiny. Servicers that move beyond reporting monthly snapshots and post comprehensive loan activity to a blockchain stand to reap significant competitive advantages. Because of the real-time consensus and sharing of dynamic loan reporting data (and perhaps of accelerated bond payments, as suggested above) investors will quickly gravitate toward deals that are administered by blockchain-enabled servicers. Sooner or later, servicers who fail to adapt will find themselves on the outside looking in.

Less Redundancy; More Trust

Much of blockchain’s appeal is bound up in the promise of an environment in which deal participants can gain reasonable assurance that their counterparts are disclosing information that is both accurate and comprehensive. Visibility is an important component of this, but ultimately, achieving consensus that what is being done is what ought to be done will be necessary in order to fully eliminate redundant functions in business processes and overcome information asymmetry in the private markets. Sophisticated, well-conceived algorithms that enable private parties to arrive at this consensus in real time will be key.

One of the enduring lessons of our structured finance proof of concept is that consensus is necessary throughout a transaction’s life. The market (i.e., issuers, investors, servicers, and bond administrators) will ultimately determine what gets posted to a blockchain and what remains off-chain, and more than one business model will likely evolve. As data becomes more structured and more reliable, however, competitive advantages will increasingly accrue to those who adopt consensus algorithms capable of infusing trust into the process. The failure of the private-label MBS market to regain its pre-crisis footing is, in large measure, a failure of trust. Nothing repairs trust like consensus.

« First ‹ Prev 14 15 161718 19 20 Next ›Last »

Underwriting is underwriting

Finance is finance

People are people

Standardization is a benefit worth fighting for

Mortgages are sold, not purchased

Regulation – A necessary evil

External constraints affect the design of local mortgage products

Affordable Housing is an issue everywhere

Concluding Thoughts

MDM Choice and Implementation Priorities

Start Small; Build Gradually; Measure Success

Practical Case Studies

Identify Your Data

Choose Your CECL Methodology

Augment Your Internal Data

Choose Proxy Data

Prepare Your Data

Identify Your Augmentation Strategy

Justify Your Proxy Data

Why Augment Internal Data?

Outlier Detection

Data Clustering

Feature (Variable) Selection

Benchmarking Applications

Sensitivity Analysis and Stress Testing

Conceptual Soundness

Process Verification

Outcomes Analysis

Methods to Estimate Lifetime Losses

Vintage Method

Loss Rate Method

PDxLGD Method

Roll Rate Method

Discounted Cash Flow (DCF) Method

Historical Data Requirements

Advantages and Challenges of CECL Methodologies

Choosing Your CECL Methodology

Senior Home Equity Continues Trending Upward

Reverse Mortgage as Retirement Planning Tool

Note on the Limitations of RMMI

Data Selection and Preparation

Algorithm Framework

Algorithm Performance Analysis

Conclusion

Why is Consensus Hard?

A Potential Solution for Structured Finance

Transparency and Real-Time Investor Reporting

Less Redundancy; More Trust

Company

Products

Security & Compliance