A Primer on HECM Loans
In September, RiskSpan announced the addition of Ginnie Mae’s loan-level Home Equity Conversion Mortgage (“HECM”) dataset to the Edge platform. The dataset contains over 330,000 HECM loans with origination dates from 2000 to 2018 and reporting periods from August 2013 to October 2018. This post is a primer on HECM loans, the HMBS securities they collateralize, and the structure of the new dataset. What is a HECM? HECMs are FHA-insured reverse mortgages that provide people 62 and older with cash payments or a line of credit in exchange for equity in their homes. Borrowers are not liable to make any payments on HECM balances until the house ceases to be their primary residence. In contrast to traditional mortgages that amortize down over time, reverse mortgage balances usually grow over time as accrued interest is added to the loan. The Federal Housing Administration (FHA) insures HECM lenders against default and loss and is paid a mortgage insurance premium in return. Because borrowers do not make principal and interest payments, the concept of HECM default differs from that of traditional forward mortgages. HECM default most commonly occurs when borrowers fail to keep current on property tax payments and insurance premiums or otherwise jeopardize the lender’s lien position on the property. Initial loan-to-value (LTV) ratios for HECMs average between 60% and 70% to allow for the balance to grow over time (taking into account borrower age and interest rate). The number of borrowers is arguably a more important factor when predicting HECM performance than when predicting traditional mortgage performance. Because reverse mortgages do not become due until all borrowers have left the property, reverse mortgages with multiple borrowers tend to have longer tenures—and consequently run a higher risk of growing beyond the point where the balance and accrued interest are supported by the underlying property’s value. Like traditional mortgages, HECM interest rates may be fixed or adjustable. Fixed-rate HECMs disburse a single, initial advance, while adjustable-rate HECMs combine a line of credit or monthly advance with an initial advance. Figure 1 (below), which was constructed using data from the newly available dataset, illustrates a steady increase in the share of ARM loans since 2013. 
Figure 1 One net result of this trend is fewer one-time lump-sum distributions and more line-of-credit (LOC) distributions over time. LOCs give borrowers access to a source of funds that they can draw upon as needed. While LOCs constitute (by far) the most common type of HECM, two other loan types—“term” and “tenure”—also occupy the HECM landscape. “Term” loans provide monthly payments for a set period of time. “Tenure” loans provide monthly payments for as long as the borrower lives in the home as a primary residence. The lender receives principal, interest and possibly a share of the home appreciation upon expiry of the fixed term (in the case of term loans) or upon borrower’s death or move-out (in the case of either loan type). The dominance of the LOC loan type relative to term and tenure HECMs is depicted in Figure 2, below. 
Figure 2 Fannie Mae had traditionally functioned as the primary investor in reverse mortgages for most of these loans’ 25-year existence. Since 2009, however, Fannie Mae has significantly scaled back its reverse mortgage portfolio, leaving the majority of the reverse mortgages to be picked up by the Ginnie Mae HMBS market. What is a HMBS? HECM loans are pooled into HECM mortgage-backed securities (HMBS) within the Ginnie Mae II MBS program. HMBS are made up of a pool of participations in the HECM loans. A participation in a HECM loan is a pro-rata share of the loan that is securitized in a HMBS. As explained above, many HECM loans are structured as a line of credit, which allows borrowers to draw on their lines as needed. When these draws occur, the drawn-down loans become a smaller pro-rata share of the loan and the participation balance doesn’t change. HMBS participations have a mandatory repurchase clause requiring a lender to buy back all the participations of a HECM loan when its LTV reaches 98%. For HECM loans, LTV is calculated as a proportion of the current HECM balance against the maximum claim amount. As of June 2018, participation unpaid balance stood at approximately $56.18 billion with 11,380,452 active participations. Figures 3 and 4, below, show the trend of participation composition (by number of participations and UPB) over time. These reflect the shift toward ARM lines of credit (and away from fixed-rate lump sum disbursements) illustrated in Figures 1 and 2. 
Figure 3 
Figure 4 HMBS Dataset Ginnie Mae provides two monthly loan-level files related to the HECMs that collateralize its HMBS offering. One of these files contains fixed-rate and annually adjusting rate loans, and the other contains monthly adjusting rate loans. Because individual security participations are spread across several different pools (often with several column values repeating for a single loan) working with this dataset can be challenging. An example of a single loan spread across multiple security participations is illustrated in the table below. Note that for a single loan ID, the current UPB and Max Claim Amount columns are repeated for each participation.
| Loan ID | Current HECM UPB | Max Claim Amount | Participation UPB |
| 1000033608 | 260,784.73 | 365,000.00 | 860.70 |
| 1000033608 | 260,784.73 | 365,000.00 | 321.87 |
| 1000033608 | 260,784.73 | 365,000.00 | 12,079.98 |
| 1000033608 | 260,784.73 | 365,000.00 | 483.81 |
Table 1 The most important risk factors associated with HECMs relate to borrower mortality and mobility (i.e., borrowers’ remaining in their homes until the increasing mortgage balance exceeds the value of the property). Borrowers are more likely to move out of their homes for health reasons as they age, but they become less likely to move out for other reasons. Having more than one borrower tends to extend the life of a HECM because the loan does not become due until the last surviving borrower leaves the property. As of the most recent reporting period, about 43% of the aggregate HMBS balance was associated with HECMs with more than one borrower. In order to calculate HECM prepayment speeds, we look at the zero balance codes provided in the dataset to exclude loans which have reached a 98% LTV from the opening balance. (As noted earlier, loans must be purchased out of the HMBS once they reach this threshold.) Because interest is deferred in HECM loans, it is added to the opening balance. We calculate the total prepayments and obtain the single monthly mortality to calculate the CPR. Figure 5, below, shows the one-month CPR by vintage over the past five years. 
Figure 5 Because borrower mortality and mobility tend to remain stable over time, HECM prepayment speeds exhibit less variability than traditional mortgages do. An important aspect of evaluating CPR includes looking at the outstanding participation balance relative to borrower age. Figure 6 contains a heatmap plotting borrower age against HECM purpose for the most recent reporting period (July 2018). 
Figure 6 Because most HECM borrowers are younger than age 80, prepayments are likely to increase as this cohort ages and becomes more likely to move out or pass away. Figure 7 below shows the five largest HMBS originators by participation as of July 2018. As discussed above, lines of credit (LOCs) are the most popular HECM type with Single Disbursement Lump Sum the next most frequent. 
Stay tuned for future blog posts in which we will use the Edge platform to glean additional insights from this newly available and very interesting dataset. For information on how to use the Edge platform to conduct your own analyses of this or any other dataset, please contact us.
CRT Exposure to Hurricane Michael
With Hurricane Michael approaching the Gulf Coast, we put together some interactive charts looking at the affected metro areas, and their related CRT exposure (Both CAS and STACR). Given the large area of impact with Hurricane Michael, we have included a nearly exhaustive selection of MSA’s. Click on a deal ID along the left-hand side of the plot to view its exposure to each MSA. Most of the mortgage delinquencies in the wake of Hurricane Harvey quickly cured. Holders of securities backed by loans that ultimately defaulted (typically because the property was completely destroyed) had much of their exposure mitigated by insurance proceeds, government intervention, and other relief provisions.
CRT Deal Monitor: Understanding When Credit Becomes Risky
This analysis tracks several metrics related to deal performance and credit profile, putting them into a historical context by comparing the same metrics for recent-vintage deals against those of ‘similar’ cohorts in the time leading up to the 2008 housing crisis. You’ll see how credit metrics are trending today and understand the significance of today’s shifts in the context of historical data. Some of the charts in this post have interactive features, so click around! We’ll be tweaking the analysis and adding new metrics in subsequent months. Please shoot us an email if you have an idea for other metrics you’d like us to track.
Highlights
- Performance metrics signal steadily increasing credit risk, but no cause for alarm.
- We’re starting to see the hurricane-related (2017 Harvey and Irma) delinquency spikes subside in the deal data. Investors should expect a similar trend in 2019 due to Hurricane Florence.
- The overall percentage of delinquent loans is increasing steadily due to the natural age ramp of delinquency rates and the ramp-up of the program over the last 5 years.
- Overall delinquency levels are still far lower than historical rates.
- While the share of delinquency is increasing, loans that go delinquent are ending up in default at a lower rate than before.
- Deal Profiles are becoming riskier as new GSE acquisitions include higher-DTI business.
- It’s no secret that both GSEs started acquiring a lot of high-DTI loans (for Fannie this moved from around 16% of MBS issuance in Q2 2017 to 30% of issuance as of Q2 this year). We’re starting to see a shift in CRT deal profiles as these loans are making their way into CRT issuance.
- The credit profile chart toward the end of this post compares the credit profiles of recently issued deals with those of the most recent three months of MBS issuance data to give you a sense of the deal profiles we’re likely to see over the next 3 to 9 months. We also compare these recently issued deals to a similar cohort from 2006 to give some perspective on how much the credit profile has improved since the housing crisis.
- RiskSpan’s Vintage Quality Index reflects an overall loosening of credit standards–reminiscent of 2003 levels–driven by this increase in high-DTI originations.
- Fannie and Freddie have fundamental differences in their data disclosures for CAS and STACR.
- Delinquency rates and loan performance all appear slightly worse for Fannie Mae in both the deal and historical data.
- Obvious differences in reporting (e.g., STACR reporting a delinquent status in a terminal month) have been corrected in this analysis, but some less obvious differences in reporting between the GSEs may persist.
- We suspect there is something fundamentally different about how Freddie Mac reports delinquency status—perhaps related to cleaning servicing reporting errors, cleaning hurricane delinquencies, or the way servicing transfers are handled in the data. We are continuing our research on this front and hope to follow up with another post to explain these anomalies.
The exceptionally low rate of delinquency, default, and loss among CRT deals at the moment makes analyzing their credit-risk characteristics relatively boring. Loans in any newly issued deal have already seen between 6 and 12 months of home price growth, and so if the economy remains steady for the first 6 to 12 months after issuance, then that deal is pretty much in the clear from a risk perspective. The danger comes if home prices drift downward right after deal issuance. Our aim with this analysis is to signal when a shift may be occurring in the credit risk inherent in CRT deals. Many data points related to the overall economy and home prices are available to investors seeking to answer this question. This analysis focuses on what the Agency CRT data—both the deal data and the historical performance datasets—can tell us about the health of the housing market and the potential risks associated with the next deals that are issued.
Current Performance and Credit Metrics
Delinquency Trends
The simplest metric we track is the share of loans across all deals that is 60+ days past due (DPD). The charts below compare STACR (Freddie) vs. CAS (Fannie), with separate charts for high-LTV deals (G2 for CAS and HQA for STACR) vs. low-LTV deals (G1 for CAS and DNA for STACR). Both time series show a steadily increasing share of delinquent loans. This slight upward trend is related to the natural aging curve of delinquency and the ramp-up of the CRT program. Both time series show a significant spike in delinquency around January of this year due to the 2017 hurricane season. Most of these delinquent loans are expected to eventually cure or prepay. For comparative purposes, we include a historical time series of the share of loans 60+ DPD for each LTV group. These charts are derived from the Fannie Mae and Freddie Mac loan-level performance datasets. Comparatively, today’s deal performance is much better than even the pre-2006 era. You’ll note the systematically higher delinquency rates of CAS deals. We suspect this is due to reporting differences rather than actual differences in deal performance. We’ll continue to investigate and report back on our findings.
Delinquency Outcome Monitoring
While delinquency rates might be trending up, loans that are rolling to 60-DPD are ultimately defaulting at lower and lower rates. The tables below track the status of loans that were 60+ DPD. Each bar in the chart represents the population of loans that were 60+ DPD exactly 6 months prior to the x-axis date. Over time, we see growing 60-DPD and 60+ DPD groups, and a shrinking Default group. This indicates that a majority of delinquent loans wind up curing or prepaying, rather than proceeding to default. The choppiness and high default rates in the first few observations of the data are related to the very low counts of delinquent loans as the CRT program ramped up. The following table repeats the 60-DPD delinquency analysis for the Freddie Mac Loan Level Performance dataset leading up to and following the housing crisis. (The Fannie Mae loan level performance set yields a nearly identical chart.) Note how many more loans in these cohorts remained delinquent (rather than curing or defaulting) relative to the more recent CRT loans. https://plot.ly/~dataprep/30.embed
Vintage Quality Index
RiskSpan’s Vintage Quality Index (VQI) reflects a reversion to the looser underwriting standards of the early 2000s as a result of the GSEs’ expansion of high-DTI lending. RiskSpan introduced the VQI in 2015 as a way of quantifying the underwriting environment of a particular vintage of mortgage originations. We use the metric as an empirically grounded way to control for vintage differences within our credit model. 
While both GSEs increased high-DTI lending in 2017, it’s worth noting that Fannie Mae saw a relatively larger surge in loans with DTIs greater than 43%. The chart below shows the share of loans backing MBS with DTI > 43. We use the loan-level MBS issuance data to track what’s being originated and acquired by the GSEs because it is the timeliest data source available. CRT deals are issued with loans that are between 6 and 20 months seasoned, and so tracking MBS issuance provides a preview of what will end up in the next cohort of deals. 
Deal Profile Comparison
The tables below compare the credit profiles of recently issued deals. We focus on the key drivers of credit risk, highlighting the comparatively riskier features of a deal. Each table separates the high-LTV (80%+) deals from the low-LTV deals (60%-80%). We add two additional columns for comparison purposes. The first is the ‘Coming Cohort,’ which is meant to give an indication of what upcoming deal profiles will look like. The data in this column is derived from the most recent three months of MBS issuance loan-level data, controlling for the LTV group. These are newly originated and acquired by the GSEs—considering that CRT deals are generally issued with an average loan age between 6 and 15 months, these are the loans that will most likely wind up in future CRT transactions. The second comparison cohort consists of 2006 originations in the historical performance datasets (Fannie and Freddie combined), controlling for the LTV group. We supply this comparison as context for the level of risk that was associated with one of the worst-performing cohorts. The latest CAS deals—both high- and low-LTV—show the impact of increased >43% DTI loan acquisitions. Until recently, STACR deals typically had a higher share of high-DTI loans, but the latest CAS deals have surpassed STACR in this measure, with nearly 30% of their loans having DTI ratios in excess of 43%. CAS high-LTV deals carry more risk in LTV metrics, such as the percentage of loans with a CLTV > 90 or CLTV > 95. However, STACR includes a greater share of loans with a less-than-standard level of mortgage insurance, which would provide less loss protection to investors in the event of a default. 
Low-LTV deals generally appear more evenly matched in terms of risk factors when comparing STACR and CAS. STACR does display the same DTI imbalance as seen in the high-LTV deals, but that may change as the high-DTI group makes its way into deals. 
Deal Tracking Reports
Please note that defaults are reported on a delay for both GSEs, and so while we have CPR numbers available for August, CDR numbers are not provided because they are not fully populated yet. Fannie Mae CAS default data is delayed an additional month relative to STACR. We’ve left loss and severity metrics blank for fixed-loss deals. 
Houston Strong: Communities Recover from Hurricanes. Do Mortgages?
The 2017 hurricane season devastated individual lives, communities, and entire regions. As one would expect, dramatic increases in mortgage delinquencies accompanied these events. But the subsequent recoveries are a testament both to the resilience of the people living in these areas and to relief mechanisms put into place by the mortgage holders.
Now, nearly a year later, we wanted to see what the credit-risk transfer data (as reported by Fannie Mae CAS and Freddie Mac STACR) could tell us about how these borrowers’ mortgage payments are coming along.
The timing of the hurricanes’ impact on mortgage payments can be approximated by identifying when Current-to-30 days past due (DPD) roll rates began to spike. Barring other major macroeconomic events, we can reasonably assume that most of this increase is directly due to hurricane-related complications for the borrowers.
The effect of the hurricanes is clear—Puerto Rico, the U.S. Virgin Islands, and Houston all experienced delinquency spikes in September. Puerto Rico and the Virgin Islands then experienced a second wave of delinquencies in October due to Hurricanes Irma and Maria.
But what has been happening to these loans since entering delinquency? Have they been getting further delinquent and eventually defaulting, or are they curing? We focus our attention on loans in Houston (specifically the Houston-The Woodlands-Sugar Land Metropolitan Statistical Area) and Puerto Rico because of the large number of observable mortgages in those areas.
First, we look at Houston. Because the 30-DPD peak was in September, we track that bucket of loans. To help us understand the path 30-DPD might reasonably be expected to take, we compared the Houston delinquencies to 30-DPD loans in the 48 states other than Texas and Florida.
Of this group of loans in Houston that were 30 DPD in September, we see that while many go on to be 60+ DPD in October, over time this cohort is decreasing in size.
Recovery is slower than the non-hurricane-affected U.S. loans, but persistent. The biggest difference is that a significant number of 30-day delinquencies in the rest of the country loans continue to hover at 30 DPD (rather than curing or progressing to 60 DPD) while the Houston cohort is more evenly split between the growing number loans that cure and the shrinking number of loans progressing to 60+ DPD.
Puerto Rico (which experienced its 30 DPD peak in October) shows a similar trend:
To examine loans even more affected by the hurricanes, we can perform the same analysis on loans that reached 60 DPD status.
Here, Houston’s peak is in October while Puerto Rico’s is in November.
Houston vs. the non-hurricane-affected U.S.:
Puerto Rico vs. the non-hurricane-affected U.S.:
In both Houston and Puerto Rico, we see a relatively small 30-DPD cohort across all months and a growing Current cohort. This indicates many people paying their way to Current from 60+ DPD status. Compare this to the rest of the US where more people pay off just enough to become 30 DPD, but not enough to become Current.
The lack of defaults in post-hurricane Houston and Puerto Rico can be explained by several relief mechanisms Fannie Mae and Freddie Mac have in place. Chiefly, disaster forbearance gives borrowers some breathing room with regards to payment. The difference is even more striking among loans that were 90 days delinquent, where eventual default is not uncommon in the non-hurricane affected U.S. grouping:
And so, both 30-DPD and 60-DPD loans in Houston and Puerto Rico proceed to more serious levels of delinquency at a much lower rate than similarly delinquent loans in the rest of the U.S. To see if this is typical for areas affected by hurricanes of a similar scale, we looked at Fannie Mae loan-level performance data for the New Orleans MSA after Hurricane Katrina in August 2005.
As the following chart illustrates, current-to-30 DPD roll rates peaked in New Orleans in the month following the hurricane:
What happened to these loans?
Here we see a relatively speedy recovery, with large decreases in the number of 60+ DPD loans and a sharp increase in prepayments. Compare this to non-hurricane affected states over the same period, where the number of 60+ DPD loans held relatively constant, and the number of prepayments grew at a noticeably slower rate than in New Orleans.
The remarkable number of prepayments in New Orleans was largely due to flood insurance payouts, which effectively prepay delinquent loans. Government assistance lifted many others back to current. As of March, we do not see this behavior in Houston and Puerto Rico, where recovery is moving much more slowly. Flood insurance incidence rates are known to have been low in both areas, a likely suspect for this discrepancy.
While loans are clearly moving out of delinquency in these areas, it is at a much slower rate than the historical precedent of Hurricane Katrina. In the coming months we can expect securitized mortgages in Houston and Puerto Rico to continue to improve, but getting back to normal will likely take longer than what was observed in New Orleans following Katrina. Of course, the impending 2018 hurricane season may complicate this matter.
—————————————————————————————————————-
Note: The analysis in this blog post was developed using RiskSpan’s Edge Platform. The RiskSpan Edge Platform is a module-based data management, modeling, and predictive analytics software platform for loans and fixed-income securities. Click here to learn more.
MDM to the Rescue for Financial Institutions
Data becomes an asset only when it is efficiently harnessed and managed. Because firms tend to evolve into silos, their data often gets organized that way as well, resulting in multiple references and unnecessary duplication of data that dilute its value. Master Data Management (MDM) architecture helps to avoid these and other pitfalls by applying best practices to maximize data efficiency, controls, and insights.
MDM has particular appeal to banks and other financial institutions where non-integrated systems often make it difficult to maintain a comprehensive, 360-degree view of a customer who simultaneously has, for example, multiple deposit accounts, a mortgage, and a credit card. MDM provides a single, common data reference across systems that traditionally have not communicated well with each other. Customer-level reports can point to one central database instead of searching for data across multiple sources.
Financial institutions also derive considerable benefit from MDM when seeking to comply with regulatory reporting requirements and when generating reports for auditors and other examiners. Mobile banking and the growing number of new payment mechanisms make it increasingly important for financial institutions to have a central source of data intelligence. An MDM strategy enables financial institutions to harness their data and generate more meaningful insights from it by:
- Eliminating data redundancy and providing one central repository for common data;
- Cutting across data “silos” (and different versions of the same data) by providing a single source of truth;
- Streamlining compliance reporting (through the use of a common data source);
- Increasing operational and business efficiency;
- Providing robust tools to secure and encrypt sensitive data;
- Providing a comprehensive 360-degree view of customer data;
- Fostering data quality and reducing the risks associated with stale or inaccurate data, and;
- Reducing operating costs associated with data management.
Not surprisingly, there’s a lot to think about when contemplating and implementing a new MDM solution. In this post, we lay out some of the most important things for financial institutions to keep in mind.
MDM Choice and Implementation Priorities
MDM is only as good as the data it can see. To this end, the first step is to ensure that all of the institution’s data owners are on board. Obtaining management buy-in to the process and involving all relevant stakeholders is critical to developing a viable solution. This includes ensuring that everyone is “speaking the same language”—that everyone understands the benefits related to MDM in the same way—and establishing shared goals across the different business units.
Once all the relevant parties are on board, it’s important to identify the scope of the business process within the organization that needs data refinement through MDM. Assess the current state of data quality (including any known data issues) within the process area. Then, identify all master data assets related to the process improvement. This generally involves identifying all necessary data integration for systems of record and the respective subscribing systems that would benefit from MDM’s consistent data. The selected MDM solution should be sufficiently flexible and versatile that it can govern and link any sharable enterprise data and connect to any business domain, including reference data, metadata and any hierarchies.
An MDM “stewardship team” can add value to the process by taking ownership of the various areas within the MDM implementation plan. MDM is just not about technology itself but also involves business and analytical thinking around grouping data for efficient usage. Members of this team need to have the requisite business and technical acumen in order for MDM implementation to be successful. Ideally this team would be responsible for identifying data commonalities across groups and laying out a plan for consolidating them. Understanding the extent of these commonalities helps to optimize architecture-related decisions.
Architecture-related decisions are also a function of how the data is currently stored. Data stored in heterogeneous legacy systems calls for a different sort of MDM solution than does a modern data lake architecture housing big data. The solutions should be sufficiently flexible and scalable to support future growth. Many tools in the marketplace offer MDM solutions. Landing on the right tool requires a fair amount of due diligence and analysis. The following evaluation criteria are often helpful:
- Enterprise Integration: Seamless integration into the existing enterprise set of tools and workflows is an important consideration for an MDM solution. Solutions that require large-scale customization efforts tend to carry additional hidden costs.
- Support for Multiple Devices: Because modern enterprise data must by consumable by a variety of devices (e.g., desktop, tablet and mobile) the selected MDM architecture must support each of these platforms and have multi-device access capability.
- Cloud and Scalability: With most of today’s technology moving to the cloud, an MDM solution must be able to support a hybrid environment (cloud as well as on-premise). The architecture should be sufficiently scalable to accommodate seasonal and future growth.
- Security and Compliance: With cyber-attacks becoming more prevalent and compliance and regulatory requirements continuing to proliferate, the MDM architecture must demonstrate capabilities in these areas.
Start Small; Build Gradually; Measure Success
MDM implementation can be segmented into small, logical projects based on business units or departments within an organization. Ideally, these projects should be prioritized in such a way that quick wins (with obvious ROI) can be achieved in problem areas first and then scaling outward to other parts of the organization. This sort of stepwise approach may take longer overall but is ultimately more likely to be successful because it demonstrates success early and gives stakeholders confidence about MDM’s benefits.
The success of smaller implementations is easier to measure and see. A small-scale implementation also provides immediate feedback on the technology tool used for MDM—whether it’s fulfilling the needs as envisioned. The larger the implementation, the longer it takes to know whether the process is succeeding or failing and whether alternative tools should be pursued and adopted. The success of the implementation can be measured using the following criteria:
- Savings on data storage—a result of eliminating data redundancy.
- Increased ease of data access/search by downstream data consumers.
- Enhanced data quality—a result of common data centralization.
- More compact data lineage across the enterprise—a result of standardizing data in one place.
Practical Case Studies
RiskSpan has helped several large banks consolidate multiple data stores across different lines of business. Our MDM professionals work across heterogeneous data sets and teams to create a common reference data architecture that eliminates data duplication, thereby improving data efficiency and reducing redundant data. These professionals have accomplished this using a variety of technologies, including Informatica, Collibra and IBM Infosphere.
Any successful project begins with a survey of the current data landscape and an assessment of existing solutions. Working collaboratively to use this information to form the basis of an approach for implementing a best-practice MDM strategy is the most likely path to success.
Making Data Dictionaries Beautiful Using Graph Databases
Most analysts estimate that for a given project well over half of the time is spent on collecting, transforming, and cleaning data in preparation for analysis. This task is generally regarded as one of the least appetizing portions of the data analysis process and yet it is the most crucial, as trustworthy analyses are borne out of clean, reliable data. Gathering and preparing data for analysis can be either enhanced or hindered based on the data management practices in place at a firm. When data are readily available, clearly defined, and well documented it will lead to faster and higher-quality insights. As the size and variability of data grows, however, so too does the challenge of storing and managing it. Like many firms, RiskSpan manages a multitude of large, complex datasets with varying degrees of similarity and connectedness. To streamline the analysis process and improve the quantity and quality of our insights, we have made our datasets, their attributes, and relationships transparent and quickly accessible using graph database technology. Graph databases differ significantly from traditional relational databases because data are not stored in tables. Instead, data are stored in either a node or a relationship (also called an edge), which is a connection between two nodes. The image below contains a grey node labeled as a dataset and a blue node labeled as a column. The line connecting these two nodes is a relationship which, in this instance, signifies that the dataset contains the column. 
There are many advantages to this data structure including decreased redundancy. Rather than storing the same “Column1” in multiple tables for each dataset that contain it (as you would in a relational database), you can simply create more relationships between the datasets demonstrated below: 
With this flexible structure it is possible to create complex schema that remain visually intuitive. In the image below the same grey (dataset) -contains-> blue (column) format is displayed for a large collection of datasets and columns. Even at such a high level, the relationships between datasets and columns reveal patterns about the data. Here are three quick observations:
- In the top right corner there is a dataset with many unique columns.
- There are two datasets that share many columns between them and have limited connectivity to the other datasets.
- Many ubiquitous columns have been pulled to the center of the star pattern via the relationships to the multiple datasets on the outer rim.
In addition to containing labels, nodes can store data as key-value pairs. The image below displays the column “orig_upb” from dataset “FNMA_LLP”, which is one of Fannie Mae’s public datasets that is available on RiskSpan’s Edge Platform. Hovering over the column node displays some information about it, including the name of the field in the RiskSpan Edge platform, its column type, format, and data type. 
Relationships can also store data in the same key-value format. This is an incredibly useful property which, for the database in this example, can be used to store information specific to a dataset and its relationship to a column. One of the ways in which RiskSpan has utilized this capability is to hold information pertinent to data normalization in the relationships. To make our datasets easier to analyze and combine, we have normalized the formats and values of columns found in multiple datasets. For example, the field “loan_channel” has been mapped from many unique inputs across datasets to a set of standardized values. In the images below, the relationships between two datasets and loan_channel are highlighted. The relationship key-value pairs contain a list of “mapped_values” identifying the initial values from the raw data that have been transformed. The dataset on the left contains the list: [“BROKER”, “CORRESPONDENT”, “RETAIL”] 
While the dataset on the right contains: [“R”, “B”, “C”, “T”, “9”] 
We can easily merge these lists with a node containing a map of all the recognized enumerations for the field. This central repository of truth allows us to deploy easy and robust changes to the ETL processes for all datasets. It also allows analysts to easily query information related to data availability, formats, and values. 
In addition to queries specific to a column, this structure allows an analyst to answer questions about data availability across datasets with ease. Normally, comparing pdf data dictionaries, excel worksheets, or database tables can be a painstaking process. Using the graph database, however, a simple query can return the intersection of three datasets as shown below. The resulting graph is easy to analyze and use to define the steps required to obtain and manipulate the data. 
In addition to these benefits for analysts and end users, utilizing graph database technology for data management comes with benefits from a data governance perspective. Within the realm of data stewardship, ownership and accountability of datasets can be assigned and managed within a graph database like the one in this blog. The ability to store any attribute in a node and create any desired relationship makes it simple to add nodes representing data owners and curators connected to their respective datasets. 
The ease and transparency with which any data related information can be stored makes graph databases very attractive. Graph databases can also support a nearly infinite number of nodes and relationships while also remaining fast. While every technology has a learning curve, the intuitive nature of graphs combined with their flexibility makes them an intriguing and viable option for data management.
