Having good Prepayment and Credit Models is critical in the analysis of Residential Mortgage-Backed Securities. Prepays and Defaults are the two biggest risk factors that traders, portfolio managers and originators have to deal with. Traditionally, regression-based Behavioral Models have been used to accurately predict human behavior. Since prepayments and defaults are not just complex human decisions but also competing risks, accurately modeling them has been challenging. With the exponential growth in computing power (GPUs, parallel processing), storage (Cloud), “Big Data” (tremendous amount of detailed historical data) and connectivity (high speed internet), Artificial Intelligence (AI) has gained significant importance over the last few years. Machine Learning (ML) is a subset of AI and Deep Learning (DL) is a further subset of ML. The diagram below illustrates this relationship:
Due to the technological advancements mentioned above, ML based prepayment and credit models are now a reality. They can achieve better predictive power than traditional models and can deal effectively with high-dimensionality (more input variables) and non-linear relationships. The major drawback which has kept them from being universally adopted is their “black box” nature which leads to validation and interpretation issues. Let’s do a quick comparison between traditional and ML models:
Within ML Models are two ways to train them:
- Supervised Learning (used for ML Prepay and Credit Models)
- Regression based
- Classification based
- Unsupervised Learning
- Clustering
- Association
Let’s compare the major differences between Supervised and Unsupervised Learning:
The large amounts of loan level time series data available for RMBS (agency and non-agency) lends itself well for the construction of ML models and early adopters have reported higher accuracy. Besides the obvious objections mentioned above (black box, lack of control, interpretation) ML models are also susceptible to overfitting (like all other models). Overfitting is when a model does very well on the training data but less well on unseen data (validation set). The model ends up “memorizing” the noise and outliers in the input data and is not able to generalize accurately. The non-parametric and non-linear nature of ML Models accentuates this problem. Several techniques have been developed to address this potential problem: reducing the complexity of decision trees, expanding the training dataset, adding weak learners, dropouts, regularization, reducing the training time, cross validation etc.. The interpretation problem is a bit more challenging since users demand both, predictive accuracy and some form of interpretability. Several interpretation methods are used currently, like PDP (Partial dependence plot), ALE (accumulated local effects), PFI (permutation feature importance) and ICE (individual conditional expectation) but each has its shortcomings. Some of the challenges with the interpretability methods are:
- Isolating Cause and Effect – This is not often possible with supervised ML models since they only exploit associations and do not explicitly model cause/effect relationships.
- Mistaking Correlation for Dependence – Independent variables have a correlation coefficient of zero but a zero correlation coefficient may not imply independence. The correlation coefficient only tracks linear correlations and the non-linear nature of the models makes this difficult.
- Feature interaction and dependence – An incorrect conclusion can be drawn about the features influence on the target when there are interactions and dependencies between them.
While ML based prepay and credit models offer better predictive accuracy and automatically capture feature interactions and non-linear effects, they are still a few years away from gaining widespread acceptance. A good use for such models, at this stage, would be to use them in conjunction with traditional models. They would be a good benchmark to test traditional models with.
Note: Some of the information on this post was obtained from publicly available sources on the internet. The author wishes to thank Lei Zhao and Du Tang of the modeling group for proofreading this post.
