Linkedin    Twitter   Facebook

Get Started
Log In

Linkedin

Category: Article
Home / Article /

Humans in the Loop: Ensuring Trustworthy AI in Private ABF Deal Modeling

As generative AI becomes a powerful tool in Private asset-backed finance (ABF), the need for precision and transparency is more critical than ever. At RiskSpan, we’re applying Large Language Models (LLMs) to automate and accelerate private ABF deal modeling and surveillance. But speed is only half the battle—accuracy is non-negotiable.

That’s where Human-in-the-Loop (HITL) validation plays a vital role. While the RiskSpan platform incorporates sophisticated AI guardrails, we believe the right blend of automation and expert oversight ensures results that are not just fast—but reliable, auditable, and production-ready.

The AI-Powered Workflow: What’s Automated

Our private ABF modeling and surveillance system uses LLMs to handle several critical tasks:

  • Data Extraction: Parsing offering memos, indentures, and loan tapes to extract structural and financial data.
  • Deal Code Generation: Producing executable waterfall models based on extracted rules.
  • Database Ingestion: Uploading validated deal terms and triggers into the RiskSpan system of record.
  • Surveillance Automation: Running periodic deal performance analyses and compliance checks.

But What About Hallucinations?

Generative models are powerful but imperfect. Without the right controls, they can fabricate securitization tranches or fees that are not present in the deal documents. They can also misinterpret waterfall rules or omit critical override conditions or generate semantically incorrect code for cashflow models. To address these challenges, RiskSpan employs a multi-layered safeguard framework, combining asset class based extraction; LLM-as-Judge; Rule-Based Guardrails and Inline Human Review

Humans in the Loop: Three Layers of Oversight

We’ve embedded human validation at three key points in the deal lifecycle:

  1. Pre-Modeling Validation: before LLM-generated outputs are finalized, RiskSpan analysts review extracted terms and model prompts—correcting anything misaligned with the source documents.
  2. Inline Oversight: during waterfall code generation, humans validate AI-generated logic in context, ensuring correct treatment of subordination, triggers, caps/floors and other.
  3. Post-Deployment Monitoring Surveillance: outputs are reviewed both by the RiskSpan team and client-side structuring or credit teams. Feedback is looped back into model tuning and prompt optimization.

Looking Ahead: RAG and Continuous Improvement

We’re actively exploring Retrieval-Augmented Generation (RAG) to reduce hallucinations even further. By grounding AI responses in deal-specific material such as offering documents, trustee reports, and internal risk memos—we aim to: 1) eliminate off-topic responses. 2) increase trust in model-derived outputs and 3) enable deeper customization per issuer or asset class.

The Takeaway

AI is transforming how private ABF deals are modeled and monitored—but it must be grounded and guided by human expertise and built for institutional rigor. At RiskSpan, we’re not just accelerating workflows—we’re raising the bar for accuracy and trust in AI-assisted private structures. Human-in-the-loop is not a fallback—it’s a strategic pillar. Want to see how our AI platform works in action? Reach out to schedule a demo or contact your RiskSpan representative to learn more.

June 2025 Models & Markets Update – Predictive Power Amid Economic Uncertainty

Register here for next month’s call: Thursday, July 17th, 2025, 1 p.m.

Each month, we host a Models & Markets call to offer our insights into recent model performance, emerging credit risks, and broader economic indicators. This month, we showcased our responsiveness to shifting macroeconomic dynamics and introduced new transparency elements (i.e., back-testing tools) to our prepayment and credit modeling.

Click here to listen to the entire 23-minute recording, or continue reading for a summary.

Agency Prepay Model: Back-testing and Enhanced Control

We are launching a new loan-level prepayment back-testing tool using nearly all agency loans (FN/FH/GN) aged 10 years or less. The tool runs every month through our models with historical home prices and interest rates. Based on this data, we have an interactive dashboard that will allow users to drill down into model performance with far more granularity than currently possible.

Key Enhancements to Prepay Model v. 3.8

A soon to be released version of the prepay model will include:

  • User-defined slope multipliers for both Out-of-the-Money (OTM) and In-the-Money (ITM) performance, offering finer control over refinance sensitivity and turnover behavior.
  • Independent knob control across CONV 30, CONV 15, FHA, and VA loan types.

A redesigned ARM prepayment framework, derived from the fixed-rate model. The new ARM component includes:

  • A realistic payment shock element that aligns prepayment spikes with rate reset events.
  • Improved seasonality and aging ramp that reflects empirical loan behavior

These updates give users the ability to more precisely tune model responses under a variety of macroeconomic and borrower scenarios.

Credit Model: V7 and Delinquency Transitions

The delinquency transition matrix incorporated into our new Credit Model V7 provides users a more nuanced credit risk assessment. This model works in conjunction with the enhanced prepayment model to better simulate the joint dynamics of default and prepay behavior across economic cycles.

Macroeconomic Context: Rates and Risk in a Holding Pattern

We remain cautious in our outlook for the remainder of 2025 and into 2026:

The Fed Funds Rate is expected to remain elevated—currently in the 4.25–4.50% range—with the first rate cut likely in September. By year-end 2025, the market expects it to settle around 3.75–4.00%.

Mortgage rates remain stubbornly high, hovering above 6.5%, putting pressure on origination volumes and reinforcing the value of accurate prepayment modeling.

Home prices and broader macro indicators like unemployment and PCE inflation remain stable, suggesting a “wait-and-see” mode for both consumers and investors.

What’s Next: More Models, More Tools, More Insights

We continue to expand our Platform with new analytics, model documentation, and client-facing tools. Users can soon access the new back-testing report directly within the Platform, alongside these updated prepayment and credit models. These developments reflect our commitment to model transparency, data-driven innovation, and practical tools for real-time market adaptation.

Contact us to learn more.

Private Credit Market Pulse: What LPs Want from Their Data and How to Deliver It

Limited Partners (LPs) continue to demand better data, faster, and with full transparency. At this week’s Private Credit Tech Summit in New York, I moderated a panel of industry leaders for a discussion on where LP expectations are heading and the challenges managers face trying to meet them.

My fellow panelists included:

·      Charlie Tafoya, Co-Founder and CEO of Chronograph

·      Marios Tsiptis, Senior Portfolio Manager

·      Arnab Mazumdar, Partner at Pantheon

We explored the evolving expectations of LPs, the operational hurdles General Partners (GPs) face, and the technology shaping the next frontier of data transparency in private credit.

What follows is where we landed.

LP Expectations Are Outpacing the Status Quo

Quarterly performance reports and aggregate numbers used to be enough. But Marios Tsiptis explained that today’s LPs (particularly insurance companies) want detailed, timely insight into exactly what they own and what risks they’re carrying.

From Pantheon’s fund-of-funds vantage point, Arnab Mazumdar laid out three foundational data pillars:

1.     Performance metrics

2.     Operating and credit-level data

3.     Consistency across managers

Lagging information and inconsistencies across investment structures, especially between SMAs and feeder funds, create significant friction. Structure matters, and data must be complete, accurate, and delivered on LP terms.

The Operational Reality: Bridging Ambition and Execution

Charlie Tafoya, who works closely with GPs via Chronograph, provided a sobering view of the day-to-day realities. Many managers are eager to modernize, but they’re grappling with:

·      Delays in data delivery

·      Data quality and validation issues

·      Fragmented internal processes

There’s just no getting around the fact that investment teams simply must be embedded in the data flow. Too often, front-office insights are siloed from the operations teams responsible for reporting. “The investment team is the source of truth,” Charlie noted, making their engagement essential to any successful data transformation.

Meanwhile, Arnab called for a convergence of internal monitoring and external reporting. Aligning what GPs see internally with what they share externally could yield benefits across the board—but cultural and technological hurdles remain.

Technology as Enabler, Not Panacea

Tools that support data interoperability, real-time reporting, and workflow automation are rapidly maturing. But challenges persist around integration with legacy systems and data standardization.

From the LP side, Marios painted a picture of an ideal future. Intuitive dashboards, seamless access across portfolios, and fully integrated delivery pipelines all featured prominently in this future. But the road leading there is still under construction.

According to Arnab, the next few years will hinge on industry standardization and early adopters gaining a competitive edge, while laggards risk being left behind.

Transparency builds trust—but it requires real operational change.

More than just a tech problem, this requires a cultural shift. GPs and LPs need to work as true partners in designing data ecosystems that are not only robust and scalable, but also reflect the growing sophistication of the private credit space.

As the private markets evolve, so too must the infrastructure that supports them. Those who succeed in this transition will not only meet the expectations of today’s LPs, but also shape tomorrow’s.

Design Smarter — How AI is Changing UX from Idea to Execution 

AI is revolutionizing everything, and the UX design process is no exception. From the earliest conceptual ideas all the way through to final execution, the transformation is not just about speeding up workflows but also about enhancing creativity and collaboration.  

Here’s how. 

Initial Ideation

Every UX journey begins with the ideation process. AI tools like Claude have become a go-to starting point for brainstorming and generating initial design prompts. By feeding basic requirements and user journeys into the AI, I can quickly generate a list of potential features and pain points. For example, when working on a new ETL tool, Claude helped identify potential difficulties in data mapping, handling large datasets, and ensuring data accuracy during the transformation process. These pain points helped Claude generate a list of requirements and user journeys, which were then used to create a first-pass prototype 

This initial step is crucial as it sets the foundation for the entire design process. 

Rapid Prototyping

Once the ideation phase is complete, the next step is creating a first-pass prototype. Claude has helped me here by generating quick, functional prototypes that provide a visual representation of the overall application. Although not fully functional, these prototypes nevertheless offer a solid starting point for further refinement. This rapid prototyping capability allows me to iterate quickly and incorporate feedback more efficiently. 

After the initial prototype is created, I import it into Figma for refinement. This is where the design gets polished with logos, color schemes, and other branding elements. This is a highly collaborative phase of the process, where designers work closely with developers and test users to finalize the look and feel of the application. This step ensures that the design is not only functional but also visually appealing. 

Code Development

The final stage involves turning the refined design into a working application. Here, remarkably, AI tools like Claude and Cursor (an AI-enhanced version of VS Code) can actually generate and refine the code itself. By providing the AI with an image of the final design, it can produce a close approximation of the user interface, which can then be fine-tuned by developers. For example, I might ask Claude to generate a sample layout based on the refined design and then use Cursor to make specific changes, such as adjusting font sizes and colors. This significantly reduces the time and effort required to build the front end of the application. 

Real-World Application and Testing — Collaboration and Continuous Improvement

The iterative nature of AI tools allows for rapid prototyping and testing, leading to a more efficient development cycle. While AI-generated code might not be perfect, the ability to quickly identify and fix bugs makes the process much faster than traditional methods. For instance, I used Cursor to highlight and fix errors in the code by simply providing and asking it to correct the issues. 

But collaboration remains supremely important. AI tools facilitate cross-functional teamwork by making it easier to share prototypes and gather feedback. This collaborative approach ensures that the final product meets the needs of all stakeholders. Additionally, the iterative nature of AI tools means that the design can continuously evolve based on user feedback and testing. 

—————- 

AI is not just a tool for speeding up the UX design process; it’s a catalyst for innovation and collaboration. By leveraging AI for ideation, prototyping, and code development, designers can create smarter, more efficient workflows that lead to better user experiences. The future of UX design is not just about working faster but also about working smarter. 

Using LLMs as judges for validating deal cash flow models: A new frontier in securitization modeling

As securitization models become increasingly complex and differentiated, validation becomes a critical challenge. We’ve experimented with an innovative approach that leverages large language models (LLMs) as impartial judges to validate models implemented across different platforms.

The Dual-Implementation Challenge

In cash flow modeling, we often maintain parallel implementations—typically in Python for flexibility and Excel for transparency. How do we ensure both versions produce consistent results?

Enter the “LLM as Judge” approach!

A Real-World Case Study: Residential Transition Loan Funding

Consider a portfolio of residential transition loans with a funding structure including:

  • 100 loans averaging $275,000 each
  • 12-month average terms at 8.75%
  • A 75% advance rate
  • 2% loss reserve build-up
  • Performance triggers based on delinquency rates

We implemented this structure in both Python and Excel, then submitted both models to an LLM for validation.

The LLM Validation Process

The LLM first analyzed the conceptual alignment between models, confirming both followed the same fundamental approach to cash flow projection, default assumptions, reserve mechanics, and triggers.

Next came a rigorous numerical comparison. The LLM detected a $100,000 investor distribution discrepancy in Month 2:

  • Python model: $1,790,702
  • Excel model: $1,690,702

Through logical analysis, the LLM determined this likely stemmed from differently evaluated trigger conditions. This kind of subtle implementation difference could easily go unnoticed in manual validation, potentially leading to significant valuation discrepancies over time.

Beyond Discrepancy Detection

The true power of this approach extends beyond finding differences. The LLM also provided:

  1. Stress testing recommendations tailored to our specific product, including scenarios for rapid defaults, extension waves, and interest rate shocks
  2. Model risk management insights highlighting documentation needs and suggesting a formal reconciliation process
  3. Code quality assessment noting strengths and weaknesses in both implementations

Why This Matters

For securitization professionals, this approach offers several advantages:

  • Efficiency: Automation of tedious line-by-line comparisons
  • Comprehensiveness: Identification of conceptual differences, not just numerical ones
  • Regulatory compliance: Better documentation for model risk management requirements
  • Objectivity: Unbiased third-party perspective

Contact us to discuss.

Mounting Pressure in Non-QM Credit: What March 2025 Data Signals for Risk Managers

This is a monthly update on non-QM delinquency rate and roll rate trends based on the March 2025 remittance data. Similar to last month’s post, I use the CoreLogic Non-Agency loan data to split out the Non-QM population by loan type. I compare the relative delinquency performance of mortgages backed by Investor properties vs. loans with full documentation vs. other Non-QM loan types (this last bucket comprises mainly Bank Statement loans). I use a slightly revised and more inclusive definition of Non-QM this month so the overall balance figures are higher and delinquency rates and roll rates are slightly lower than those reported in last month’s post.

The first chart shows that the non-performing delinquency rate (60+ dpd loans as a percentage of the overall population) has risen from a post-COVID low of 0.85% in July 2022 to 3.09% as for the most recent remit month. This increase has been driven by deterioration in the credit performance across all Non-QM loan types. Notably, the delinquency rate for Investor loans increased to 3.56% as of March, up more than three-fold from post-COVID lows of 1.1% in October 2022 and up 91bp year over year. Full Doc Non-QM loans continue to outperform other segments significantly, but their delinquency rates still rose to 0.85%, a new post COVID recovery high.

The other driver of the increase in delinquency rates for the Aggregate Non-QM loan population is a gradual shift in their mix away from the Full Doc loans, which have a better credit profile. As shown in the graph below, Full Doc loans as a percentage of the overall NQM mix have fallen from over 50% of NQM population as of the end of 2018 to only just under 31% in March. Meanwhile, Investor loans have increased from only 3% of the Non-QM population as of the end of 2018 to 10% just before COVID to over 24% as of March.

The last graph considers the gateway transition of mortgages to non-performing status: the current to 30 roll rates, or the percentage of current loans that roll to 30 days delinquent in any given month. These trends are broadly in line with what we see for the overall delinquency rates: roll rates have increased significantly since their post COVID lows.

In the March remittance data, overall Non-QM C->30 roll rates increased to 1.18%, their highest level since December 2020. All 3 non-QM segments broken out in this graph also hit new post COVID highs, with Investor-backed loans experiencing a 1.43% C->30 roll rate, 2.9x the 0.50% roll rate experienced by Full Doc Non-QM loans.

As non-QM mortgages show signs of growing distress amid broader economic uncertainty, we recommend heightened vigilance for investors and risk managers with Non-QM exposure in their portfolios. RiskSpan’s credit models forecast delinquency roll rates directly, and our modeling team calibrates our suite of models to capture both the overall trends and the differentiated performance across loan and product types. These models are just one component of our scaled analytics solutions to help our clients evaluate risk and make investment decisions.

Contact me to discuss.

From AI Hype to Helpful Assistant: AI Agents are coming soon to the RiskSpan Platform!

When agentic AI first hit the scene, we were intrigued—but skeptical. Was this just another over-hyped trend or something that could drive real value?

Fast forward a few months, and we’ve got our answer.

At RiskSpan, we’ve quietly integrated AI agents into our internal workflows through a dedicated Agent Desktop. These agents are now core to how we manage our business—monitoring client health, tracking system usage and perhaps most impressively, performing deep research across the massive datasets we store. What began as an experiment has become indispensable.

The real breakthrough is manifest, however, when Agents proactively uncover insights, flag anomalies, and automate routine analyses. Our Dev and Client teams are saving hours and making faster, more informed decisions because relevant information finds them.

Coming soon, our clients will be able to use Agents in the RiskSpan Platform to query their own data, analyze GSE performance data and run on-demand analysis instantly—all without waiting on a queue or building custom reports.

Designed for portfolio risk, surveillance, analyzing loan-level data, or exploring market trends, the AI agents will help you go from question to answer in seconds. Check out a sample below and reach out to learn more!

Non-QM Credit Stress by the Numbers: Investor and Full Doc Loan Performance Diverge

This is a follow-up to Bernadette Kogler’s short piece last month on stress in the Non-QM mortgage market. In this post, I use the CoreLogic Non-Agency loan data to split out the Non-QM population by loan type and look at the relative delinquency performance of mortgages backed by Investor properties vs. loans with full documentation vs. other Non-QM loan types (this last bucket comprises mainly Bank Statement loans).

As the following chart illustrates, the non-performing delinquency rate (60+ dpd loans as a percentage of the overall population) has risen from a post-COVID low of 1.01% to 3.59% as of March 2025. This increase has been driven by deterioration in the credit performance across all Non-QM loan types. Notably, the delinquency rate for Investor loans increased to 3.82% as of March, up more than three-fold from post-COVID lows of 1.1% in October 2022. While they remain the best performing loan type, even the Full Doc loans have seen a doubling of delinquency rate, to 1.11%.

The other driver of the sharp uptick in delinquency rates for the Aggregate Non-QM loan population is a shift in their mix away from the strongly performing Full Doc loans. As shown in the graph below, Full Doc loans as a percentage of the overall NQM mix have fallen from over 50% of NQM population as of the end of 2018 to only 22% in March. Meanwhile, Investor loans have increased from only 3% of the Non-QM population as of the end of 2018 to 10% just before COVID to 28% as of March.

Finally, we look at the gateway transition of mortgages to non-performing status: the current to 30 roll rates, or the percentage of current loans that roll to 30 days delinquent in any given month due to a missed payment. Not surprisingly, these trends are broadly in line with what we see for the overall delinquency rates: roll rates have increased significantly since their late 2022 lows.

But these roll rates give us a more real-time perspective on how different loan types are performing relative to each other than the delinquency rate levels, which represent the cumulative effect of historical performance. In the most recent remittance data, Investor-backed loans experienced a 1.42% C->30 roll rate, which was 2.5x the 0.58% roll rate experienced by Full Doc Non-QM loans. By contrast, that multiple was only 1.8x in October 2022 when NQM loans were experiencing their lowest post-COVID roll rate performance.

Given the deteriorating performance of Non-QM mortgages and backdrop of macroeconomic uncertainty, it is important for investors to monitor their portfolios that have Non-QM exposure. Our credit models at RiskSpan model these delinquency roll rates directly, and our modeling team calibrates our suite of models to capture both the overall trends and the differentiated performance across loan and product types. These models are just one component of our scaled analytics solutions to help our clients evaluate risk and make investment decisions.

Contact me to discuss.

The Future of Private Credit: Growth Challenges, and How RiskSpan is Leading the Way

Private credit is having a moment, as they say, now approaching $7 trillion in global assets, and is poised to double in size over the next decade. As traditional banks tighten lending due to regulatory constraints, private credit is stepping in to provide flexible, high-yield investment opportunities for institutional investors. However, this expanding market brings challenges, including illiquidity, bespoke deal structures, and complex risk assessments.

Chartis Research, in collaboration with RiskSpan, explores these evolving dynamics in a recent report, shedding light on the forces shaping private credit’s expansion and the critical role of technology in mitigating risk.

As private credit markets grow, effective risk management is crucial for investors seeking stable returns. Advanced technologies like AI and machine learning are revolutionizing private credit risk assessment, enhancing cash flow modeling, pricing accuracy, and portfolio diversification. RiskSpan leads the industry with innovative solutions, leveraging loan-level data and cloud-based platforms to provide real-time analytics. Whether you’re an asset manager, institutional investor, or lender, understanding the latest private credit trends is essential for success.

Read the full article to explore how private credit is transforming finance and why technology-driven risk management is the key to sustainable growth.

Contact us to learn more about how RiskSpan’s platform can support your private credit analytics.

Non-QM Delinquencies Are Rising—And Home Prices Aren’t Helping 📉

The non-QM mortgage market is showing clear signs of stress, and the latest delinquency data confirms it. RiskSpan analysis shows 60+ day delinquencies are rising, with 2022 and 2023 vintages deteriorating faster than prior years. Non-Qualified Mortgages (Non-QM) are loans that don’t meet traditional underwriting guidelines and often include self-employed borrowers, investors, and those with alternative income documentation.

What’s Driving the Spike?

Sustained higher mortgage rates have created pressure for some non-QM borrowers with fewer refinancing options. A more granular analysis shows loan attributes and risk layering driving high delinquencies, particularly those with cashout refi as the loan purpose. In a slowing home price appreciation (HPA) environment, borrowers who took out cash-out refis may be struggling with payment shock and limited home equity growth.

But the Real Problem? The Changing Housing Market.

Since the Covid crisis, many believed low housing inventory would keep prices elevated, but not anymore. The Wall Street Journal reported last week that housing inventory rose by 16% compared to the previous year. Further, the Federal Reserve Economic Data (FRED) shows 2024 HPA at just +3.5%, the slowest since 2020 with certain MSAs declining.  Florida remains its own unique case, while DC faces recession fears following recent Trump policy changes. 2025 looks even weaker – WS research projects HPA at just +2.5%, signaling even slower home price growth ahead.

The Risk: What Comes Next?

Slower home price growth means reduced equity cushions and borrowers with less ability to absorb financial shocks. This means refinancing and selling become less viable options leading to rising delinquencies & liquidity concerns. The markets could certainly stabilize or non-QM delinquencies could continue their upward climb.

Company

Products

Security & Compliance

Amazon Web Services
Microsoft

Linkedin

SOC2 Type II Certified

Get Started
Log in

Linkedin   

risktech2024