Resi Whole Loans: Why Banks, Insurers, and Investors Are Rethinking Mortgage Credit Risk 

Off The Curve, Episode 4 — with James M. Deitch, CMB, and RiskSpan CEO Bernadette Kogler 

🎙️ Listen to the episode on Spotify and Apple Podcasts

Life insurers have nearly quadrupled their investments in residential whole loans over the past few years, riding the boom in annuity sales. Most banks have watched from the sidelines. I spend my days talking to both groups, and the question keeps coming up: who’s actually holding mortgage credit risk today, and who should be? 

So for this episode of Off The Curve, Bernadette Kogler and I sat down with Jim Deitch, a banker, fintech entrepreneur, and best-selling author who has built and sold more companies than most of us have worked at. Jim advises banks on exactly these calls, and he’s run them himself, so I was glad to finally get his take on the record. Here’s what stuck with me. 

Insurers found the duration they were looking for 

The insurer story is mostly about matching liabilities. Annuity writers need assets in the seven-to-15-year duration range, and residential whole loans fit that window well. They’re also happy to pick up a little illiquidity premium in non-QM rather than stick to agency or top-tier jumbo borrowers, which is exactly what I see in the bids. With commercial real estate no longer the obvious home for that money, resi has filled the gap. 

Banks are in a different spot. Jim noted that smaller banks, those under roughly $100 billion in assets, are far more loaned up than the large ones. Add in lingering memories of the financial crisis and regulators’ long-standing discomfort with mortgage assets, and you can see why bank portfolios haven’t grown. Still, Jim sees a real opening: a portfolio loan can deepen a customer relationship and bring in other business, especially wealth management assets. 

The insurer bid matters for banks, too. In March 2020, non-QM securitization froze overnight and loans got stuck on warehouse lines, and nobody who lived through those phone calls has forgotten it. A deep whole loan buyer base with a low cost of capital makes a repeat of that freeze less likely. 

The G-fee math is getting hard to ignore 

This was the question I came in wanting to ask. Back in the day, the G-fee was around 25 basis points before negotiation. Jim now puts it at about 65, including amortized LLPAs. Meanwhile, Fannie Mae’s credit losses last year came in under two basis points. 

For a bank with room on its balance sheet and enough capital, Jim thinks that adds up to a case for holding loans rather than selling them. Whole loans also aren’t marked to market, so there’s no OCI hit to manage. The fit depends on size, though. A community bank under $10 billion with a high loan-to-deposit ratio will struggle to carry a 30-year fixed asset. ARMs help with duration, and Jim is hearing real interest in near-miss jumbos that banks can’t easily sell anyway. 

Bernadette pushed on a hypothesis I share: with rates around 7%, is this the time to load up? Jim said yes, as long as you have the capital and liquidity to hold the loans. Maybe we see 10%, but we’re not going back to 2% anytime soon. And if we ever did, a book of 7% loans would look awfully good. 

The warehouse risk sits in the bottom quintile 

Jim pointed to Mortgage Bankers Association data on independent mortgage banker profitability for the second quarter of 2026. The top 20% of IMBs earned about 120 basis points. The average was 25 basis points, and the bottom 20% lost 76. That’s a gap of roughly 200 basis points, and someone is still providing warehouse lines to the lenders at the bottom. 

If stress shows up, Jim expects it there first. Many of those lenders are likely already outside their warehouse covenants, and the open question is how many quarters a bank tolerates that. He also worries that when lenders are struggling just to make ends meet, the temptation to stretch can turn into fraud. In his view, the market hasn’t become as selective on haircuts and pricing as it should be. 

This lines up with what I’m hearing from the warehouse lenders we talk to. They’ve been perfectly happy lending against agency collateral, but they’re getting dragged into non-QM just to stay competitive. That’s especially true for banks with private equity owners who need margin, because agency warehouse spreads are razor thin. 

The least-tested risk is the servicer 

When Bernadette asked Jim what advice he’d give investors considering this asset class, his answer was discernment. You need to know the borrower, the originator and whether it can stand behind its loans, and how regional home prices are behaving. But he said the servicer matters more than most people think. Early detection and early intervention have an outsized effect on what a delinquency ultimately costs. 

The catch is that this capability hasn’t been tested. Delinquencies have been low for years, and most crisis-era loans have long since been resolved, so many servicers have scaled their loss mitigation teams way down. When early delinquencies start to rise, can they ramp back up fast enough to get ahead of it? Nobody really knows yet. 

Home prices are the other untested piece. Jim lives in Naples, where prices are flattening, and he noted that Cape Coral, about 30 miles away, has already seen real declines. Strong appreciation has kept loan performance clean because a struggling borrower could always sell. We haven’t had a meaningful drop in home values since before COVID, and loans without a guarantee or mortgage insurance would feel one first. 

My takeaway 

Jim likes residential whole loans, and so do I. The credit quality on recent vintages is strong, and he’s been pleasantly surprised by how well-underwritten bank portfolios have performed. We’re already seeing some non-bank investors price through the GSEs. Securitization is heating up again, too: I read Asset-Backed Alert at 6 a.m. every Friday, and non-QM deals are coming back fast. For banks with the balance sheet room, today’s rates and G-fee economics make a real case for keeping loans. 

But the old lessons from the crisis still apply. Know your originator, because they aren’t all the same. Know your servicer, and how it will handle delinquencies when they come. And whatever the structure, know what’s actually in the collateral, loan by loan. Plenty of insurers have figured this out by owning originators and servicers outright. Everyone else has to get that visibility another way, and that’s the conversation I have with clients every week at RiskSpan. 

Thanks to Jim for a candid and fun conversation, and to Bernadette for steering it. We’ll be watching to see whether servicers and home prices get their first real test.