The history of American mortgage lending in pretty colors

Who dominates the U.S. mortgage market? Well, depends when you're asking. I spent some time today playing with the Federal Reserve's Flow of Funds data on home mortgage lending. Here's what I found. First, the long-term picture:

Graphic by Feilding Cage/TIME.com

As you can see, thrifts (savings & loan companies and savings banks) were the biggest mortgage lenders until the early 1980s, when their troubles paved the way for the rise of Fannie Mae and Freddie Mac (a.k.a. the government-sponsored enterprises, or GSEs). Fannie and Freddie remained dominant until about five years ago. For a better view of that, here's the picture over the past decade:

Graphic by Feilding Cage/TIME.com

The quarterly data are a little noisy, I realize (it's actually significantly more work to get the annual numbers, and I don't have time for that right now). That huge mirror movement--banks up, thrifts down--in the fourth quarter of 2006, for example, was the result of several thrifts becoming banks. (For example: Wachovia swallowed up Golden West that quarter and switched it to a bank charter.) But the basic picture is pretty clear: Fannie and Freddie have dominated U.S. mortgage lending since the early 1980s--except from 2004 through 2006, when the asset-backed securities issuers, a.k.a. Wall Street, took over. And that's when the craziest excesses of the mortgage boom happened.

The thing that's most amazing in retrospect is the fact that alarm bells didn't go off all over the place when private securitizers began muscling Fannie and Freddie (and the FHA) aside in 2004. Because of their formerly implicit government guarantee, the GSEs can usually easily outbid their private ABS competitors for mortgages. That's always been the complaint--that the guarantee makes it impossible for truly private companies to compete against them. So when a bunch of private companies were suddenly able to steal market share from the GSEs right and left, shouldn't everybody have been able to sense that something was terribly wrong in mortgageland?