Correlation breakdown & tail dependence

Diversification is the one free lunch in finance, and it is priced off a single number — the correlation between assets. This lesson shows that number is not a constant, that it moves in the worst possible direction, and that the standard mathematical machinery for combining risks assumed the problem away.

Measure it in calm and in stress

Take the S&P 500 and the NASDAQ Composite over the same ten years as M3.1 — 2,513 overlapping days — and compute the correlation of daily returns three ways: over everything, over quiet days, and over turbulent ones.

   sample                            correlation      n
   ────────────────────────────────────────────────────────
   all days                             0.949       2,513
   calm days      (|r| < 1σ)            0.871       2,021
   stressed days  (|r| > 2σ)            0.985         111

The correlation you would use for portfolio construction — the full-sample 0.949 — is higher than the calm-market value you experience almost all of the time, and lower than the crisis value that determines whether you survive. It is a blend of two regimes and describes neither.

The direction is what matters. Going from calm to stressed, correlation rises from 0.871 to 0.985. Whatever diversification existed between these two indices is essentially gone on the days it would have mattered.

Two caveats, because this comparison is easy to overstate. First, US large-cap indices are close cousins, so 0.87 is already high; the effect is more dramatic across genuinely different assets. Second, conditioning on large moves mechanically inflates measured correlation even for a bivariate normal — selecting on extremes is not a neutral operation. So part of this gap is a statistical artefact. But only part: the effect survives careful treatment, it shows up across every asset pair anyone has looked at, and traders have a blunter name for it.

"In a crisis, all correlations go to one."

Why it happens

The mechanisms are not statistical, which is why they don't respect your covariance matrix:

• Common factor dominance. In calm markets, idiosyncratic news drives most of each asset's variance and the assets look different. In a crisis a single factor — risk appetite — dominates everything, and every asset becomes a proxy for it.
• Forced selling. Leverage (M0.7) and margin calls force liquidation of whatever can be sold, not what a manager wants to sell. This transmits stress into assets with no fundamental connection to the shock, correlating them by the mechanics of deleveraging.
• Liquidity withdrawal. Makers widen or leave simultaneously across markets (M1.2, as μ rises everywhere at once), so prices in unrelated assets move together simply because the same liquidity providers stepped back from all of them.

Each is a story about plumbing, and none of them appears anywhere in a correlation estimate.

Tail dependence, the right concept

Correlation is a single number summarising the whole joint distribution — mostly the middle of it, where most of the data is. What a risk manager actually wants is a tail question:

   λ  =  lim   P( X extreme  |  Y extreme )
        u→1

   "given one asset has a once-in-a-hundred-days move,
    what is the chance the other does too?"

That is tail dependence, and it is not determined by correlation. Two joint distributions can share an identical correlation and differ completely in whether their extremes arrive together.

This is where copulas come in — a copula separates the individual (marginal) behaviour of each asset from the structure binding them together, so you can fit fat tails to each asset and separately choose how their extremes co-occur. Useful, and it exposes exactly where the standard choice goes wrong:

   copula          tail dependence λ      meaning
   ──────────────────────────────────────────────────────────────
   GAUSSIAN            EXACTLY 0          joint extremes are
                                          asymptotically impossible,
                                          for ANY correlation < 1
   Student-t            > 0               extremes cluster; heavier
                                          with lower ν
   Gumbel               > 0 upper tail    asymmetric — models joint
                                          crashes or joint booms

Sit with the first row, because it is genuinely startling. Under a Gaussian copula, even at a correlation of 0.99, the probability that both assets are simultaneously extreme tends to zero as you go further into the tail. The model does not merely underestimate joint disasters. It asserts they cannot happen.

The 2008 version of this sentence

That mathematical property was not academic. David Li's 2000 paper introduced the Gaussian copula as a way to price correlated default in CDOs, and it became the market standard because it was tractable and everyone could agree on it — much as Black–Scholes became a language in M2.4.

The difference is that Black–Scholes' failure mode is visible: the volatility surface (M2.4) is the market openly disagreeing with the model, in public, every day. The Gaussian copula's failure mode was invisible, because its central error concerned an event that had not yet occurred. It said that large numbers of mortgages defaulting simultaneously was vanishingly unlikely — a senior CDO tranche could only be hit by a wave of correlated defaults, so senior tranches were rated safe.

Correlated default is precisely what happened. The model's zero tail dependence was not a rounding error; it was the assumption that the actual failure mode was impossible.

The general lesson generalises well past 2008: a model's most dangerous assumption is the one that makes your specific disaster have probability zero, because no amount of calibration to ordinary data will ever reveal it.

What to do instead

   ✓ Estimate correlations on STRESSED subsamples, not full
     history — or at minimum look at both and know the gap.
   ✓ Use a t-copula, or any structure with non-zero tail
     dependence, if you are aggregating tail risk at all.
   ✓ Stress-test with correlations set to 1. If the portfolio only
     survives because of diversification, it does not survive.
   ✓ Prefer structural diversification to statistical: assets that
     differ because of what they ARE, not because their historical
     returns happened not to line up.
   ✓ Treat a low measured correlation with suspicion in proportion
     to how much of your risk budget depends on it.

That last point is the honest summary of both this lesson and M3.4. Risk estimates are least reliable exactly where they are load-bearing. The response is not a better estimate — it is a portfolio that does not need the estimate to be right, which is M3.6.

Source: McNeil, Frey & Embrechts, Quantitative Risk Management, ch.7 on copulas and tail dependence — the definitive treatment, and clear about the Gaussian copula's defect years before it mattered publicly. Longin & Solnik, “Extreme correlation of international equity markets” (2001) for the empirics. Felix Salmon’s “Recipe for Disaster: The Formula That Killed Wall Street” (Wired, 2009) is the readable version of the Li story, and overstates it — read it after the maths, as with Flash Boys in M1.6.