Survivorship bias: why 51% of the companies in a backtest no longer trade

When someone shows the historical result of a strategy, the question almost nobody asks is which list of companies it was tested on. It sounds like an administrative detail and it is what decides whether the number means anything. If the list was built from the companies that exist today, the result is inflated before the test even starts — and nothing in the output tells you it happened.

What survivorship bias is

Suppose you want to test some rules on S&P 500 stocks over the last ten years. The natural thing is to download today's index list and run the test backwards. The problem is in that first step: those companies are in the index today because they did reasonably well. The ones that went bankrupt, merged, were absorbed, or simply collapsed until they fell out of the index are not on that list.

So the test never buys a company that later disappeared. In real life you would have bought it: at the time it traded, it met the rules, and it looked as promising as the rest. The result comes out better than reality, and the error leaves no trace — no warning, no flag, no number in red. Just a nicer result.

It is the equivalent of measuring how dangerous a sport is by asking only the people still playing.

How many, and how they are counted

The universes this system uses — built year by year, with the composition each index actually had at the time — contain 7,202 distinct tickers between 1995 and 2026. Of those, 3,695 no longer trade: 51%.

The number is not estimated, it is compared. NASDAQ publishes, openly and free of charge, the list of every security listed today on US markets; there are 13,262. A ticker from the historical universes that is not on that list does not trade. It can be recalculated at any time, and it should be: the number ages.

That it can be compared rather than estimated is precisely the point. There is a tempting shortcut — looking at which tickers stopped getting new prices — that does not work: a price database only updates the tickers in use, so a ticker without recent prices may be delisted or simply out of the working list. Two different things with the same symptom. Against the official list there is no ambiguity: either the security is listed today or it is not. A number you publish has to be recomputable, and this one is.

"No longer trades" is not the same as "went bankrupt"

Among those 3,695 there are real bankruptcies. AAMRQ was American Airlines in Chapter 11; the trailing Q is the marker given to companies in that process. AABA was what remained of Yahoo.

But there is something else too. AAXN is now AXON and is doing very well: only the name changed. AAWW was Atlas Air, taken private. ABC was AmerisourceBergen, renamed. None of those three failed.

For survivorship bias the distinction changes nothing: all of them vanish from a list built with today's names, which is why all of them are missing from a badly built test. But saying "half the companies went bankrupt" would be false, and that is exactly the kind of exaggeration that makes everything else you say suspect.

What to do with this when looking at someone else's backtest

You do not need to be a programmer to spot the problem. It is enough to ask:

• Which list of companies was it tested on, and from what date is that list? If the answer is "the index ones", the next question is: the index as of when.
• Are the ones that stopped trading included? If the answer is no, or "that data isn't available", the result is not comparable to actually trading.
• At what price is each trade executed? A signal computed from the close and executed at that same close uses information that, in real life, was not yet there.
• Are commissions and slippage deducted? In high-turnover strategies, that difference eats a good part of the result.

None of the four questions is technical. All four are hard to answer well if the backtest was built badly, and that is what makes them useful.

What this does not fix

Keeping the companies that disappeared makes a backtest honest. It does not make it predictive. A well-built historical result is still a measurement of the past, with rules that already knew that past when they were defined. It is useful for discarding what clearly did not work, and for knowing the shape of the declines you will have to tolerate; it is not useful for knowing what will happen.

That is why one thing is worth more than any backtest: the live result, published with its date, with its drawdowns, and with every trade visible — winners and losers alike.

Information, not investment advice. Past results, simulated or real, do not guarantee future results. Risk of loss. Not available to residents of the UK, EEA, Australia or India.