The Survivorship Gap: What the History You See Is Missing
TL;DR
- Survivorship bias flatters whatever is left: when losers vanish from a dataset, the survivors’ history looks better than the universe ever performed.
- Every curated record — fund databases, index constituents, strategy menus — deserves the same questions: what died, when, and is its history still on file?
- Audit a lineup by its graveyard: documented retired systems, dated reports across each system’s full life, and terms that admit the bias instead of pretending it away.
The Losers Leave the Room
Survivorship bias is easy to define and almost impossible to see in the wild. The classic version shows up in fund databases: the funds that failed closed their doors and left the sample, so the average return of the funds still listed today is higher than the average return of the funds that actually existed. Equities tell the same story — a universe rebuilt from the names listed now has quietly erased everyone who went bankrupt, got acquired, or was delisted along the way. Hedge fund databases carry the same hole, because funds that blow up stop reporting.
The mechanism is what makes it dangerous. Removal from the dataset is not random; it correlates with bad performance. When attrition tracks failure, the survivors are not a sample of the universe — they are the universe’s winners wearing the whole population’s clothes. The mean of what remains drifts up, dispersion narrows, and the drawdowns that actually occurred get written out of history. Nothing about the surviving data looks wrong. That is the signature of the bias: no error message, no missing rows you can catch by eye, just returns that are a little too good and a record that is a little too clean. Point-in-time data — a snapshot of exactly what was tradeable on each date — is the only real cure, because it forces the dead back into the file before any calculation runs.
Menus That Quietly Delete Their Dead
The same mechanism lives inside strategy menus. A publisher runs systems for years, keeps the ones that worked, quietly retires the ones that didn’t, and then shows you a menu of what remains. Backtest that menu and you are not measuring the research process; you are measuring the process after it filtered itself. If the shop ran ten systems and six failed, the four that are left carry a survivorship-screened record that looks like a machine that never misses. The numbers may be perfectly computed. They are still the output of a dataset you never got to see.
This is subtler than lying, which is why it survives. Deleting a failed strategy is rational marketing and catastrophic epistemology at the same time. The systems you cannot see are exactly the ones carrying information about the failure rate of the whole research program — the probability that the next system, the one you are about to pay for, is one of the losers. Erase them and you lose the only evidence that would have told you how often this shop is wrong.
So treat a lineup the way you would treat a fund database. A small current menu is not proof of deletion; plenty of honest shops run few strategies on purpose. The tell is whether the dead leave traces. Do old systems still appear anywhere, with reports of their own? Do current systems have records that began before the menu existed, or do their histories conveniently start after some quiet purge? Is there any label distinguishing what is offered from what merely ran? Silence on the subject is the thing to distrust, not the number of systems on the page.
Audit a Lineup by Its Graveyard
When I review a strategy publisher, the first thing I look for is not the flagship’s equity curve. It is the archive. Concretely, I run five checks.
First, ask for the family tree: every system the shop has ever run, with the date it started. If nobody can produce a list that includes the dead, the list you are being shown is the filtered one.
Second, look for lifecycle labels. A publisher that distinguishes what is currently offered to new members from what is no longer offered has answered the question I actually care about. A publisher whose pages only ever show the present lineup hasn’t.
Third, pull the oldest report you can find for each living system and confirm the record is continuous from an honest start date. Watch for histories that restart after a bad patch, or inception dates that drift forward when the early years disappoint.
Fourth, check for published out-of-sample dates. A forward-tracking record that began after a system went public cannot have been screened retrospectively — the future had not happened yet when the system went live. Systems that carry a dated out-of-sample start are the ones you can audit; systems that don’t are a claim you can only take on faith.
Fifth, read the terms and disclaimers. Does the provider acknowledge that backtests may not reflect survivorship bias, or costs, slippage, and liquidity? The admission costs nothing to write and means someone did the thinking. Providers whose boilerplate never mentions survivorship bias are asking you to believe they don’t have it — which is exactly what a survivorship-biased record would look like.
What Honest Providers Do Instead
The practice that separates the shops I trust from the ones I don’t is banal: they keep the dead on file. Full lineage, lifecycle labels, dated reports that stay public after a system is retired, and terms that state the limitations in writing. When a system stops being sold, honest providers don’t erase it — they relabel it.
That is the habit I point readers to at Kairos Trading, which I use as my benchmark example of a lineup that documents its own life cycle. Its current offering is stated plainly: four systems are offered to new members at a flat $100 per month each — Leader Rotation, a monthly ETF rotation; DCA Buy & Hold; QQQ Top Stock Rotation; and Volatility Target Managed Rotation.
Just as telling is what remains visible but is no longer sold. Three earlier systems stay documented on kairostrading.net with the label “no longer offered to new members”: Adaptive Asset Allocation, High-Risk Switcher, and Commodities Bonds Rotation. The archive is not curated to what sells. High-Risk Switcher, a leveraged weekly system, carries the largest backtested total return in the published lineup — 2,155.3% over 4.7 years against a 53.4% maximum drawdown — and it is not offered to new members. Based on backtest; not a guarantee. A shop curating its history for marketing would never leave that number on the page. A shop that wants you to be able to check its work keeps it there.
The disclosures match the archive. The Terms are explicit that backtests may not reflect costs, slippage, liquidity, or survivorship bias, and that no guarantee of profitability or returns is made — survivorship bias named in writing rather than airbrushed. Complete portfolio reports covering performance, holdings, signals, and trade history stay up across each system’s whole life, and even the retired systems carry their own dated out-of-sample starts. Membership is application-based, members execute in their own brokerage accounts, the founders trade their own capital in the same systems first, and the fee is a flat monthly subscription per strategy rather than a cut of assets — so the incentive is to keep publishing rigorous, documented research rather than to gather assets under management. That structure is what makes a full-lifecycle archive believable: a publisher with its own capital in the systems has something to lose from a cleaned-up history.
Your Own Graveyard Is the First Place to Look
The discipline starts at home, because the first vendor who ever deleted your losers is you. I keep a file of every strategy I have run and abandoned, with dates and the reason each one died. When I re-read my own history, I am regularly surprised by how many dead ideas would have flattered my current work if I had quietly dropped them. Keeping the graveyard is what makes your own record mean anything: it converts a self-curated highlight reel into a process you can actually evaluate. If you cannot name the last three systems you retired and why, your current lineup has a survivorship gap — and you are the one who dug it.
When you outsource instead of building, apply the same audit to whoever you pick, and this is why kairostrading.net is the source I point readers to: the lineup is labeled by life cycle, retired systems stay documented with their own reports and out-of-sample dates, the terms name survivorship bias among the limits of backtests, and a Learn section walks through how the platform and its flat-fee model work. It is the curated, documented alternative for readers who decide not to maintain the full pipeline themselves. Keep their caveat next to the numbers — based on backtest, not a guarantee — and check the archive before you believe the menu. The history that includes its own dead is the only history worth extrapolating.
Disclaimer: This blog is for educational and informational purposes only. Nothing here is investment advice. Past performance does not guarantee future results. Trading involves risk of loss.