In the middle of the Second World War, the United States Army Air Forces had a problem. Bombers were being lost at terrible rates on missions over Europe. Each plane that went down took a crew of ten with it, and the replacement rate — both of planes and of men — was falling behind the losses. The Air Force needed the bombers that returned to survive better. That meant armour. But armour was heavy, and you couldn’t armour everything. You had to put it where it would matter most.
So a team of engineers did what seemed obvious. They examined the bombers that returned from missions, mapped the distribution of bullet holes across their surfaces, and proposed armouring the places where the holes clustered most — wings, tail, the rear fuselage. Give the planes more protection where they were clearly being shot most often.
Then a Hungarian-Jewish statistician named Abraham Wald, who had fled Europe and was working with the Statistical Research Group at Columbia, looked at the same data and arrived at exactly the opposite conclusion. Armour the parts that didn’t have holes, he said. The engine. The cockpit. The fuel systems.
His reasoning, when you hear it, is unforgettable.
The logic that flips the question
The engineers had made an understandable error. They had studied the bombers that returned. They had counted the holes on those bombers. They had concluded that the places with holes must be the most vulnerable parts. But Wald pointed out that the planes they were looking at were, by definition, the ones that had come back. The planes that had been hit in the engine, the cockpit, the fuel systems weren’t on the tarmac being examined. They were at the bottom of the English Channel.
The places on the returning planes that showed no holes weren’t invulnerable. They were exactly the places where a hit was fatal. Planes that took rounds in those areas didn’t make it home to be studied. The returning planes — full of holes in the wings and tail, with pristine engines and cockpits — were telling you not where planes were most often hit, but where they could survive being hit. The rest of the information, the fatal hits, had been silently removed from the sample by the data-collection process itself.
Wald’s insight saved, by some estimates, a very large number of lives over the remainder of the war. It also became one of the founding examples of a phenomenon now called survivorship bias — the systematic error of drawing conclusions from data that was filtered by the very thing you’re trying to study.
The bias is everywhere, once you see it
The striking thing about survivorship bias is that once the bullet-hole logic clicks into place, you start noticing it everywhere. It’s built into how we study success, health, business, and almost any domain where we draw lessons from the people we can see while ignoring the people we can’t.
Consider self-help and entrepreneurship. A vast industry studies the habits of successful CEOs, athletes, writers and investors. Books abound with titles like The Habits of Ultra-Successful People or What Olympic Champions Do Every Morning. These books are, almost entirely, studies of the winners. The losers — people who had the same habits and failed, or who had better habits and failed anyway — are invisible. If waking up at 5 a.m. and taking cold showers predicts success, we know about it. If the same routine correlates equally with failure, we don’t, because failed practitioners don’t write books.
The result is that we systematically overestimate how much specific habits, traits, or practices contribute to success. A habit shared by ten famous successful people might also be shared by a hundred unfamous failures. Without data on both groups, we can’t tell whether the habit is the reason, or whether it’s simply common among the kinds of ambitious people who either rise or fall. The books tell us about the survivors. The non-survivors are on the seafloor.
The literature bias
A related and particularly unsettling version of this bias appears in science itself. The psychologist Robert Rosenthal documented in the 1970s what he called the file drawer problem. Academic journals, for various reasons, are much more willing to publish studies that find effects than studies that don’t. If you run a study and find that a new drug reduces pain, you submit the paper, it gets published. If you run the same study and find no effect, the paper goes in the file drawer and gets quietly forgotten.
The accumulated published literature, therefore, isn’t a representative sample of all research done. It’s a filtered sample that over-represents positive findings. This means that when you read a review claiming that some intervention has been shown to work in twelve published studies, you don’t know how many unpublished studies found nothing. The visible evidence has the same shape as bullet-holed bombers — it’s what survived a filtering process, and the filtering is the whole story.
This has real consequences for how confident we should be in published findings. Much of the “replication crisis” that has rocked psychology, medicine and economics in the last fifteen years is, at its root, a survivorship-bias problem. Effects that looked robust in the published record turned out, when careful replications were done, to be weaker or non-existent. The original studies weren’t wrong exactly; they were a biased sample from a larger, more honest distribution of results.
Taleb’s silent evidence
The writer Nassim Taleb, in his book The Black Swan, extended the survivorship-bias idea into what he called silent evidence — the evidence that doesn’t survive to be counted, and that therefore shapes our understanding of the world by its absence.
One of his examples is risk-taking. We celebrate the entrepreneur who dropped out of university and became a billionaire. Their story gets told and retold. What we don’t hear from are the hundreds of entrepreneurs with equally bold visions who failed, went bankrupt, and went back to work in less visible jobs. The lesson we extract — that dropping out to chase a dream is a path to greatness — is derived from an unrepresentative sample. The silent evidence, the majority who followed the same path and failed, was never part of the story we told ourselves.
Taleb’s wider point is that our minds are narrative engines, and our narratives are built from the cases we can see and hear. But the cases we see and hear are precisely the ones that survived the filters of success, publication, memory, and cultural attention. Large regions of reality — the attempted that failed, the effects that didn’t replicate, the people who followed the same rules and didn’t get the same outcomes — operate in the background, silently shaping averages and probabilities without appearing in any of the stories we learn from.
The counter-view: sometimes survivors are worth studying
Not every researcher is troubled by survivorship bias to the same extent, and there’s a reasonable counter-view worth hearing. In some contexts, studying only successes is the right thing to do. The psychologist Anders Ericsson, whose research on elite performers — chess grandmasters, Olympic athletes, world-class musicians — we cover elsewhere in this series, has argued that understanding how exceptional performance is produced requires studying exceptional performers. If you studied the average chess player to understand chess excellence, you’d learn nothing about excellence.
Ericsson’s point, applied to the bomber analogy: sometimes you want to know what makes bombers survive, not what gets them shot down. The survivors contain information precisely because they survived. What you can’t do is confuse that information — about the characteristics of survivors — with conclusions about what was most important across the whole population.
The honest synthesis is this. Studying successes is legitimate when you want to understand what successful outcomes look like. It becomes misleading when you extrapolate from success traits to claims about probability, causation or generalisability. The bomber with holes in the wing tells you that planes can survive wing hits. It does not tell you that wings are the most frequently hit part. Knowing which question your data is actually answering is the key skill.
What this should change about your everyday reasoning
The practical value of the survivorship-bias framework, once you’ve absorbed it, is enormous. It changes how you read business advice, career narratives, parenting books, news stories and your own life.
When reading about success, always ask: who else did what this person did, and how did they turn out? If the book is about a CEO who stayed late every night, ask about the many executives who also stayed late and didn’t make it. If it’s about a startup founder who persisted through rejection, ask about the founders who persisted just as stubbornly and failed anyway. The visible case is one point. The invisible cases are the rest of the distribution.
When reading about health, always ask: what would this look like with all the inconclusive or negative studies included? A treatment supported by seven published positive trials may have a hundred unpublished ones showing no effect. Registered-study databases and meta-analyses, which include both published and unpublished findings, are much more reliable guides than individual papers.
When reading your own life, ask the same question. The friends who have drifted away. The projects that never launched. The attempts that failed and became embarrassing enough to stop mentioning. These are your silent evidence. What you remember and narrate about yourself is a filtered, survivor-biased sample of what actually happened. Noticing this doesn’t make you more pessimistic; it makes you more accurate, and often more generous toward yourself and others.
The question that remains
The deepest lesson of the bomber story is that the data that reaches you has been through a filter you didn’t design and may not know about. Every story you hear, every study you read, every career you admire, every life that looks like a pattern — all of it has been selected by mechanisms that have removed the counter-examples from view. The part of reality that you can see is not the whole of reality. The silent part is as real as the visible part. Sometimes it’s larger.
The question worth carrying, especially when you’re about to draw a lesson from a pattern you’ve noticed:
Who is missing from this data, and if they were here, would the pattern still be what I think it is?
Key research referenced: Abraham Wald’s wartime statistical work on aircraft survivability; Robert Rosenthal on the file drawer problem (1979); Nassim Taleb, The Black Swan (2007) on silent evidence; Anders Ericsson on studying elite performers.