Y11W15VC The bullet holes that weren't there

During the Second World War, American planes were returning from raids over Europe with bullet holes in specific patterns. The military wanted to armour the places where the planes had been hit. A statistician named Abraham Wald suggested they armour the places where the returning planes hadn't been hit. This week's article examines why — and what the principle reveals about how we learn from visible success.

Core Vocabulary

bias

/ˈbaɪəs/|bi·as

noun

A systematic distortion in thinking or data collection that consistently pushes results in one direction; a tendency to favour certain outcomes over others in a predictable way.

Word family: biased (adj.), biases (n. plural), unbiased (antonym)

Synonyms: systematic error, distortion, skew

Collocations: systematic bias, cognitive bias, confirmation bias

Example: Every research method has potential sources of bias that must be identified and controlled.

In the articleIt also became one of the founding examples of a phenomenon now called survivorship bias — the systematic error of drawing conclusions from data that was filtered by the very thing you're trying to study.

distorted

/dɪˈstɔːtɪd/|dis·tort·ed

adjective

Twisted out of its original or correct form; altered so as to be misleading or inaccurate.

Word Breakdown: dis- (prefix "apart") + Latin *torquere* "to twist"

Word family: distort (v.), distortion (n.), distortedly (adv.)

Synonyms: twisted, warped, misrepresented

Collocations: distorted image, distorted view, distorted reasoning

Example: The data was distorted because it only included survivors, leaving out the evidence of failures.

conspicuous

/kənˈspɪkjʊəs/|con·spic·u·ous

adjective

Clearly visible and attracting attention; standing out prominently from the surroundings.

Word Breakdown: con- (prefix "together") + Latin *specere* "to look"

Word family: conspicuously (adv.), conspicuousness (n.)

Synonyms: obvious, visible, prominent

Collocations: conspicuous damage, conspicuous absence, conspicuous success

Example: The bullet holes were conspicuous on the returned bombers — and that made them misleading.

absence

/ˈæbsəns/|ab·sence

noun

The state of not being present; a non-existence or lack of something in a place where it might be expected.

Word family: absent (adj.), absentee (n.)

Synonyms: non-presence, lack, omission

Collocations: conspicuous absence, notable absence, absence of evidence

Example: The absence of the planes that never returned was the most important data the analysts were missing.

In the articleTaleb's silent evidence The writer Nassim Taleb, in his book The Black Swan, extended the survivorship-bias idea into what he called silent evidence — the evidence that doesn't survive to be counted, and that therefore shapes our understanding of the world by its absence.

extrapolate

/ɪkˈstræpəleɪt/|ex·trap·o·late

verb

To extend or project conclusions from known cases to new or broader ones; to estimate beyond the known data.

Word Breakdown: extra- (prefix "outside") + -polate (from Latin *polire* "to smooth, extend")

Word family: extrapolation (n.), extrapolating (v.)

Synonyms: extend, project, infer beyond the data

Collocations: extrapolate from data, extrapolate a trend, difficult to extrapolate

Example: It became misleading when analysts tried to extrapolate from success traits alone.

In the articleIt becomes misleading when you extrapolate from success traits to claims about probability, causation or generalisability.

prevailing

/prɪˈveɪlɪŋ/|pre·vail·ing

adjective

Most common, widespread, or generally accepted at a particular time; dominant in a given situation.

Word family: prevail (v.), prevalence (n.)

Synonyms: dominant, widespread, most common

Collocations: prevailing view, prevailing wind, prevailing assumption

Example: The prevailing assumption was that the bullet-hole pattern showed where planes needed reinforcement.

inherent

/ɪnˈhɪərənt/|in·her·ent

adjective

Built into the very nature of something; existing as a permanent and inseparable part of something.

Word Breakdown: in- (prefix "in") + Latin *haerere* "to stick, cling"

Word family: inherently (adv.), inherence (n.)

Synonyms: built-in, intrinsic, fundamental

Collocations: inherent flaw, inherent risk, inherent limitation

Example: Survivorship bias is not accidental — it is inherent in any dataset that only records what survived.

anomaly

/əˈnɒməli/|a·nom·a·ly

noun

Something that deviates from the normal pattern or expected rule; an irregular or unexpected occurrence.

Word Breakdown: an- (Greek prefix "without") + homos (Greek "same")

Word family: anomalous (adj.), anomalies (n. plural)

Synonyms: deviation, irregularity, exception

Collocations: statistical anomaly, spot an anomaly, unexplained anomaly

Example: Wald noticed an anomaly: the returned bombers had damage where a plane could survive, not where it could not.

Technical Terms

survivorship bias

/səˈvaɪvəʃɪp ˈbaɪəs/|sur·vi·vor·ship bi·as

noun phrase

reasoning from visible successes while overlooking the invisible failures

Synonyms: survivor fallacy, selection illusion, success visibility bias

Collocations: affected by survivorship bias, survivorship bias distorts, classic survivorship bias

Example: The business school invited only successful founders to speak — a curriculum shaped by survivorship bias that left students with no sense of the far larger population of failed ventures launched under identical conditions.

In the articleIt also became one of the founding examples of a phenomenon now called survivorship bias — the systematic error of drawing conclusions from data that was filtered by the very thing you're trying to study.

selection bias

/sɪˈlɛkʃ(ə)n ˈbaɪəs/|se·lec·tion bi·as

noun phrase

systematic error from non-random sampling

Synonyms: sampling bias, non-representative sampling, skewed sample

Collocations: introduce selection bias, selection bias in research, control for selection bias

Example: The clinical trial enrolled only patients healthy enough to travel to the hospital, introducing a selection bias that made the treatment appear far more effective than it would prove to be in the general population.

base rate

/beɪs reɪt/|base rate

noun phrase

the underlying frequency of something in the population, often ignored in reasoning

Synonyms: prior probability, background frequency, baseline rate

Collocations: ignore the base rate, base rate neglect, apply the base rate

Example: The diagnostic test was accurate 90% of the time, but because the disease affected only one in a thousand people, ignoring the base rate led doctors to dramatically overestimate the likelihood of a positive result being genuine.

counterfactual

/ˌkaʊntəˈfæktʃuəl/|coun·ter·fac·tu·al

noun / adjective

what would have happened under different conditions

Synonyms: hypothetical alternative, what-if scenario, alternative history

Collocations: construct a counterfactual, counterfactual reasoning, counterfactual world

Example: The pilot who survived credited his skill — but the counterfactual question, what would have happened to an equally skilled pilot in the same conditions, is one he cannot answer and rarely considers.

confirmation bias

/ˌkɒnfəˈmeɪʃ(ə)n ˈbaɪəs/|con·fir·ma·tion bi·as

noun phrase

the tendency to seek evidence supporting existing beliefs

Synonyms: myside bias, belief perseverance, selective evidence weighting

Collocations: susceptible to confirmation bias, confirmation bias distorts, overcome confirmation bias

Example: He read every article that supported his investment thesis and dismissed contradictory evidence as poorly researched — confirmation bias turning a sceptical market signal into something he never seriously examined.

Figurative Phrases

below the radar

not visible to observation

Etymology/Type: idiom from military detection

Synonyms: undetected, not attracting notice, outside awareness

Example: The organisational dysfunction had operated below the radar for years — visible to those inside the system but never surfacing in the metrics that senior leadership actually reviewed.

tell the tale

reveal the truth

Etymology/Type: idiomatic; 'tale' carries weight beyond 'story'

Synonyms: survive to report, be evidence of success, indicate what happened

Example: Only the companies that pivoted successfully are here to tell the tale — the ones that did not are absent from the case studies in a way that makes the curriculum misleadingly optimistic.

come back to haunt

affect later in negative ways

Etymology/Type: idiom with figurative 'haunt'

Synonyms: cause future problems, return as a consequence, resurface as a difficulty

Example: The decision to cut corners on the initial launch came back to haunt the company three years later, when the cumulative technical debt became impossible to service without a full rebuild.

the survivors' stories

accounts from those who made it

Etymology/Type: figurative, implies unseen others

Synonyms: accounts from those who succeeded, testimony of the remaining cases, visible success narratives

Example: The survivors' stories are compelling, but they are also systematically misleading — because for every founder whose persistence paid off, many more persisted just as long and failed just as completely.

In the articleThe books tell us about the survivors.

sample of one

inadequate evidence

Etymology/Type: idiom; statistical shorthand for unreliability

Synonyms: a single case, an anecdote rather than data, an unrepresentative example

Example: Her grandfather smoked until ninety and never developed cancer — but a sample of one tells us nothing about the population-level risk, and treating it as evidence is a classic survivorship error.

In the articleThe rest of the information, the fatal hits, had been silently removed from the sample by the data-collection process itself.

shine a light on

reveal or expose

Etymology/Type: metaphor; no literal illumination

Synonyms: reveal, expose, draw attention to

Example: The investigation aimed to shine a light on the selection bias embedded in industry awards — pointing out that entries were self-submitted and that unsuccessful companies simply did not enter.

Confusing Words

bias vs prejudice

Both words describe systematic distortions in judgement, but they differ in the degree of awareness, intentionality, and emotional loading involved.

  • bias — a systematic tendency to favour one outcome, direction, or group over others, which may operate without awareness or intent. A biased sample is not a dishonest one; a researcher can introduce bias through design choices they did not recognise as skewed. Bias is broader and more neutral in its connotations.
  • prejudice — a preconceived judgement or attitude, typically negative, toward a group or individual, formed without adequate evidence. Prejudice carries stronger emotional loading and moral weight than bias — it implies not just skewed judgement but an entrenched, often unfair, disposition that resists correction by evidence.

If describing a systematic distortion that may be unintentional and value-neutral, use bias. If describing an entrenched preconceived attitude with moral or emotional weight, use prejudice.

extrapolate vs interpolate

Both words describe drawing inferences from existing data, but they differ in the direction of that inference relative to the available evidence.

  • extrapolate — to project beyond the range of known data, using existing patterns to estimate values or outcomes outside the observed range. To extrapolate from a trend is to extend it into unknown territory. The further the extrapolation, the less reliable it becomes.
  • interpolate — to estimate values within the known range of data, filling in between observed points. Interpolation operates within the existing evidence and is generally more reliable than extrapolation because it does not venture beyond the established boundaries.

If projecting beyond the known range of evidence into new territory, use extrapolate. If estimating within the existing range of known data, use interpolate.

anomaly vs outlier

Both words describe something that departs from the expected pattern, but they differ in their quantitative precision and the contexts in which they are used.

  • anomaly — something that deviates from what is expected, normal, or typical in a way that is notable and may be significant. An anomaly invites explanation: it could signal an error, a new pattern, or something genuinely unusual. The word is used broadly in both formal and informal contexts.
  • outlier — a data point that lies far from the bulk of a dataset in a measurable, often statistical, sense. Outliers are identified numerically and may be removed from analysis, investigated separately, or treated as signals. The word belongs primarily to statistical and quantitative contexts.

If describing something that deviates from expectation in a general sense, use anomaly. If referring specifically to a data point that falls far outside the statistical distribution, use outlier.