In 2005, Stanford epidemiologist John Ioannidis published what would become the most cited paper in modern medical history: ‘Why Most Published Research Findings Are False.’ Ioannidis proved that the credibility of a scientific claim does not depend solely on statistical significance (p < 0.05); it is bounded by the bayesian false positive risk. When a hypothesis tests an implausible or radical idea with a low prior probability, even a formally significant test with 80% statistical power is more likely to be a false positive than a true discovery.
The Base Rate Fallacy in Modern Science
To understand the bayesian false positive risk, consider a medical screening test that is 99% accurate (1% false positive rate) testing a rare disease that afflicts 1 in 10,000 people. If an individual tests positive, the probability that they actually have the disease is not 99%, it is less than 1%. Why? Because the vast sea of healthy individuals (9,999) produces 100 false positives for every 1 true case. The base rate dominates the test accuracy.
Where (1 – beta) is statistical power, alpha is the false positive threshold (0.05), and R is the pre-study odds that the tested relationship is true (R = Pr(True) / Pr(False)). When prior odds R are low, PPV collapses toward zero.
The Epistemic Collapse of Implausible Hypotheses
The table below demonstrates how the bayesian false positive risk evolves across different scientific domains depending on the prior plausibility of the research hypothesis:
| Research Context | Prior Probability Pr(True) | Pre-Study Odds (R) | Post-Study PPV (Power=80%, alpha=0.05) |
|---|---|---|---|
| Confirmatory Clinical Trial | 50.0% | 1:1 (R = 1.0) | 94.1% (High credibility) |
| Exploratory Social Science | 10.0% | 1:9 (R = 0.11) | 64.0% (Substantial error risk) |
| High-Throughput Genome Screens | 0.1% | 1:999 (R = 0.001) | 1.6% (Over 98% false positives!) |
Extraordinary Claims Require Bayesian Invariants
Carl Sagan famously popularized Marcello Truzzi’s aphorism: ‘extraordinary claims require extraordinary evidence.’ The bayesian false positive risk is the formal mathematical proof of this principle. When a researcher tests an extraordinary hypothesis (such as precognition or cold fusion), a modest p-value of 0.04 provides almost zero evidence of truth. To move the posterior probability of an extraordinary claim from near-zero to plausible, the evidence must achieve particle physics standards: 5-sigma significance (p < 3 * 10^-7).
Continue Your Exploration
This series translates the core principles of the replication crisis, statistical power, causal DAGs, Bayesian false positive risks, and adversarial collaboration into actionable mental models. For formal mathematical proofs, empirical literature reviews, and institutional protocols, consult our foundational treatise: Replication Crisis: 5 Empirical Rules of Epistemic Independence.
Empirical Framework
In 2005, Stanford epidemiologist John Ioannidis published ‘Why Most Published Research Findings Are False’ in PLoS Medicine, currently the most cited paper in the history of the journal. Ioannidis established a rigorous Bayesian proof demonstrating that the true probability that a research finding is true, termed the Positive Predictive Value (PPV), is a strict function of three independent variables: statistical power (1 – beta), the significance threshold (alpha), and the pre-study odds that the relationship is true (R).
Ioannidis proved that when exploring novel or counterintuitive hypotheses where the pre-study odds are low (e.g. R = 0.05, representing highly speculative exploratory screening), even a nominally sound experiment with 80% power and alpha = 0.05 yields a PPV of only 45.7%. If researchers introduce modest investigator bias (u = 0.20), the PPV plummets below 25%. In modern exploratory sciences, the majority of statistically significant claims are mathematically guaranteed to be false positives unless validated by strict prior probability bounds.
The Replication Crisis & Epistemic Independence: 5 Empirical Rules
An empirical exploration through the foundational dynamics, systemic trade-offs, and practical architecture of the replication crisis & epistemic independence: 5 empirical rules.