Conditional probability theorem
Bayes' Theorem
Evidence updates a prior belief by comparing how expected that evidence is under competing possibilities.
P(H | E) = P(E | H) P(H) / P(E)
The posterior probability of hypothesis H after evidence E depends on the prior probability of H, the likelihood of E if H is true, and the total probability of E.
The population lab begins with 90% sensitivity and a 5% false-positive rate, then lets both vary. Its 10,000-person counts make base rates and the denominator explicit.
(%)
Move the prior prevalence above. The test itself can stay unchanged while the meaning of a positive result changes dramatically.
- CHANGE
- Prior probability
- WATCH
- posterior
- MEANING
- The population lab begins with 90% sensitivity and a 5% false-positive rate, then lets both vary. Its 10,000-person counts make base rates and the denominator explicit.
A positive signal is not the same as the condition.
When the prior is low, false positives can be numerous even with a good test. Bayes makes the denominator visible.
What it actually says
Bayes' theorem is a rule for reversing conditional probability. It lets us move from the likelihood of evidence given a hypothesis to the probability of the hypothesis given the evidence.
The theorem is formal and exact under the probability model. The hard work is not the algebra; it is specifying the hypotheses, priors, likelihoods, and evidence without smuggling in bad assumptions.
"A useful law compresses a pattern. It does not erase the conditions that make the pattern true."
How the idea developed
The modern form emerged through observation, argument, and later refinement. The timeline separates the first insight from the version now used in textbooks and practice.[1]
Richard Price publishes Thomas Bayes' posthumous essay on inverse probability.
Laplace independently develops and extends Bayesian inverse probability.
Frequentist and Bayesian schools debate interpretation, inference, and objectivity.
Bayesian methods power diagnostics, forecasting, machine learning, and scientific modeling.
How the pattern works
The relation becomes useful only when its mechanism, measurement process, and operating range are visible.
Start with a probability before seeing the new evidence.
Ask how probable the evidence would be if each hypothesis were true.
Divide by the total probability of the evidence across possibilities.
The posterior probability of hypothesis H after evidence E depends on the prior probability of H, the likelihood of E if H is true, and the total probability of E.
Where it earns its keep
Applications are strongest when the law changes a decision, measurement, model, or experiment rather than merely providing an analogy.
Diagnostic interpretation
ApplicationA positive test can mean very different things in low-prevalence and high-prevalence populations.
Sensitivity and specificity are not enough; prevalence matters.
Signal triage
ApplicationRare threats create many false alarms unless base rates and costs are modeled explicitly.
Prior odds prevent alert systems from becoming superstition machines.
Model comparison
ApplicationEvidence shifts support toward models that predicted it better than their rivals.
Updating is only as good as the hypothesis space.
Where it stops working
Bayes' theorem does not tell you which prior to choose, whether your likelihood model is true, or whether your hypotheses are exhaustive. It is a valid updating rule, not an automatic truth engine.
In messy domains, dependence between signals, selection bias, measurement error, and changing base rates can dominate the neat formula.
"A 95% accurate test means 95% chance I have it"
Better: That ignores prevalence and false positives."Bayesian means subjective guessing"
Better: Priors can be subjective, empirical, hierarchical, or chosen by design; the theorem itself is mathematical."Just update forever"
Better: Bad models can update confidently toward wrong answers."P(E) is a technical nuisance"
Better: The denominator is exactly where alternative explanations compete.Sources and further reading
Original publications and serious secondary scholarship are prioritized over summaries.
- Bayes and Price - An Essay towards solving a Problem in the Doctrine of ChancesThe 1763 publication associated with Bayes' theorem.https://doi.org/10.1098/rstl.1763.0053
- Stanford Encyclopedia of Philosophy - Bayesian EpistemologyPhilosophical and methodological context for Bayesian updating.https://plato.stanford.edu/entries/epistemology-bayesian/
- NIST/SEMATECH e-Handbook - Bayes' theoremApplied statistical reference for Bayes' theorem.https://www.itl.nist.gov/div898/handbook/eda/section3/eda366i.htm
- McGrayne - The Theory That Would Not DieHistorical account of Bayesian methods and controversies.https://yalebooks.yale.edu/book/9780300188226/the-theory-that-would-not-die/