FILE SELECTED // FILE 00
Multiple Comparisons
“The more doors you try, the more likely one opens by chance alone.”
01 / SUMMARY
In short
Testing many hypotheses raises the chance of at least one extreme result under the nulls. Procedures such as Bonferroni, Tukey, or false-discovery control serve different goals.
03 / SCENE
In the wild
With twenty independent tests at the 0.05 level, the chance of at least one false positive is far above 5%.
04 / LIMIT
The limit
There is no universal correction; controlling a family of errors differs from controlling the expected false-discovery proportion.
FINAL NOTE
What remains
Count how many opportunities the analysis had to find something striking.