IDEA FOUND // IDEA 125
P-Values
“A p-value asks about the data under a model; it does not give the probability that a hypothesis is true.”
01 / PLAINLY
What it means, plainly
A p-value summarizes how unusual these data, or more extreme data, would be under a specified statistical model. It does not tell you the probability that a hypothesis is true.
02 / CONTEXT
A little more
A p-value indicates how incompatible the data are with a specified statistical model, counting outcomes at least as extreme. Its meaning depends on design, assumptions, and analytic choices.
03 / WHY IT MATTERS
Why it matters
A careful reading keeps one number from becoming a verdict about truth, importance, or whether a result will appear again.
04 / EXAMPLE
A familiar example
p = 0.03 does not mean there is a 3% probability that the null hypothesis is true.
05 / LIMIT
What it does not mean
It also does not measure effect size, practical importance, or probability of replication.
06 / NOTICE
Notice it in your day
Rewrite ‘p = 0.03, so the hypothesis has a 3% chance of being true’ without assigning a probability to the hypothesis, then note what information is missing.
FINAL NOTE
The idea worth keeping
Read the p-value alongside the estimate, uncertainty, design, and context.
QUESTIONS / 02
Questions people still have
Does p = 0.03 mean the null hypothesis has a 3% chance of being true?
No. It describes data under a specified model, not the probability that the hypothesis is true.
What does a p-value not measure on its own?
Effect size, practical importance, or probability of replication.