IDEA FOUND // IDEA 128
Statistical Power
“A study may miss a signal because its flashlight was too weak.”
01 / PLAINLY
What it means, plainly
Statistical power is the probability that a test detects a specified effect when that effect is present in the stated scenario. It depends on the design and the magnitude being considered.
02 / CONTEXT
A little more
Power is the probability that a test rejects the null for a specified effect and design when the considered alternative is true. It generally rises with larger samples and stronger signals.
03 / WHY IT MATTERS
Why it matters
It shows which differences a study could detect and keeps every nonsignificant result from being read as proof of no effect.
04 / EXAMPLE
A familiar example
With few participants, two treatments may look indistinguishable even when a moderate difference exists.
05 / LIMIT
What it does not mean
Low power does not automatically turn a nonsignificant result into evidence for a hidden effect.
06 / NOTICE
Notice it in your day
Compare two otherwise identical plans with different numbers of observations. Note which should be more sensitive and what that fact alone cannot prove.
FINAL NOTE
The idea worth keeping
Plan sensitivity before collecting data and report which magnitudes the study could detect.
QUESTIONS / 02
Questions people still have
What is a test's power tied to?
A specified effect and design, including sample size and signal magnitude.
Does low power turn a nonsignificant result into proof of a hidden effect?
No. It only calls for caution about what the study could detect.