[A] p-value is the probability of our observing the sample data we have observed when, in fact, the two population averages are identical.
- Anthony Davies, Understanding Statistics (2017)
An Intuitive Explanation
Imagine you are comparing two observations (a la some difference of means test): the mean and standard deviation of employment rate in California and New York, USA.
You can think of the p-value as the probability that the apparent difference in employment rate between California and New York is due to random chance.
Think about this:
- The higher that probability, the more likely any disparity can be (effectively) written off.
- The lower that probability, the more significant the difference is and more you should consider investigating further.
Closely related to this idea is [statistically_significant statistical significance], where a p-value of 0.05 is typical and 0.01 is used in more stringent trial.
[todo: Mathematical explanation]
[todo: link into a statistics learning path]
[todo: equation from mean-value test]