Saturday, May 12, 2012

What is a p value?

A p-value is a number associated with a statistical test. There are two things you need to just know to understand p-values (enough to get you through a paper that throws around the term, anyway).

First, statistical tests look for differences between two quantities. For instance, is the mean height of men different from the mean height of women? The null hypothesis is that the two things being compared are the same.

The second thing to understand is the p-value itself. The p-value is the probability that the two quantities are indeed the same. That is, how likely is it that your data would look the way it does if you assume the null hypothesis is true?

For instance, let's say you randomly select 20 men and 20 women, and the mean height of the men in your sample is five foot ten, and the mean height of the women is five foot seven. Would such data demonstrate that men (on average) are taller than women, or could the observed height difference simply be due to chance? This is what the p-value tells you. It tells you the probability that men and women have the same mean height, given what you observed in your experimental sample.

Two corollaries:
a. If the p-value is very high (e.g., 0.99), that means it is likely that the two quantities you are comparing are the same, i.e., that the null hypothesis is true. E.g., males and females are the same height.
b. If the p-value is very low (the convention is below 0.05), this suggests that the two quantities you are comparing are truly different, i.e., that the null hypothesis is not true. E.g., the two means are different.

------------------------

For the mavens:  technically the p-value is the probability of observing what you did, or a more extreme value, but that is a wrinkle that makes things too complicated for a dirty synopsis.

1 comment:

Anonymous said...

Thank you for putting that into brief but clear terms!