Understanding what a p-value is lies at the heart of our existence. However precise the professor’s description is, no one can actually make sense of it. So, we went on a quest: can we find an intuitive and memorable meaning of this devilled value? We went on the streets of Rotterdam to find out what students think is a good definition of the almighty p-value.
A measly 22 responses were captured - half good, half terrible. Most of the respondents are proper econometrics students, the rest study something inferior, like computer science. Let’s take a deeper look at the statistics.
On the top, you can see a pie chart of the “correctness” of the students’ answer, and below it we present a second pie chart that shows the same figure but of “intuitiveness”. In general, not so great. Some people find good ways to explain the p-value, but they almost all lack intuition. A good definition also needs to be unforgettable, so we decided to use our state-of-the-art econometric techniques to come up with the “memorability” factor. Overall, the responses don't deserve a nice rating for their memorability. Most definitions are bland and lack imagination, but a few stand out. Here’s the few:
-
“The probability that you think that you’re right and you’re still wrong”
-
“If there’s a chance of me falling down a cliffside, and I know I’ll slide off once I’ve gone down 4 metres already. If me not sliding down at all is the null hypothesis, and the chance of me dying is relevant, I’ll want to know the P-value of 4 metres.
-
“The edges of the normal distribution”
-
“Probability of type 1 error. Usually reject when larger than 0.05”
Naturally, we have to assess these individually, and pick out a winner. So, we rounded up with the Estimator team to carefully judge each definition, and come up with a final decision. This time we ask the respondents to rate each definition on a scale of 1 to 5 based on “correctness”, “intuitiveness”, and “memorability”. The next figure shows the sample means of these responses for each definition.
Definition 3 has a clear problem. It lacks depth and generality, and is borderline concerning. The Estimators seem to agree with this, though they do find it intuitive. Definition 4 is comical. Not only is it completely wrong, but the Estimators seem to rate its correctness as the highest across the 4 definitions. Also concerning… Definition 1 has many traits. In terms of “correctness”, it’s horrendous. That doesn’t matter as the definition succeeds on other aspects. It’s plain and harmonious. Not only does it get a good rating in the “memorability” factor, it also seems to be the most intuitive answer, according to the Estimator team.
(The probability of a type 1 error is the significance level, and usually we reject when the p-value is lower than 0.05)
Definition 2 takes the cake, however, as it surpasses all levels of expectation. Rich and creative, in addition to being memorable. The Estimators agree. Truly a masterpiece.
We would like to thank all respondents for partaking in this important experiment. We sincerely believe that we achieved our quest, though we learned some perplexing things. We learned that most econometrics students just give a vague subset of the definition from the Heijbel, and not try to think of their intuition on the definition.