- There's a lot of misconceptions around what p-values are, why they're important and how we should use them. And, I think those misconceptions really influence how we carry out experiments and how we interpret data in a lot of pretty perverse ways. 

By and large, and in biology, we have this like holy grail of the p-value being less than zero point zero five in order to say that our work is meaningful or that you should take it seriously. And, it is a bit of an arbitrary cut off. And, as it was initially intended, it was not supposed to be the end of a line of inquiry. Like, just because you got this p-value, does not mean you can stop and then say, my results are important now. It was actually to indicate that the researcher is heading in the right direction and that they should consider moving in this direction further. So, it's to increase their confidence that the hypothesis that they're testing may actually indeed be pointing them toward truth. 

Most statistical tests that are used to calculate what the p-value is, by and large, they are an expression of how big your effect size is, so how big is the difference between some conditions you're looking at and how big is your uncertainty around that variable. And then the p-value is kind of an expression of what the ratio is between these two factors. In addition to that, the level of uncertainty you have in the data also affects the p-value. 

So, you can run into instances where the effect size is pretty small, but then the uncertainty around that is also extremely small. And, for whatever reason, the uncertainty value is what sort of takes priority. And then you end up with a significant p-value. So, you might have a lot of confidence around a very small effect. So, let's say you have a drug that you're using to treat acne or something like that. And, you can show that you have really really really high confidence that it decreases the level of acne by one percent. And, like, the one percent is not really that big of a deal. But, because you're super confident about it, that makes your p-value really small and really significant. So, the p-value doesn't necessarily mean that you're having a really big effect.