Usage of p values sparks heated debate among academics

P values are widely used to indicate whether any difference between groups, relationships, etc., could be due to chance or the variable(s) that are being studied. Conventionally, results are considered to be statistically “significant” when the p value is below 0.05. A couple of months ago, some academics stated that the threshold should be lowered to 0.005 to prevent false positives from being reported. This proposal sparked several discussions and debates in the scientific community. Some academics have recently responded to this proposal, indicating that a better way of tackling the misuse of p values is to mandate academics to justify the way they have used p values.

A widely discussed problem is that the accepted baseline of p values leaves a lot of room for misuse and misinterpretation. In July 2017, a group of 72 prominent researchers posted a manuscript on the PsyArXiv preprint server which argued that for the social and biomedical sciences, the p value threshold should be lowered to 0.005. Daniel Benjamin, one of the paper’s lead authors and an economist at the University of Southern California in Los Angeles, said that, “Claims with P values between 0.05 and 0.005 should be treated merely as “suggestive evidence” instead of established knowledge.”  

On 18 September, a group of researchers posted a response countering this idea. Daniel Lakens, an experimental psychologist at Eindhoven University of Technology in the Netherlands and lead author of this counterargument, argues that enforcing a strict p value threshold could create new problems such as an increase in the number of false negatives. Thus, the solution to one problem could become the origin of a new one. Lakens also highlights another potential problem of “file drawer,” which means several studies may remain unpublished as negative findings are less likely to get accepted by journals. “Before you implement any policy, you want to be more certain that there are no unintended negative consequences,” he said. He proposes that researchers should make a note of the p value threshold they select in their registered reports along with a justification for their selection before they begin data collection. According to him, this would not only lead to responsible usage of p values but also make the analysis of the usage easy.  

The problem of irreproducible results is one of the biggest challenges academics are facing, which makes responsible usage of p values important. Some journals have gone to the extent of banning p values as they are not replicable. There seems to be a clear divide among academics about the way p values should be used. It remains to be seen whether academics are able to arrive at a unanimous decision on this issue.

Does your research involve the use of p values? What are your views on reducing the significance threshold? Would that affect your research? If yes, how so? Leave your comments below.

Related reading:

The correct way to report p values

Is my research significant? Why you shouldn't rely on p values

References:

Big names in statistics want to shake up much-maligned P value

'One-size-fits-all’ threshold for P values under fire