beyond reasonable doubt: a significant improvement

For the second time in a week I have removed the word “significant” from a draft manuscript written by a colleague of mine in clinical medicine. In a significantly p’d I wrote about the myth of significance – that is about the ubiquitous use of the term “significant” in the medical literature to mean a specific probability  incorrectly rejecting the hypothesis that two things (eg two treatments) are the same (you may need to read that twice).  What I pointed out was the “significant” does not mean “meaningful.”   Here I want to propose an alternative.  But first, I need to discuss two major problems with the term.

Where common is not specific

In my experience the common usage of “significant” to mean important is the normal interpretation of the word in the science literature even by many medically trained people and sometimes the authors of articles themselves.

The tyranny of p<0.05

When the maths wiz Ronald Fisher talked about significance (in an agricultural journal not a medical one!) he used 0ne in 20 (p<0.05) as an acceptable error rate in agricultural field trials so that trials did not have to be repeated many times.  That p<0.05 has taken on almost magical proportions (‘scuse the pun) in the medical literature is scary and shameful.  I don’t want to delve into all that now.  If you want to, a starting point maybe the Nature article here.

My proposal

I propose that in all scientific literature that authors replace the term “significant” with the phrase “beyond reasonable doubt” and that they only be allowed to publish the article if in the methods section they define what p value they choose to represent “beyond reasonable doubt” and they defend why they have chosen this value and not another.  “Beyond reasonable doubt” is a term used in the New Zealand judicial system where those charged with a crime are presumed innocent (Null hypothesis) until proven otherwise.  Perhaps those of us in science could learn something from our lawyer friends.


Got something to say? Don't be shy

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s