Tag Archives: Science

To march or not to march?

When I’ve marched in the past it has been to protest or celebrate.  The call for a March for Science, due to take place in New Zealand on the 22nd of April, has me confused as to its purpose.

When I first heard the suggestion of a March for Science in New Zealand I admit I was immediately sceptical (occupational hazard).  The suggestion had come in response to the policies of the Trump administration in the USA.  I am appalled by many of them and by the apparent ignoring of the scientific consensus – but then given the flip-flop on so much that was said in the campaign, it would take a brave person to predict there won’t be a similar flip-flop with respect to climate change policies and the like.  That aside, is the March in New Zealand intended to be a protest against Trump?

Nicola Gaston in a persuasive blog post  writes that with her Bachelor of Arts in her back pocket she will be marching for science and the scientists. Paraphrasing Niemoller she writes “First they came for the scientists, but I was not a scientist, so I did not speak out”. She hit a nerve with me, it is a sentiment that has resonated strongly in me ever since I walked though Auschwitz concentration camp and spent several years living in a country soon after the communists had relinquished power. It is right and proper to speak out for the oppressed, whoever they are and whether we agree with them or not. However, the title of Nicola’s post “Why scientists need to go to the barricades against Trump – and for the humanities” and the first few paragraphs paint the call to march  as a political protest against Trumpian rhetoric and policy.  This, for me, is not an encouragement to march in NZ.  There are many many countries and issues around the world that I abhor and that I think reflect more closely Niemoller’s sentiments– “First they came for the migrants”, “First they came for the children (for the sex trade)”, “First they came for private property” – and I struggle with what I can do about any of them.  However, marching in New Zealand protesting policies in another country is not something I see as effective unless we are demanding action from our government against those countries.

Photo-_Brandon_Wu_(32048341330)

Photo: Brandon Wu 20 Jan 2017 , Wikimedia Commons.

 

Since Nicola wrote that piece, the March organisers have written about the reasons for the March (here and here).  While what has happened in the US is still very much to the fore, the organisers’ attentions seems to have turned towards a protest against policies of the current government “our current government has and continues to be ineffective in defending our native species and environment” (Geni- Christchurch organiser), “The government believes they are improving freshwater, yet they aren’t utilizing NZ freshwater ecology research outputs or freshwater scientists for these decisions.” (Erin-Palmerston North), “you only have to look at the Land and Water forum to open the discussion about the government ignoring the advice of scientists in regards to water quality.” (Steph-Auckland), and on the March for Science websiteThe dismissal of scientific voices by politicians is perhaps best encapsulated by our former Prime Minister’s dismissal of concerns about the impact of our dairy industry on water quality

 

Critique

The organisers in the spirit of peer review invite critique.  My first thought is that if people want to protest the government’s actions with respect to water quality – then please do so.  But, please don’t dress it up as a “March for Science” as if NZ politicians are inherently anti-science.  It comes across as a belief that the NZ Government is tarred with the same brush as the Trump administration with respect to its treatment of science.  I don’t think that comparison is fair.

As an aside, I believe we must be careful with the generalisation “anti-science”, a phrase I’ve regularly heard from the voices and pens of scientists in the past few years.  The phrase has almost always been used to describe people who take stances in opposition to the scientific consensus on matters such as vaccinations, fluoridation, or climate change.  I don’t believe these people are anti-science per se – indeed, they often try (and fail) to use science to back their views. Furthermore, they may well embrace the findings of science in general.  Troy Campbell and Lauren Griffen’s recent post in Scientific America is a good panacea against the loose and pejorative use of the term “anti-science”.

Another aspects of the call to March that I find difficult is the statement “We acknowledge that in Aotearoa New Zealand the scientific community has yet to live up to the principles of Te Tiriti o Waitangi, and that there is an ongoing process of decolonization required to achieve greater inclusion of Māori in the scientific community.” I admit I’m not entirely sure what this means. However, as a member of the scientific community it sounds like I’m being slapped over the wrist.  Further, I feel it is accusing me of some form of racism.  I’m sure this was not the intention, but it is the impression I get and one I don’t like getting.

This is all a pity, as I’d hoped that the March for Science would be more of a celebration with the added value of standing in solidarity with scientists who have been silenced or disenfranchised.  To be fair, celebration is obviously on the mind of some of the organisers such as Cindy from Dunedin “together to celebrate the quest for knowledge and the use of knowledge to protect and enhance life… hope that the March for Science Global initiative will empower scientists and other knowledge-seekers to continue their important work and to share it widely.”  However, this does not seem to reflect the overall tone of the call.

One of the goals of the March is to highlight that “Good scientists can be political.”  I applaud this sentiment and it is something I have tried to be take on board in the past – twice I stood as a political candidate in the general election (2005 and 2008).  Beyond protest, I would encourage all scientists to spend a few minutes with their local MP explaining why and what they do.  The temptation is to bemoan the lack of funding, but I would suggest that funding follows understanding, and we need to engage with politicians and as we do so to recognise the complexity of the decision making with all the competing interests that they have to make.

I began with a question, to march or not to march?  As I’ve written this, I’ve come to the conclusion that, on balance, the call has not resonated with where I’m at, or with what I think of as effective dialogue with politicians, therefore I will not be marching.  I appreciate that others will disagree, nevertheless I wish them a very positive experience.

Advertisements

Policy our lives depend on: Health research in election 2014

We all care about health – ours, our family’s, and even that of one or two politicians (perhaps). We also care that the 15 billion dollar annual health budget is spent on health care that works.  I contend that both these cares are only as good as the health research that underpins the treatments we receive.  Therefore, I have compiled what I could discover about health research policy from the policy documents available online of the political parties contending the current NZ general election. I have tried to focus on where health research in a particular area is promised or on health research infrastructure. In some places I’ve extracted from a more general science and/or innovation policy those policies I think likely to impact health research.  Obviously some parties are still releasing policy.  I invite them to send me any policies that they think relevant and I will update.  I think you will be surprised at what is missing in the list below.

The parties are in reverse alphabetical order.

United Future*

Health Policy: http://www.unitedfuture.org.nz/policy/health

  • Increase funding for health research to bring New Zealand’s funding up to at least the OECD average as a proportion of GDP;
  • Establish a national register for Type 1 Diabetes, a diabetes research fund, and increase funding for Type 2 Diabetes testing;
  • Make no change to the legal status of cannabis for medicinal use until a robust regulatory testing regime is developed that proves cannabis use causes minimal harm to an individual’s health
  • Introduce a sabbatical scheme that would allow health professionals to take a year out of work every five years to update their skills and knowledge;
  • Promote more research to address youth related health problems such as suicide, alcoholism, and bulimia.

Science Policy: http://www.unitedfuture.org.nz/policy/research-science-and-technology

Too long to put in detail, but policies such as “simplifying different funding mechanisms” and specifying biotech as one of half a dozen key research areas requiring focus are likely to impact on health research.

Health spokesperson (Associate Minister of Health): Peter Dunne MP peter.dunne@parliament.govt.nz

 

New Zealand First

Health Policy: http://nzfirst.org.nz/policy/health

  • Ensure an on-going commitment to the funding of health research, research institutes, and for training.

Science Policy: None

RS&T Portfolio holder: Tracey Martin MP tracey.martin@parliament.govt.nz

Health Portfolio holder: Barbara Steward MP   barbara.stewart@parliament.govt.nz

 

National

Health Policy: https://www.national.org.nz/news/features/health

No specific policy on any health research

Science Policy: None

Health spokesperson (Minister of Health): Tony Ryall tony.ryall@national.org.nz

Science spokesperson (Minister of Science and Innovation): Steven Joyce steven.joyce@national.org.nz

 

Maori Party

Policy: http://maoriparty.org/our-policies-kawanatanga/

  • We will support: … Roadshows to promote educational pathways in areas where Māori are under-represented – ie health science academies (Te Kura Pūtaiao Hauora) or science camps.

Science Policy: No specific policy but some comments in the policy above about research and development include establishing an investment fund for Māori Research and Development which may impact on health research.

Health or Science spokespeople: Unknown

Contact: Teururoa Flavell MP teururoa.flavell@parliament.govt.nz

 

Mana

Health Policy: http://mana.net.nz/policy/policy-health/

No policy specifically dealing with health research

Science Policy: None

Contact: Hone Harawira MP hone.harawira@parliament.govt.nz

 

Labour

Health Policy: http://campaign.labour.org.nz/full_health_policy

  • We need a health system that is based on evidence about what works – not fixated on manufactured targets or political slogans

Health spokesperson: Annette King annette.king@parliament.govt.nz

Science Policyhttps://www.labour.org.nz/sites/default/files/issues/science_and_innovation_policy.pdf (UPDATE – released 25 August)

  • Reinstate post-doctoral fellowships for recent PhD graduates (scaling up to %6m per year)
  • Prioritise an increase in our public science spend to link New Zealand to the OECD average over time
  • review and reform the National Science Challenges, on the basis of advice from the science community and building on the success of respected funding bodies such as the Marsden Fund

    provide integrated support for innovation across the Crown Research Institutes and tertiary institutions, and through private-sector research activities, and sectoral and regional initiatives

    review the criteria of the Performance Based Research Fund to ensure that a broad range of research success is recognised

    support research in universities, including through a continued commitment to Centres of Research Excellence

    encourage closer association between business and university commercialisation centres to ensure ‘discoveries’ within the universities are most effectively brought to market and have the best chance for success

    support and foster a collaborative university system, where each of our universities is enabled to focus on its areas of research and teaching strength.

  • support research in universities, including through:
    • a continued commitment to Centres of Research Excellence,
    • ensuring the sustainability of the Marsden Fund and other research funds
    • supporting the career pathways of graduates, to encourage our researchers to develop their careers and contribute to New Zealand.

Science Spokesperson: Moana Mackey MP moana.mackey@parliament.govt.nz

 

Internet

Health Policy: https://docs.google.com/document/d/1g4RY7Sh-vYZN1WAIx_A-AEZlYzNjMhzY81KnfKLMGp0/edit

Copyright and Open Research Policy: https://docs.google.com/document/d/1Le3rY0wlh9tJaBzpxK5xrpeWID-j5FmeE4dqONdQATE/edit

  • Mandate that all taxpayer-funded research be open access with the public able to freely access and re-use it.

Health or Science spokespeople: Unknown

Contact: hello@internet.org.nz

 

Green

Health Policy: No general health policy, but some on particular issues.

Update 25 Aug:  I have been informed that the Greens have a health policy on a different web site https://home.greens.org.nz/policy/health-policy.  Their election site http://www.greens.org has no health policy.

No policy specifically dealing with health research.

Green innovation Policy: https://www.greens.org.nz/policy/smarter-economy/smart-green-innovation

Some aspects of this policy may impact health research, in particular:

  • $1 billion of new government funding over three years for research and development to kick-start a transformational shift in how our economy creates wealth;
  • The Green Party will fund an additional 1,000 places at tertiary institutions for students of engineering, mathematics, computer science, and the physical sciences.

Health or Science spokespeople: Unknown

Contact greenparty@greens.org.nz

 

Conservatives

Health Policy: None

Science Policy: None

Health or Science spokespeople: Unknown

Contact: Office@conservativeparty.org.nz

 

ACT

Health Policy: http://www.act.org.nz/policies/health-0

No policy specifically dealing with health research

Science Policy: No science policy

Health or Science spokespeople: Unknown

Contact: info@act.org.nz

________________________________________________________________________________________________

*Disclaimer: I used to be a member of United Future and made submissions on the health and science policies in 2008. A few echoes of those submissions remain in the policies.

Happy birthday “New Zealand Science Today”

It’s one year, 2184 articles, 460 subscribers, and 11,721 “flips” since the online Flipboard magazine which collates articles about what NZ scientists are doing and saying was first published.  Thanks to all the contributors, and especially to all those out in NZ Science-ville who are making a difference and letting the world know about it.  New Zealand Science Today can be found on Flipboard or on the internet here.NZ Science Today

 

 

Teach creationism; undermine theology

My fellow science blogger Alison Campbell recently wrote a blog post entitled “teach creationism, undermine science”  in which she highlighted some of the concerns shared by many scientists.  As a Christian and as a scientist I believe the issue is far worse than the undermining of science.  Because so many teaching creationism are so well meaning it saddens me to say this, but the teaching of creationism is anathema to the Christian gospel.  Three reasons:

1. Creationism misrepresents the Bible.  When the Hebrews were standing on the banks of the Jordan wondering what would befall them should they cross, they were not asking “How did God create the world?” Rather, they were wondering if Yahweh who had led their parents generation out of Egypt, and seemed to be in charge in the desert, actually knew anything about farming across the Jordan.  It seemed to them that the local fertility gods must know something – after all it was a rich land.  The two stories of creation we now find in the book of Genesis speak to the fears of the Hebrews then, and in later generations to fears held when they were in exile.  The message is clear – Yahweh is in charge, and the so called gods (eg the sun and moon) are mere creations of his.  Those stories are definitely not scientific accounts – indeed science writing was not to be invented for thousands of more years. They answer “whose in charge” and “what’s my purpose” questions, not “How” questions.

2. Creationism rejects truth about the workings of creation which science has revealed. Science is a gracious gift which is to be cherished and put to good use.  It is under God’s sovereignty and requires the participation of his people. Indeed, creationism opts out of kingdom building, the task of the Church.  Our destination is not some super-spiritual, non-material eternal existence in heaven (indeed no where does the Bible explicitly say we will “go to heaven”), rather it is a new earth (material) where God dwells amongst us and God’s rule applies (heaven).  How this will come about, no-one can be certain, but our pursuit of knowledge through science and our applying that knowledge as good stewards of the Earth is part of the process of building the kingdom.

3. Creationism puts a stumbling block to faith. Sadly, propagating creationism results in an easy, and sometimes convenient, target for scientists who may otherwise be willing to listen to what Christianity has to say.  To use Paul’s terminology, it is a stumbling block. Many pupils taught creationism as a science will later learn the falsehood when they are exposed to all of science in its full glory. Sadly, many will react against Christianity and throw the baby out with the bathwater.  When this happens, those who taught those pupils creationism as if it were science will become accountable.

Should scientists respond to pseudo-science?

Do not answer a fool according to his folly, or you yourself will be just like him.

Answer a fool according to his folly, or he will be wise in his own eyes. (Proverbs 26:4, 5 NIV)

The editors of this particular list of proverbs were not fools – they knew they appeared contradictory.  Their purpose is to get us chewing over how we decide when we should speak up and when we shouldn’t.  When I heard these proverbs on Sunday my mind wandered (sorry Rev) immediately to my fellow science bloggers and the choices we make to respond or not respond to pseudo-science.  When we respond we do so wth hope.  Hope that the second proverb applies and the fool will recognise their own folly rather than keep on believing in their own wisdom.  A question I have for my fellow bloggers, how often does this actually take place?  I suspect, rarely.  At what point are we casting “pearls before swine”?  How do we know?

Perhaps more importantly, other than wasting our own time, could we be doing more harm than good (the first proverb)? By putting our scientific standing behind our reponses could we be enhancing the reputation of the pseudo-scientist in their own eyes or, worse, the eyes of readers? I think scientists are still paying the price for the over-confidence in science as solution to the world’s problems.  This has lead to some skepticism and a willingness to look at solutions that are not “main-stream” (especially if government funded or big-pharma).  By responding to the obvious nonesense, do we merely spread it further?

Some pseudo-science is addressing issues which also have non-scientific ethical issues that need to be respected.  Furthermore, the pseudo-science proponent may hold similar hopes to their scientist critic – eg hope for improved health.  I’m thinking particularly of issues such as vaccination or additives to food or water in which we need to weigh up the rights individuals with our responsibilites to others. Here, a scientist may express their opinion and their methodology of arriving at that opinion, but they need to tread very carefully not to appeal to Science with a capital “S” as if that is the ultimate standard against which all ethical decisions should be measured.

Here endeth the sermon.  Let us chew.

Don’t call this scientist soft!

I’m a soft money scientist, not because I’m cuddly (I am), or because I’m an easy mark for a fiver (I’m not), but because my job and my scientific output depend on my ability/luck at raising money.  As my 100th blog post I thought it time to describe this precarious state of affairs, especially as your taxes may be contributing to it.  Also, when the penny dropped with some friends of mine, so did their jaws.

Before I get into the description, let me say this: It is the best of jobs, it is the worst of jobs.  It is a privilege to spend most of my time solving the puzzle that are the diseases I study with the hope of making a difference to patients in the future.  It is appallingly frustrating that I cannot conduct long-term research or even rely on having an income next year because of the continued axe floating a few feet above my cranium.

In New Zealand, at least, scientists come in many flavours.  There is the industrial scientist earning a salary in a company somewhere who will sink or swim along with the fortunes of the company, there are the scientists in Callaghan Innovation, Ag Research, and other government entities that interface between academia, the commercial world, and the provision of scientific services.  I understand they have a variety of funding sources – in recent years the government side of it has moved from project grant based towards more bulk funding.  Given what is happening with Ag Research, I don’t know if that means more secured tenure for these scientists or not … I’ll let them describe their predicament.  Then, in academic institutions, there are the lecturer scientists who both teach and research.  Traditionally the spend their time 40% teaching, 40% researching, 20% in administration, but there are many variations on the theme. Normally, these people have a more-or-less permanent position (at least as long as students keep coming to do the courses they teach).  To get funding for their research (though not their salary unless they want to “buy out” some teaching time) they need to apply for grants.  In my institution, University of Otago Christchurch, most of the teachers are also active senior medical staff with joint appointments with the CDHB.

Then there are the soft-money scientists.  Most PhD students go on to do a 1 or 2 year post-doc (or two) which is funded by a grant that has been obtained by a senior researcher somewhere.  This is “soft-money” – meaning of limited duration and usually directed at a particularly project.  Most post-docs move into lecturing or leave academia.  A few may pick up additional fellowships or join a group which has the funds to employ them.  To continue in their chosen career they must contribute to the gathering of resources (money money money).  They have no training in this, but after the first few grant rejections begin to learn.  They realise they are competing against scientists who are lecturers or in other entities who already have their salaries covered.  However, the first thing they must put on their grant is their own salary + overheads (113% in my institution).  This, of course, limits what they may be able to say they will do in a grant application as they are not able to write into the grant all the expenses they’d like.  This puts them at a competitive disadvantage.  Another source of income for some groups may be commercial.  This may be the testing in their labs of some equipment  or a new product, or some forensic work etc. Not everyone has that option.

My own sojourn has been a little off the beaten path as six years ago at the age of 40 mumble I returned to the scientific fold after 15 years out of it.  My return was funded for two years initially by a Health Research Council Grant (HRC; your tax dollars) and by a private company who had obtained some government funding for development (Syft).  Since then I’ve had grants from the Australia New Zealand Society of Nephrologists (twice 🙂  ), Lottery Health, University of Otago Research Grant, and the Marsden Foundation.  My current funding till the end of the year is 41% from a Marsden Foundation grant and 59% from the profits of the last project (a commercial one) our lab-based group ran (alas … another long story, there is now no lab-based group).  Having multiple sources of income is not at all unusual for the more senior research scientists.  Indeed, the current funding levels of even the largest of the grants (HRC and Marsden) are not sufficient to fund a full time senior scientist along with all the associated costs of running a larger project (which these are intended for). The application success rates (7%) make it unlikely that anyone, other than in large established groups with broad funding basis whose success breeds grant success (rightly so!), will be able to sustain a long-term career based on grant funding alone.

One source of funding that I’ve not talked about is philanthropy.  This plays a vital, though small, role in New Zealand science.  Most are familiar with the likes of the Heart Foundation or the Cancer Society which take donations and use some of them for research projects.  An intriguing, though seldom visited, new source of funding is so called “crowd sourcing” where someone pitches a project online to raise money – Dr Siouxsie Wiles successfully raised US$4,480 last year doing just that. This, of course, will not sustain a scientist like myself.  What will?  What do you think is reasonable to spend on science and scientists?  How about the same as we spend per classroom?  According to a Principal acquaintance it costs about $17K per pupil p.a. to run a school.  The average class size is about 23 pupils making it a tad under $400K p.a per classroom.  I think what I do has similar value to educating a class full of kids, but right now I’d settle for half the amount.  Governments, of course, must make choices and impose certain limits on spending.  The current NZ government’s moves to increase spend in research are welcome, but this will at best make a small dent in the grant funding success rate.  Individuals with discretionary disposable income, though, may have other priorities.  I believe that for New Zealand to do more than tread water in the scientific world that it will require those individuals who recognise the value of science to be willing to donate substantial amounts towards science, particularly towards supporting scientists (scientists first, projects second). Indeed, for my own growth and survival as a scientist – for me to be able to put the vision I articulated last week into practice, I see that it will only be possible through the generosity of others.

Significantly p’d

I may be a pee scientist, but today is brought to you by the letter “P” not the product.  “P” is something all journalists, all lay readers of science articles, teachers, medical practitioners, and all scientists should know about.  Alas, in my experience many don’t and as a consequence “P” is abused. Hence this post.  Even more abused is the word “significant” often associated with P; more about that later.

P is short for probability.  Stop! – don’t stop reading just because statistics was a bit boring at school; understanding maybe the difference between saving lives and losing them.  If nothing so dramatic, it may save you from making a fool of yourself.

P is a probability.  It is normally reported as a fraction (eg 0.03) rather than a percentage (3%).  You will be familiar with it when tossing a coin.  You know there is a 50% or one half or 0.5 chance of obtaining a heads with any one toss.  If you work out all the possible combinations of two tosses then you will see that there are four possibilities, one of which is two heads in a row.  So the prior (to tossing) probability of two heads in a row is 1 out 4 or P=0.25. You will see P in press releases from research institutes, blog posts, abstracts, and research articles, this from today:

“..there was significant improvement in sexual desire among those on  testosterone (P=0.05)” [link]

So, P is easy, but interpreting P depends on the context.  This is hugely important.  What I am going to concentrate on is the typical medical study that is reported.  There is also a lesson for a classroom.

One kind of study reporting a P value is a trial where one group of patients are compared with another.  Usually one group of patients has received an intervention (eg a new drug) and the other receives regular treatment or a placebo (eg a sugar pill).  If the study is done properly a primary outcome should have been decided before hand.  The primary outcome must measure something – perhaps the number of deaths in a one year period, or the mean change in concentration of a particular protein in the blood.  The primary outcome is how these what is measured differs between the group getting the new intervention and the group not getting it.  Associated with it is a P value, eg:

“CoQ10 treated patients had significantly lower cardiovascular mortality (p=0.02)” [link]

To interpret the P we must first understand what the study was about and, in particularly, understand the “null hypothesis.”  The null hypothesis is simply the idea the study was trying to test (the hypothesis) expressed in a particular way.  In this case, the idea is that CoQ10 may reduce the risk of cardiovascular mortality.  Expressed as a null hypothesis we don’t assume that it could only decrease rates, but we allow for the possibility that it may increase as well (this does happen with some trials!).  So, we express the hypothesis in a neutral fashion.  Here that would be something like that the risk of cardiovascular death is the same in the population of patients who take CoQ10 and in the population which does not take CoQ10.  If we think about it for a minute, then if the proportion of patients who died of a cardiovascular event was exactly the same in the two groups then the risk ratio (the CoQ10 group proportion divided by the non CoQ10 group proportion) would be exactly 1.  The P value, then answers the question:

If the risk of cardiovascular death was the same in both groups (the null hypothesis) was true what is the probability (ie P) that the difference in the actual risk ratio measured from 1 is as large as was observed simply by chance?

The “by chance” is because when the patients were selected for the trial there is a chance that they don’t fairly represent the true population of every patient in the world (with whatever condition is being studied) either in their basic characteristics or their reaction to the treatment. Because not every patient in the population can be studied, a sample must be taken.  We hope that it is “random” and representative, but it is not always.  For teachers, you may like to do the lesson at the bottom of the page to explain this to children.  Back to our example, some numbers may help.

If we have 1000 patients receiving Drug X, and 2000 receiving a placebo.  If, say, 100 patients in the Drug X group die in 1 year, then the risk of dying in 1 year we say is 100/1000 or 0.1 (or 10%).  If in the placebo group, 500 patients die in 1 year, then the risk is 500/2000 or 0.25 (25%).  The risk ratio is 0.1/0.25 = 0.4.  The difference between this and 1 is 0.6.  What is the probability that we arrived at 0.6 simply by chance?  I did the calculation and got a number of p<0.0001.  This means there is less than a 1 in 10,000 chance that this difference was arrived at by chance.  Another way of thinking of this is that if we did the study 10,000 times, and the null hypothesis were true, we’d expect to see the result we saw about one time.  What is crucial to realise is that the P value depends on the number of subjects in each group.  If instead of 1000 and 2000 we had 10 and 20, and instead of 100 and 500 deaths we had 1 and 5, then the risks and risk ratio would be the same, but the P value is 0.63 which is very high (a 63% chance of observing the difference we observed).  Another way of thinking about this is what is the probability that we will state there is a difference of at least the size we see, when there is really no difference at all. If studies are reported without P values then at best take them with a grain of salt.  Better, ignore them totally.

It is also important to realise that within any one study that if they measure lots of things and compare them between two groups then simply because of random sampling (by chance) some of the P values will be low.  This leads me to my next point…

The myth of significance

You will often see the word “significant” used with respect to studies, for example:

“Researchers found there was a significant increase in brain activity while talking on a hands-free device compared with the control condition.” [Link]

This is a wrong interpretation:  “The increase in brain activity while talking on a hands-free device is important.” or  “The increase in brain activity while talking on a hands-free device is meaningful.”

“Significant” does not equal “Meaningful” in this context.  All it means is that the P value of the null hypothesis is less than 0.05.   If I had it my way I’d ban the word significant.  It is simply a lazy habit of researchers to use this short hand for p<0.05.  It has come about simply because someone somewhere started to do it (and call it “significance testing”) and the sheep have followed.  As I say to my students, “Simply state the P value, that has meaning.”*

sig

_____________________________________________________________

For the teachers

Materials needed:

  • Coins
  • Paper
  • The ability to count and divide

Ask the children what the chances of getting a “Heads” are.  Have a discussion and try and get them to think that there are two possible outcomes each equally probable.

Get each child to toss their coin 4 times and get them to write down whether they got a head or tail each time.

Collate the number of heads in a table like.

#heads             #children getting this number of heads

0                      ?

1                      ?

2                      ?

3                      ?

4                      ?

If your classroom size is 24 or larger then you may well have someone with 4 heads or 0 (4 tails).

Ask the children if they think this is amazing or accidental?

Then, get the children to continue tossing their coins until they get either 4 heads or 4 tails in a row.  Perhaps make it a competition to see how fast they can get there.  They need to continue to write down each head and tail.

You may then get them to add all their heads and all their tails.  By now the proportions (get them to divide the number of heads by the number of tails).  If you like, go one step further and collate all the data.  The probability of a head should be approaching 0.5.

Discuss the idea that getting 4 heads or 4 tails in a row was simply due to chance (randomness).

For more advanced classes, you may talk about statistics in medicine and in the media.  You may want to use some specific examples about one off trials that appeared to show a difference, but when repeated later it was found to be accidental.

_____________________________________________________________

*For the pedantic.  In a controlled trial the numbers in the trial are selected on the basis of pre-specifying a (hopefully) meaningful difference in the outcome between the case and control arms and a probability of Type I (alpha) and Type II (beta)  errors.  The alpha is often 0.05.  In this specific situation if the P<0.05 then it may be reasonable to talk about a significant difference because the alpha was pre-specified and used to calculate the number of participants in the study.