Category Archives: $100Dialysis

More on the PBRFs new clothes

A few of weeks ago I outed the multi-million-dollar exercise that is the Quality Evaluation component of the performance based research fund (PBRF) as a futile exercise because there was no net gain in research dollars for the NZ academic community.  Having revealed the Emperor’s new clothes, I awaited the call from the Minister in charge to tell me they’d cancelled the round out of futility.  When that didn’t come, I pinned my hope on a revolt by the University Vice-Chancellors. Alas, the VCs aren’t revolting.  This week, my goal is for there to be mass resignations from the 30 or so committees charged with assessing the evidence portfolios of individual academics and for individual academics to make last minute changes to their portfolios so as to maintain academic integrity.

I love academic metrics – these ways and means of assessing the relative worth of an individual’s contribution to academia or of the individual impact of a piece of scholarly work are fun.  Some are simple, merely the counting of citations to a particular journal article or book chapter, others are more complex such as the various forms of the h-index. It is fun to watch the number of a citations of an article gradually creep up and to think “someone thinks what I wrote worth taking notice of”.  However, these metrics are largely nonsense and should never be used to compare academics.  Yet, for PBRF and promotions we are encouraged to talk of citations and other such metrics.  Maybe, and only maybe, that’s OK if we are comparing how well we are performing this year against a previous year, but it is not OK if we are comparing one academic against another.  I’ve recently published in both emergency medicine journals and cardiology journals.  The emergency medicine field is a small fraction the size of cardiology, and, consequently, there are fewer journals and fewer citations.  It would be nonsense to compare citation rates for an emergency medicine academic with that of a cardiology academic.

If the metrics around individual scholars are nonsense, those purporting to assess the relative importance (“rank”) of an academic journal are total $%^!!!!.  The most common is the Impact Factor, but there are others like the 5-year H-index for a journal.  To promote them, or use them, is to chip away at academic integrity.  Much has been written elsewhere about impact factors.  They are simply an average of a skewed distribution.  I do not allow students to report data in this way.  Several Nobel prize winners have spoken against them.  Yet, we are encouraged to let the assessing committees know how journals rank.

Even if the citation metrics and impact factors were not dodgy, then there is still a huge problem that faces the assessing committee, and that is they are called on to compare apples with oranges.  Not all metrics are created equal.  Research Gate, Google Scholar, Scopus and Web of Science all count citations and report h-indices.  No two are the same.  A cursory glance at some of my own papers sees a more than 20% variation in counts between them.  I’ve even paper with citation counts of 37, 42, 0 and 0.  Some journals are included, some are not depending on how each company has set up their algorithms. Book chapters are not included by some, but are by others. There are also multiple sites for ranking journals using differing metrics.  Expecting assessing committees to work with multiple metrics which all mean something different is like expecting engineers to build a rocket but not to allow them to use a standard metre rule.

To sum up, PBRF Evidence Bases portfolio assessment is a waste of resources, and encourages use of integrity busting metrics that should not be used to rank individual academic impact.


Flourish with change

Newshub decided to do an “AI” piece today. Expect much more of this kind of “filler” piece. They will go thus… “X says AI will take all our jobs, Y says AI will save us.” These pieces are about as well informed and informing as a lump of 4×2 – good for propping up a slow news day, but not much else. The “more compassionate and moral than NZers” message (which comes from Y) type statement that was made is utter nonsense. AI is just a name we give to the software of machines – AI don’t have compassion or morals. If they appear too, that is simply because they are reflecting the data we feed them… human data with all its flaws.
Yes, there is change coming because of this technology. In the past we have been particularly poor at predicting what the future will look like & I think this time the possibilities are far too numerous and complex for us to predict what will be.  Statements like “30-50% of people will lose their jobs” (said X) are simply guesses because there is no precedent on which to base the numbers. All the reports talk about truck drivers and accountants loosing jobs and not a lot else. They are shallow – and probably necessarily so – because we just can’t anticipate what creative people may come up with for this technology.  Having said that, I must admit I just am not sure what to advise my children (as if they’d take it).  Should they all learn to code? Maybe not, as most interaction with machines may not be via coding languages. Should they become artisans for niche markets where the technology doesn’t penetrate?  Maybe for some, but not for all.  I think that perhaps the best we can do is to encourage what enhances creativity and resilience to, or even better a flourishing with, change. It is my hope that flourish with change will become the mantra not just the next generation, but for all current generations, for how we determine to approach the coming changes is likely as important to the well being of our society as the changes themselves.

This is what happens when you talk to your mother about artificial intelligence

Artificial Intelligence 

Artificial Intelligence

So we don’t need to think.

Everything is done for us

In just an eyelid blink.


Artificial Intelligence

So we don’t need to think.

Just take the Robot, plug it in

And go and have a drink.


When you come back your work is done;

You haven’t even thunk.

The Robot’s done the washing too;

Oh dear, I think it’s shrunk.


Perhaps I shouldn’t have bought this one,

I didn’t even think,

I got it second hand you see

From prisoners in the clink.


And when they programmed it you see

I think that they were drunk

‘Cos now it’s full of nasty words;

I really should have thunk.


So artificial Intelligence

Depends upon the thought

That someone programmes into it,

And that may come to nought.


And so beware when buying one,

You may be feeling sunk,

It may be right for it to think,

But you also should have thunk!

(c) K.A. Pickering, October 2017

AI Robot copy (1)

Artificial Intelligence (c) K.A. Pickering, October 2017

The wrong impact

“We just got a paper in an Impact Factor 10 journal … and hope to go higher soon.”  That’s a statement made to me last week.  It is wrong on so many levels, but does it matter?   Nobel Prize winners think so. This video from appeared in my twitter feed on Friday.  Before you watch it, consider this, academics in NZ are being encouraged in promotion applications and in preparing for the next round of NZ Performance Based Research Fund (PBRF), which will allocate millions of dollars to academic institutions, to include a metric of the ranking of the journal.  The Impact Factor is the most common metric available.


ps. I would not allow a student working with me to present a raw mean of a highly skewed distribution because it so very poorly represents the distribution.  However, this is exactly what the Impact Factor does (for those who don’t know the most common impact factor for a journal in any given year is simply the sum of citations of articles from the preceding two years divided by the total number of articles published.  The citation distribution is usually skewed because the vast majority of articles receive very few citations in such a short time, but a few receive a lot).  There are numerous other problems with it, not the least that it can’t be used to compare “impact” between different disciplines.

A vision of kiwi kidneys

Sick of writing boring text reports.  Take a leaf out of Christchurch nephrologist Dr Suetonia Palmer’s (@SuetoniaPalmer) book and make a visual abstract report.  Here are two she has created recently based on data collected about organ donation and end stage renal failure by ANZDATA (@ANZDATARegistry). Enjoy.

Suetonia C-18RfJXUAApRcU

Suetonia C-16lBZXsAERoeM

ps. The featured image is of the Kidney Brothers.  Check out the great educational resources at The OrganWiseGuys.

An even quicker way to rule out heart attacks

The majority of New Zealand emergency departments look for heart muscle damage by taking a sample of blood and looking for a particular molecule called a high-sensitivity troponin T (hsTnT).  We have now confirmed that rather than two measurements over several hours just one measurement on arrival in the ED could be used to rule out heart attacks in about 30% of patients.

What did we do?

We think this is a big deal. We’ve timed this post to meet the Annas of Internal Medicine timing for when our work appears on their website – here.  What we did was to search the literature to find where research groups may have measured hsTnT in the right group of people – namely people appearing in an emergency room whom the attending physician thinks they may be having a heart attack. We also required that the diagnosis of a heart attack, or not, was made not by just one physician, but by at least two independently.  In this way we made sure we were accessing the best quality data.

Next I approached the authors of the studies as asked them to share some data with us – namely the number of people who had detectable and undetectable hsTnT (every blood test has a minimum level below which it is said to be “undetectable” in hsTnT’s case that is just 5 billionths of a gram per litre, or 5ng/L).  We also asked them to check in these patients if the electrical activity of the heart (measured by an electrocardiogram or “ECG”) looked like there may or may not be damage to the heart (a helpful test, but not used on its own to diagnose this kind of heart attack).  Finally, we asked the authors to identify which patients truly did and did not have a heart attack.

What did we find?

In the end research groups in Europe, UK, Australia, NZ, and the US participated with a total of 11 studies and more than 9000 patients.  I did some fancy statistics to show that overall about 30% of patients had undetectable hsTnT with the first blood test and negative ECGs.  Of all those who were identifiable as potentially “excludable” or “low-risk” only about 1 in 200 had a heart attack diagnosed (we’d like it to be zero, but this just isn’t possible, especially given the diagnosis is not exact).

VisualAbstract AnnalsIM 170411

Pickering, J. W.*, Than, M. P.*, Cullen, L. A., Aldous, S., Avest, ter, E., Body, R., et al. (2017). Rapid Rule-out of Myocardial Infarction With a High-Sensitivity CardiacTroponin T Measurement Below the Limit of Detection: A Collaborative Meta-analysis. Annals of Internal Medicine, 166(10). *joint first authors.

What did we conclude?

There is huge potential for ruling out a heart attack with just one blood test.  In New Zealand this could mean many thousands of people a year can be reassured even more swiftly that they are not having a heart attack. By excluding the possibility of a heart attack early, physicians can put more effort into looking for other causes of chest-pain or simply send the patient happily home.   While not every hospital performed had the same great performance, overall the results were good.  By the commonly accepted standards, it is safe.  However, we caution that local audits at each hospital that decides to implement this “single blood measurement” strategy are made to double check its safety and efficacy.

Acknowledgment: This was a massive undertaking that required the collaboration of dozens of people from all around the world – their patience and willingness to participate is much appreciated. My clinical colleague and co-first author, Dr Martin Than provided a lot of the energy as well as intelligence for this project. As always, I am deeply appreciative of my sponsors: the Emergency Care Foundation, Canterbury Medical Research Foundation, Canterbury District Health Board, and University of Otago Christchurch. There will be readers who have contributed financially to the first two (charities) – I thank you – your generosity made this possible, and there will be readers who have volunteered for clinical studies – you are my heroes.




To march or not to march?

When I’ve marched in the past it has been to protest or celebrate.  The call for a March for Science, due to take place in New Zealand on the 22nd of April, has me confused as to its purpose.

When I first heard the suggestion of a March for Science in New Zealand I admit I was immediately sceptical (occupational hazard).  The suggestion had come in response to the policies of the Trump administration in the USA.  I am appalled by many of them and by the apparent ignoring of the scientific consensus – but then given the flip-flop on so much that was said in the campaign, it would take a brave person to predict there won’t be a similar flip-flop with respect to climate change policies and the like.  That aside, is the March in New Zealand intended to be a protest against Trump?

Nicola Gaston in a persuasive blog post  writes that with her Bachelor of Arts in her back pocket she will be marching for science and the scientists. Paraphrasing Niemoller she writes “First they came for the scientists, but I was not a scientist, so I did not speak out”. She hit a nerve with me, it is a sentiment that has resonated strongly in me ever since I walked though Auschwitz concentration camp and spent several years living in a country soon after the communists had relinquished power. It is right and proper to speak out for the oppressed, whoever they are and whether we agree with them or not. However, the title of Nicola’s post “Why scientists need to go to the barricades against Trump – and for the humanities” and the first few paragraphs paint the call to march  as a political protest against Trumpian rhetoric and policy.  This, for me, is not an encouragement to march in NZ.  There are many many countries and issues around the world that I abhor and that I think reflect more closely Niemoller’s sentiments– “First they came for the migrants”, “First they came for the children (for the sex trade)”, “First they came for private property” – and I struggle with what I can do about any of them.  However, marching in New Zealand protesting policies in another country is not something I see as effective unless we are demanding action from our government against those countries.


Photo: Brandon Wu 20 Jan 2017 , Wikimedia Commons.


Since Nicola wrote that piece, the March organisers have written about the reasons for the March (here and here).  While what has happened in the US is still very much to the fore, the organisers’ attentions seems to have turned towards a protest against policies of the current government “our current government has and continues to be ineffective in defending our native species and environment” (Geni- Christchurch organiser), “The government believes they are improving freshwater, yet they aren’t utilizing NZ freshwater ecology research outputs or freshwater scientists for these decisions.” (Erin-Palmerston North), “you only have to look at the Land and Water forum to open the discussion about the government ignoring the advice of scientists in regards to water quality.” (Steph-Auckland), and on the March for Science websiteThe dismissal of scientific voices by politicians is perhaps best encapsulated by our former Prime Minister’s dismissal of concerns about the impact of our dairy industry on water quality



The organisers in the spirit of peer review invite critique.  My first thought is that if people want to protest the government’s actions with respect to water quality – then please do so.  But, please don’t dress it up as a “March for Science” as if NZ politicians are inherently anti-science.  It comes across as a belief that the NZ Government is tarred with the same brush as the Trump administration with respect to its treatment of science.  I don’t think that comparison is fair.

As an aside, I believe we must be careful with the generalisation “anti-science”, a phrase I’ve regularly heard from the voices and pens of scientists in the past few years.  The phrase has almost always been used to describe people who take stances in opposition to the scientific consensus on matters such as vaccinations, fluoridation, or climate change.  I don’t believe these people are anti-science per se – indeed, they often try (and fail) to use science to back their views. Furthermore, they may well embrace the findings of science in general.  Troy Campbell and Lauren Griffen’s recent post in Scientific America is a good panacea against the loose and pejorative use of the term “anti-science”.

Another aspects of the call to March that I find difficult is the statement “We acknowledge that in Aotearoa New Zealand the scientific community has yet to live up to the principles of Te Tiriti o Waitangi, and that there is an ongoing process of decolonization required to achieve greater inclusion of Māori in the scientific community.” I admit I’m not entirely sure what this means. However, as a member of the scientific community it sounds like I’m being slapped over the wrist.  Further, I feel it is accusing me of some form of racism.  I’m sure this was not the intention, but it is the impression I get and one I don’t like getting.

This is all a pity, as I’d hoped that the March for Science would be more of a celebration with the added value of standing in solidarity with scientists who have been silenced or disenfranchised.  To be fair, celebration is obviously on the mind of some of the organisers such as Cindy from Dunedin “together to celebrate the quest for knowledge and the use of knowledge to protect and enhance life… hope that the March for Science Global initiative will empower scientists and other knowledge-seekers to continue their important work and to share it widely.”  However, this does not seem to reflect the overall tone of the call.

One of the goals of the March is to highlight that “Good scientists can be political.”  I applaud this sentiment and it is something I have tried to be take on board in the past – twice I stood as a political candidate in the general election (2005 and 2008).  Beyond protest, I would encourage all scientists to spend a few minutes with their local MP explaining why and what they do.  The temptation is to bemoan the lack of funding, but I would suggest that funding follows understanding, and we need to engage with politicians and as we do so to recognise the complexity of the decision making with all the competing interests that they have to make.

I began with a question, to march or not to march?  As I’ve written this, I’ve come to the conclusion that, on balance, the call has not resonated with where I’m at, or with what I think of as effective dialogue with politicians, therefore I will not be marching.  I appreciate that others will disagree, nevertheless I wish them a very positive experience.