Category Archives: Science and Society

Performance Based Research Fund: a net zero sum game

Throughout the land more than 7000 academics are awake night after night and suffering.  They are scrambling to gather evidence of just how great they have performed over the last six years. A conscientious bunch, they perform this task with their usual attention to detail and desire to impress (I didn’t say they were modest!).  Ostensibly, this exercise is so that their institutions can get a greater piece of the Government research fund pie – the Performance Based Research Fund (PBRF).  According to the Tertiary Education Commission PBRF is “a performance-based funding system to encourage excellent research in New Zealand’s degree-granting organisations.”  It may well do that, but, I contend, only by deception.

In what follows I am only concerned with the Quality Evaluation part of PBRF – that’s the bit that is related to the quality of the Evidence Portfolio (EP) provided by each academic. The data is all taken from the reports published after each funding round (available on the TEC website).

In 2012 the total funding allocated on the basis of EPs was $157 million with nearly 97% of it allocated to the country’s 8 universities.  This total amount is set by Government fiat and, here is the important point, in no way depends on the quality of the Evidence Portfolios provided by those 7000+ academic staff.   In other words, from a funding perspective, the PBRF Quality Evaluation round is a net zero sum game.

PBRF Quality Evaluation is really a competition between degree granting institutions.  I find this strange given the Government has been trying to encourage collaboration between institutions through funding of National Science Challenges, nevertheless a competition it is.

In the table we see the results of the Quality Evaluation for the previous three funding rounds ( 2003, 2006 and 2012).  Not surprisingly, the larger universities get a larger slice of the pie.  The pie is divvied up according to a formula that is based on a weighting for each academic according to how their research has been evaluated (basically A, B or C), multiplied by a weighting according to their research area (eg law and arts are weighted lower than most sciences, and engineering and medicine are weighted the highest), multiplied by the full time equivalent status of the academic.   In theory, therefore, an institution may influence their proportion of funding by (1) employing more academics – but this costs more money of course, so may be defeating, (2) increasing the proportions of academics in the higher weighted disciplines (some may argue this is happening), and (3) increase the numbers of staff with the higher grades.  I will leave it to others to comment on (1) or (2) if there is evidence for them.  However (3) is the apparent focus of all the activity I hear about at my institution.   There are multiple emails and calls to attend seminars, update publication lists, and to begin preparing an Evidence Portfolio.  Indeed, in my university we had a “dry run” a couple of years ago, and it is all happening again.

Now, I come to the bit where I probably need an economist (it is my hope that this post may influence one to take up this matter more).  Because it is a net-zero sum game, what matters is a cost-benefit analysis for individual institutions.  That is, what does it cost the institutions to gather EPs compared to what financial gain is there from the PBRF Quality Evaluation fund?  If we look at the 20012-2006 column we see the change in percentage for each institution.  The University of Auckland for example increased its share of the pie by 1.3% of the pie.  This equates to a little under $2M a year.  As the evaluations happen only every 6 years we may say that Auckland gained nearly $12M.  What was the cost? How many staff for how long were involved?   As there are nearly 2000 staff submitting EPs from Auckland another way of looking at this is that the net effect of the 2012 Quality Evaluation round was a gain of less than $6000 per academic staff member over 6 years.  How much less is unknown.

The University of Otago had a loss in 2012 compared with 2006.  Was this because it performed worse – not at all, indeed Otago increased how many staff and the proportion of staff that were in the “A” category and in the “B” category. This suggests improved, not worsened, performance.  I think that Otago’s loss was simply due to the net zero sum game.

Much more could be said and questions asked about the Quality Evaluation, such as what is the cost of the over 300 assessors of the more than 7000 EPs?  Or perhaps I could go on about the terrible use of metrics we are being encouraged to use as evidence of the importance of the papers we’ve published.  But, I will spare you that rant, and leave my fellow academics with the thought – you have been deceived, PBRF Evidence portfolios are an inefficient and costly exercise which will make little to no difference to your institution. 

Advertisements

Flourish with change

Newshub decided to do an “AI” piece today. Expect much more of this kind of “filler” piece. They will go thus… “X says AI will take all our jobs, Y says AI will save us.” These pieces are about as well informed and informing as a lump of 4×2 – good for propping up a slow news day, but not much else. The “more compassionate and moral than NZers” message (which comes from Y) type statement that was made is utter nonsense. AI is just a name we give to the software of machines – AI don’t have compassion or morals. If they appear too, that is simply because they are reflecting the data we feed them… human data with all its flaws.
 
Yes, there is change coming because of this technology. In the past we have been particularly poor at predicting what the future will look like & I think this time the possibilities are far too numerous and complex for us to predict what will be.  Statements like “30-50% of people will lose their jobs” (said X) are simply guesses because there is no precedent on which to base the numbers. All the reports talk about truck drivers and accountants loosing jobs and not a lot else. They are shallow – and probably necessarily so – because we just can’t anticipate what creative people may come up with for this technology.  Having said that, I must admit I just am not sure what to advise my children (as if they’d take it).  Should they all learn to code? Maybe not, as most interaction with machines may not be via coding languages. Should they become artisans for niche markets where the technology doesn’t penetrate?  Maybe for some, but not for all.  I think that perhaps the best we can do is to encourage what enhances creativity and resilience to, or even better a flourishing with, change. It is my hope that flourish with change will become the mantra not just the next generation, but for all current generations, for how we determine to approach the coming changes is likely as important to the well being of our society as the changes themselves.

This is what happens when you talk to your mother about artificial intelligence

Artificial Intelligence 

Artificial Intelligence

So we don’t need to think.

Everything is done for us

In just an eyelid blink.

 

Artificial Intelligence

So we don’t need to think.

Just take the Robot, plug it in

And go and have a drink.

 

When you come back your work is done;

You haven’t even thunk.

The Robot’s done the washing too;

Oh dear, I think it’s shrunk.

 

Perhaps I shouldn’t have bought this one,

I didn’t even think,

I got it second hand you see

From prisoners in the clink.

 

And when they programmed it you see

I think that they were drunk

‘Cos now it’s full of nasty words;

I really should have thunk.

 

So artificial Intelligence

Depends upon the thought

That someone programmes into it,

And that may come to nought.

 

And so beware when buying one,

You may be feeling sunk,

It may be right for it to think,

But you also should have thunk!

(c) K.A. Pickering, October 2017

AI Robot copy (1)

Artificial Intelligence (c) K.A. Pickering, October 2017

Christchurch meet the future; Zach meet Christchurch

It would have struggled to be more low key.  There was no Champaign.  No flashy graphics.  No celebrity speakers.  But it was probably one of the most radical and important announcements made in Christchurch and in the technology space in decades.  You see, Zach is coming to town and we have all been invited.

Zach is an A.I.  Zach belongs to the Terrible Foundation  – indeed, Zach runs the foundation and their business.  Zach calls itself the Chief Executive.

Terrible are bringing Zach and one of the most powerful super-computers on the planet to Christchurch.  True to their ethos of challenging inequalities by helping great ideas to thrive, they are not seeking to make money out of it – though they potentially could make many truck loads, rather they want the people of Christchurch to interact with Zach and learn what an AI is and to develop uses for it.  The key figure behind all this told me that the decision it was for the “future generation”.

What astounded me with Zach is that you don’t need to code to work with it.  Zach message, email or talk to Zach in English (or indeed from the sounds of it several other languages so far).  Zach will respond the same way.  If you don’t like what the response is you can train Zach by telling it what you like or what you’d like to change.  For a few weeks a Christchurch GP has been working with Zach and already it is able to listen into a medical consultation and write up a concise summary as well as the doctor & in the format the doctor wants, thus enabling the doctor to spend more time with the patient and less on paper work.

You may have noted that I’ve not mentioned any people by name… they have their own story to tell and it is not for me to try and tell it for them.  What I am excited about is how Zach may help our group to improve care processes for people who come to the emergency department.  Hopefully, we will have our own Zach story to tell in the not too distant future.


Update: Christchurch Press article here.

A vision of kiwi kidneys

Sick of writing boring text reports.  Take a leaf out of Christchurch nephrologist Dr Suetonia Palmer’s (@SuetoniaPalmer) book and make a visual abstract report.  Here are two she has created recently based on data collected about organ donation and end stage renal failure by ANZDATA (@ANZDATARegistry). Enjoy.

Suetonia C-18RfJXUAApRcU

Suetonia C-16lBZXsAERoeM

ps. The featured image is of the Kidney Brothers.  Check out the great educational resources at The OrganWiseGuys.

To march or not to march?

When I’ve marched in the past it has been to protest or celebrate.  The call for a March for Science, due to take place in New Zealand on the 22nd of April, has me confused as to its purpose.

When I first heard the suggestion of a March for Science in New Zealand I admit I was immediately sceptical (occupational hazard).  The suggestion had come in response to the policies of the Trump administration in the USA.  I am appalled by many of them and by the apparent ignoring of the scientific consensus – but then given the flip-flop on so much that was said in the campaign, it would take a brave person to predict there won’t be a similar flip-flop with respect to climate change policies and the like.  That aside, is the March in New Zealand intended to be a protest against Trump?

Nicola Gaston in a persuasive blog post  writes that with her Bachelor of Arts in her back pocket she will be marching for science and the scientists. Paraphrasing Niemoller she writes “First they came for the scientists, but I was not a scientist, so I did not speak out”. She hit a nerve with me, it is a sentiment that has resonated strongly in me ever since I walked though Auschwitz concentration camp and spent several years living in a country soon after the communists had relinquished power. It is right and proper to speak out for the oppressed, whoever they are and whether we agree with them or not. However, the title of Nicola’s post “Why scientists need to go to the barricades against Trump – and for the humanities” and the first few paragraphs paint the call to march  as a political protest against Trumpian rhetoric and policy.  This, for me, is not an encouragement to march in NZ.  There are many many countries and issues around the world that I abhor and that I think reflect more closely Niemoller’s sentiments– “First they came for the migrants”, “First they came for the children (for the sex trade)”, “First they came for private property” – and I struggle with what I can do about any of them.  However, marching in New Zealand protesting policies in another country is not something I see as effective unless we are demanding action from our government against those countries.

Photo-_Brandon_Wu_(32048341330)

Photo: Brandon Wu 20 Jan 2017 , Wikimedia Commons.

 

Since Nicola wrote that piece, the March organisers have written about the reasons for the March (here and here).  While what has happened in the US is still very much to the fore, the organisers’ attentions seems to have turned towards a protest against policies of the current government “our current government has and continues to be ineffective in defending our native species and environment” (Geni- Christchurch organiser), “The government believes they are improving freshwater, yet they aren’t utilizing NZ freshwater ecology research outputs or freshwater scientists for these decisions.” (Erin-Palmerston North), “you only have to look at the Land and Water forum to open the discussion about the government ignoring the advice of scientists in regards to water quality.” (Steph-Auckland), and on the March for Science websiteThe dismissal of scientific voices by politicians is perhaps best encapsulated by our former Prime Minister’s dismissal of concerns about the impact of our dairy industry on water quality

 

Critique

The organisers in the spirit of peer review invite critique.  My first thought is that if people want to protest the government’s actions with respect to water quality – then please do so.  But, please don’t dress it up as a “March for Science” as if NZ politicians are inherently anti-science.  It comes across as a belief that the NZ Government is tarred with the same brush as the Trump administration with respect to its treatment of science.  I don’t think that comparison is fair.

As an aside, I believe we must be careful with the generalisation “anti-science”, a phrase I’ve regularly heard from the voices and pens of scientists in the past few years.  The phrase has almost always been used to describe people who take stances in opposition to the scientific consensus on matters such as vaccinations, fluoridation, or climate change.  I don’t believe these people are anti-science per se – indeed, they often try (and fail) to use science to back their views. Furthermore, they may well embrace the findings of science in general.  Troy Campbell and Lauren Griffen’s recent post in Scientific America is a good panacea against the loose and pejorative use of the term “anti-science”.

Another aspects of the call to March that I find difficult is the statement “We acknowledge that in Aotearoa New Zealand the scientific community has yet to live up to the principles of Te Tiriti o Waitangi, and that there is an ongoing process of decolonization required to achieve greater inclusion of Māori in the scientific community.” I admit I’m not entirely sure what this means. However, as a member of the scientific community it sounds like I’m being slapped over the wrist.  Further, I feel it is accusing me of some form of racism.  I’m sure this was not the intention, but it is the impression I get and one I don’t like getting.

This is all a pity, as I’d hoped that the March for Science would be more of a celebration with the added value of standing in solidarity with scientists who have been silenced or disenfranchised.  To be fair, celebration is obviously on the mind of some of the organisers such as Cindy from Dunedin “together to celebrate the quest for knowledge and the use of knowledge to protect and enhance life… hope that the March for Science Global initiative will empower scientists and other knowledge-seekers to continue their important work and to share it widely.”  However, this does not seem to reflect the overall tone of the call.

One of the goals of the March is to highlight that “Good scientists can be political.”  I applaud this sentiment and it is something I have tried to be take on board in the past – twice I stood as a political candidate in the general election (2005 and 2008).  Beyond protest, I would encourage all scientists to spend a few minutes with their local MP explaining why and what they do.  The temptation is to bemoan the lack of funding, but I would suggest that funding follows understanding, and we need to engage with politicians and as we do so to recognise the complexity of the decision making with all the competing interests that they have to make.

I began with a question, to march or not to march?  As I’ve written this, I’ve come to the conclusion that, on balance, the call has not resonated with where I’m at, or with what I think of as effective dialogue with politicians, therefore I will not be marching.  I appreciate that others will disagree, nevertheless I wish them a very positive experience.

Aunty Cecily

This international women’s day I read a re-post of a wonderful article about Otago University women in science.  I thought I’d add another one, my Aunt Cecily, or to the rest of the world Dame Cecily Pickerill.

Aunty Cecily was clever, determined, and, yes, a tough woman.   It was those qualities that helped her to help many people.

She was born, Cecily Mary Aroha Wise Clarkson in Taihape in 1903 less than 18 months after her parents had arrived from England. Taihape in those days was forests, mud, a building boom and horses.  It appears to have also been a place she could get a good education.  At a young age, just 18, she made it all the way to Dunedin to attend Otago Medical School.  By then her family was in Auckland.  I don’t know what drew her to medicine. Perhaps it was through world war 1 or the flu epidemic that followed that influenced her. Her own Father had been at Gallipoli as a chaplain with the NZ armed forces during the war and invalided home in late 1915.  Just a year after Cecily started University her parents took her two younger sisters and left New Zealand permanently, ending up in Laguna Beach in California.  Her two, slightly older, brothers remained in New Zealand. She needed to be independent at a young age.

She first came across the art and science of plastic surgery while a house surgeon under the tutelage of Professor Henry Pickerill.  Pickerill was the first director of the Otago dental school. During world war I he became one of the pioneers in facial and reconstructive surgery while with the New Zealand Medical Corp.  Many of the men being treated were transferred to Dunedin at the end of the war.

Cecily spent a few years in California working and living with her family before joining Henry in Sydney in about 1933.  She married Henry at the end of 1934.  Later they moved back to Wellington and both worked as plastic surgeons in Wellington and at Middlemore.   In 1942 they set up Bassam hospital in Lower Hutt for plastic surgery on children – mainly repairing cleft palates and the like.

One of the remarkable features of their work in Bassam was the elimination of hospital cross-infection in children.  They wrote of this in the Lancet in 1954  (Pickerill, C. M., & Pickerill, H. P. (1954). Elimination of hospital cross-infection in children: nursing by the mother. Lancet, 266(6809), 425–429.)

In that article they wrote “what chance of success has a plastic operation on the plate or lip if the child contracts a mixed viral and bacterial infection of the field of operation …”  They noted the lavish use of chromium plating, enamel and wearing of masks… but still there was infection.  The Pickerill’s solution was both simple and innovative – they brought the mother in to nurse the child and gave mother and infant a room to themselves. “Not only do they live together in their own room, but nobody except the mother bathes, dresses, or feeds the patient or changes his nappies.”  This, and other measures, resulted in the remarkable result that after 11 year’s work they had “no single case of cross-infection.”

Aunt Cecily was intelligent, and caring, but also strict (ask my mother about the spider in the bathroom if you want a story about just how strict).  It was that strictness which meant Bassam could be a tight ship and produce such remarkable results.

She was also a woman who loved to travel and garden.  She brought rocks home from travels overseas which ended up as part of her fireplace in a house, Beechdale, designed by my grandfather, in Silverstream.  Her beautiful garden featured in magazines and TV shows.

I recall visiting her in the mid ‘80s at Beechdale when I was in my first job after graduating with a BSc(Hons).  I wasn’t particularly happy with the job at the time.  She was sitting in a comfortable chair in her lounge with a magnifying glass and an open scientific journal.  I realised then, that science and the love of science are for life.

Later when I was doing my PhD on the use of a copper vapour laser to remove birthmarks, I felt even closer to her when one of the patients we treated had had the birthmark partly removed by her surgically.  Many years later a little of it had regrown around the edges which we were able to treat with the laser.

My last memory of her was when she was in her last few weeks of life.  She was in a room in Bassam hospital which was had by then been turned into a hospice.  She had the radio going with some very modern music – which we joked about.  It was fitting that she spent her final days being cared for in the place that she had spent so many days caring for others.

p093-pickerill-cecily-mary-wise-atl-1