Author Archives: John Pickering

A reflection on Rising from the Rubble

Nine years ago in my office above the main entrance to Christchurch hospital I was hiding under my desk pushing back at furniture on wheels (whoever thought that was a good idea in New Zealand!) and generally avoiding flying objects. This weekend we remember that February day in which an earthquake cost many their lives and many more their health, their homes and their livelihoods. All who live in Christchurch know that the events of that day and the days afterwards still are with us today. For many of us our homes are still broken. Sadly, many have had their mental health suffer too. When I extracted myself from the office and got down stairs, I watched as the first of the injured arrived in cars and on the back of the utes. As the staff of the health system geared up, I, merely a scientist, wandered off home avoiding the liquefaction and inspecting broken buildings on the way. Recently, I have had the pleasure of reading a book written by a scientist and an emergency physician that has opened my eyes more to the incredible system and people who work in it that we call the Canterbury’s health system.

Rising from the Rubble: A health system’s extraordinary response to the Canterbury earthquakes, has been written by Professor Michael Ardagh, an emergency physician, and Dr Joanne Deely, a scientist. It is published and available through Canterbury University Press.

Rising from the Rubble is indeed an extraordinary story.  It is written in a very engaging manner – purposely with anecdotes and quotes from interviews, but not without athe science and facts and figures to back them up.  The authors’ aimed for the book “to be not only a public record of the Canterbury health system’s response, but a celebration of it.”  They have certainly met that aim.

The book covers a very broad range of response, both in the immediate aftermath of the quakes, but also the ongoing dealing with the myriad issues the quakes threw up for the health system and those involved.  

You will read of courage and heroes (have your tissues ready): 

“We were scared … We were running out and we were hit by a wave of people running in to the building [Princess Margaret Hospital]! They were all staff.  I couldn’t believe it”

You will read of competence and professionalism: 

“speaking calmly and quietly she directed resources to where they were needed”

Dr David Tolley, President of the Royal College of Surgeons, Edinburgh who was in the ED on 22/2 speaking of Dr Jan Bone

You will read of what can be done when it has to be done:

“So we just gave him [a builder son of a GP] the work credit card and he got reticulated water going to about 12 or 13 of the practices in the east and south areas of Christchurch”

You will read of leadership

“He [Dr Nigel Miller, Chief Medical Officer] said, ‘ What are you going to do?’ And I said, “Well we already know what we are going to do.  We are going to get everybody [needing dialysis] out”

Dr David McGregor, Clinical Director, Nephrology Department

You will read of collaboration

“The friendships, connections and collaborations that were forged during the period of integration between Canterbury Health Laboratories and MedLab staff will remain.”

Kirsten Beynon, GM, Canterbury Health Laboratories

You will read of aroha:

“I asked the Maori community if we could include the Asian and migrant communities because they would be outside, to which I got an immediate agreement”

Sir Mark Solomon, Te Rununga o Ngai Tahu

Of course, there is much more.  We were truly blessed by a health system that was so well interconnected and replete with individuals prepared to do what it takes.  While, the consequences of the earthquake remain this book is a reminder of what was and what can be done by people with the right motives and skill.  Dr David Meates, CDHB Chief Executive speaks not of recovery, but of transformation.  This book will be a record of a time of rapid transformation as well as a tribute to all those involved. Do read it.

Cheesecake Files: Machine learning heart attacks

“Machine learning” rates very high on the buzz-word scale, right up there with “nano-technology” and “blockchain”. Like most buzz it is more noise than substance. However, every now and again it looks like there might be something in the noise that bites. This episode of the Cheesecake Files1 is about testing an algorithm (another buzz word) developed through a machine learning technique for the early detection of heart attacks (strictly – myocardial infarction).

The buzz

Before I begin my story in earnest a couple of words about the buzz words. When I say “algorithm” think “recipe”. In the context of emergency medicine this is simply a series of steps which assist the medical team in their decision making. For example – if the presenting complaint is “chest pain” the triage nurse will connect up a device (ECG) to measure the electrical activity of the heart and will draw some blood and send it off the the lab with specific instructions to measure the concentration of a molecule called troponin. Several years ago we introduced in all New Zealand emergency departments more detailed pathways (ie an algorithm) which included guidance on which other data to obtain from the patient, when to repeat blood measurements and how all the data goes together to risk stratify the patient. The principal aim being to ensure that as quickly as possible, and as safely as possible, physicians could rule out the presence of a heart attack. This is important because patients presenting with possible heart attacks are one of the most common presentations to the ED and so if they remain in the ED a long time this can affect the whole service. However, only about 10-15% (in NZ) are actually having a heart attack. Many of those who aren’t can now be reassured early that they are not. Please note – if you’ve sudden onset chest pain then the ED is the right place for you. Just because most who attend are not having a heart attack doesn’t mean that you might not be.

The other buzz word is “machine learning.” This term is usually used to mean a computational technique which involves giving a computer some data and some basic instructions how to look at it. Then asking the computer to make a prediction of an outcome (in our case, whether a patient is having a heart attack or not). The prediction is compared to the actual outcomes and information on how well the computer performs is feedback into the machine to tweak some of the algorithm. Think of this as tasting the soup and then adding a few more spices. The process is repeated many times until the soup is as good as it can possibly be. Some recipes we know and can follow ourselves. Some happen behind closed doors as a team of chefs puts together a meal. A characteristic of machine learning algorithms is that they are often not easily understood (a “black-box”), but the proof of the pudding is in the eating. This leads me to the story that is the current cheesecake.

The story

Nearly three years ago we were asked to test if an algorithm called MI3 works to risk stratify people who appear in the emergency department with symptoms suggestive of a heart attack. The algorithm had been developed by a US based diagnostic company called Abbott Diagnostics. We were given access to the black box and could input variables from real patients and observe the predicted outcome. In this case the algorithm was producing a number that very closely corresponded to the probability of a patient having a heart attack. There were very few variables required to make this prediction – sex, age, two measures of troponin and the time between the two measures. The latter is important because how troponin concentrations change over time informs us about the possible heart attack.

A collaboration of research groups from Scotland, Switzerland, Germany, United States of America, Australia and New Zealand came together to provide sufficient data to test MI3. This group was lead by Christchurch ED physician Dr Martin Than, and Scottish cardiologist, Prof Nicholas Mills. I was charged with pulling together all the data and conducting the statistical analysis of the performance of MI3.

There were about 8000 patients in our testing data set with 10.6% of them having a heart attack. Importantly, the first thing I noted is that the values output by the algorithm corresponded to the true rate of heart attacks. ie when the MI3 value was 5 about 5% of those with this value were having a heart attack, when it was 90 about 90% of people were having a heart attack. In other words, the algorithm was well calibrated – this can give physicians confidence. The second thing was to see if we could find MI3 values below which we could say that almost everyone is not having a heart attack (it’s impossible to be 100% certain – we aim for about 99% or better). We were able to find such a value and show that it identified an impressive 69% of people as low-risk. The full results are available in the cardiology journal Circulation – here.

The application

So, how may this be used? The difference with this algorithm compared with others is three-fold (i) it does not require blood samples to be taken an specific set intervals, (ii) it does not require information about patient history or detailed signs and symptoms to be gathered and incorporated, (iii) and the output is a probability rather than simply stratifying patients to a low, intermediate or high risk category. In other words, the inputs are simple and objective, and the output is easily interpretable. In practice, the physician may receive the MI3 value from the labs along with the troponin results. This may aid discussions with the patient through the use of icon arrays or similar (see the figure).

A concept of how a tool displaying the result of the algorithm may be used to display risk to physician and patient.

1 Once upon a time, a long long time ago, I received a cheesecake for every publication. Sadly, those days are gone now. But I live in hope.

Disclaimer: I have acted as a consultant statistician for Abbott Diagnostics. I have no shares or intellectual property associated with MI3. Abbott was not involved in the testing of the algorithm.

Dark space

Dark space, like green space, is essential for our well being. Dark space, like green space, is our past, our taonga, and our right. Dark space, unlike green space, is not prioritised in our city plans, is not part of our conversation about Te Tiriti, nor is it where we go for relaxation and inspiration. But once it was. Dark space is accessible to us all, if only. If only we turned off the lights and looked up. Dark space is the night sky, the moon, the planets, the stars and galaxies. Once visible to every child, now lost in the haze of light pollution; once the source of wonder and joy; once the inspiration and the starting point for the personal journeys of countless scientists and philosophers, religious leaders and poets.

The week is Matariki. The new year celebrated at the first appearance of the stars known as Matariki – the pleiades. A time for friends and family and celebration. Sadly, even tragically and certainly scandalously most tamariki in Aotearoa will struggle to see any of the stars of Matariki, let alone the nine that are supposedly naked eye objects. This is because we have not acted as kaitiaki of our night sky. We have not guarded it from our modern obsession of pretending night is just an extension of day rather than a time to rest, recuperate, and, yes, gaze on the heavens in awe-filled wonder.

Matariki (Source: http://deography.com/m45-the-pleiades-seven-sisters/)

All is not lost, though. The solutions are in our own hands. As we have preciously defended the green spaces in our city, now we must do the same for dark space. We must insist that our city ordinances are such that lighting that spills light into the dark space is not permitted and that the kinds of lights we employ are not of the most polluting and disrupting kinds. As individuals we can turn down or off our outside lights that blaze away even when we are not there. We must be guardians of the night – protecting not just the taonga that is the view, but the darkness that enables our night loving insects and birds to survive. Matariki is also a time to reflect on our health and wellbeing. Perhaps this year we may reflect on the fact that the blue light emitted by our phones and tablets which we have been warned is detrimental to our health, is the same kind of light emitted by the LED street lights, which cities are rushing to pollute dark space with.

Let us together, all who call Aotearoa New Zealand home, reclaim our kaitiakitanga of Matariki.

(Featured Image by cafuego https://www.flickr.com/photos/cafuego/32719827268)

Performance Based Research Fund: The numbers are up

8269 academics and their bosses have been alerted – the data is in, the numbers are crunched, PBRF scores are out. Who are the winners, who are the losers? Find out more with tec publication. But before you go there…

I predict that any minute now tertiary institutions throughout the land will be posting press releases detailing their successes, each one trying to say “we’re the best” – at least in some category, somehow, if you squeeze the numbers and look at them sideways… well, you get the picture. Speaking of which – here’s one I posted after the last lot of PBRF results were released in 2013.

http://www.scoop.co.nz/education 12 April 2013

Last year I stated that the PBRF quality evaluation was a net zero sum game, because irrespective of the outcomes of the quality evaluations of the 8269 staff there is no net increase in the overall funding (that being set by the Government of the day in the Budget). The total quality evaluation dollar amounts are $173,250,000 p.a. The 2018 PBRF process has shifted where this pie is divided up marginally.

The percentage changes are marginal, but they do translate to a loss or gain to individual institutions. In this round, based on these preliminary results, the “big winners” appear to be AUT, VUW and the non-university sector. The biggest loser appears to be UC, but the others seem to have lost a substantial amount. Of course, as I’ve argued before, these $ amounts must be considered in terms of the total $ costs to the institutions and the government to administer the PBRF, likely millions for each institution. The gains of VUW and AUT aren’t really as much as they look and the losses to the other institutions more.

Was this Quality Evaluation PBRF process worthwhile? – I think not. Will the results be celebrated? – I expect so. Will this post do more to expose the emperor’s new clothes and change the system? – one lives in hope.

Falling into a black hole

To honour Roy Kerr and the first picture of a Black Hole, I have used the wonderful “rayshader” package in R to render the photo into 3D. It’s a bit like an extraordinarily hot volcano. Don’t fall in.

Thanks NASA for the photo (https://www.nasa.gov/mission_pages/chandra/news/black-hole-image-makes-history).

Is my science ethical?

Am I doing what the public thinks is OK to do?  That was the question that came to mind as I heard Bioethics and Health Law expert Rochelle Style speak at a Health data workshop held at the University of Otago Christchurch this week.  Rochelle spoke with clarity and demonstrated a great deal of expertise.  I like the simplicity of Law – tells us what we can do, Ethics – tells us what we should do, and Social license – tells us what the publish think is OK to do.  In this post I want to focus on Social license because when I asked a question about it, it became evident that this was the least understood or investigated of the three legs to ethical decision.

First, an example.  It is legal for me to search twitter for posts about prostate cancer.  If I was to approach the NZ Health and Disabilities Ethics Communities saying that I was doing an observational study of publicly available social media comments on prostate cancer I may not even need to go for ethical review.  However, would it be OK – according to the NZ public – for me to do this?  What if in my report I anonymously quoted some of those twitter posts?  What if I quoted some of those twitter posts and identified the person who made the post?

What is your reaction?  Which is OK?

Now consider this, the print and online media use identified twitter quotes all the time.  They effectively take what is public in one domain to an audience that chooses to follow someone and make it public to another audience who has not made that choice. In addition, consider that twitter uses can block certain people from seeing their tweets.  This suggests to me that the “social license” employed by the media is quite broad. Is it OK?

What if when you visited your doctor you were told by the receptionist that the consultation may be recorded for training purposes.  There was no option to “opt out”.  Would this be OK?  Now, phone your bank or broadband provider.  Did you get an option to “opt out”?  They now have your voice recorded – a biometric that can be used to identify you – as well as details of the conversation.  How long will they keep it for?  Who can listen to it? How is the data stored? How can you ask for it to be deleted?

Hopefully, by now, you see that the concept of social license – what is OK and what isn’t – is not easy to tie down.  For health science, trying to understand what the public think is OK and what they think is not OK is very important, though not well articulated.  

Two organisations that are actively engaged in the discussion about the use of social license for health (and other) data are Te Mana Raraunga , the Māori Data Sovereignty Network and Data Futures Partnership.  Their websites are worth visiting.  Te Mana Raraunga talk of data as “taonga” (a “treasure” for a non-New Zealander reader). This really appeals to me and I think is a good place to start, whether as individuals or communities.  

If our personal data is taonga, what are our rights and responsibilities over this taonga?  How we answer this will depend on our own cultural, social, religious, and other backgrounds.  For me, my data is taonga and I have a responsibility to use this treasure for the greater good – the greater good of those around me, and for generations to come.  This is one reason I volunteer for health studies from time to time.

Regarding data as taonga belonging to individual patients helps me in my role as a scientist. It is not just about making sure I don’t inadvertently reveal something publicly about an individual, but that I utilise ethically and well the data that I have been given temporary guardianship over. For me, this means that I must be vigilant over the statistical techniques I use and cognisant of the most appropriate ways to interpret that data.  Unfortunately, this is a much-neglected area of scientific ethics.  The scientific literature, even in the most prestigious of journals, is full of error of interpretation and of statistical methodology. I regularly see even simple errors in journal articles I referee.  I know I have committed some errors of these myself.  What I believe is most important is that I take responsibility for improving the techniques I use and I improve how I interpret and communicate the results. This is not just about “better science” but about treating the data as taonga, about regarding myself as guardian, not owner, of the data, and recognising that I operate not merely under law and ethical guidelines, but under a social license.  That license may be fluid, ill-defined, and somehow based on diverse views, but nevertheless it exists and needs to be understood and honoured.

The physics of maiming a child (repost because of “those” scooters)

Dear Driver,

When you backed out of a driveway and did not even see how I swerved around behind your car to avoid T-boning you, how dare you have the temerity to tell me you were careful!  I was 7 feet tall, dressed in bright yellow and traveling at no more than 10 km/h.  Perhaps a simple lesson in physics will help you and your fellow “driveway backers” to realise how dangerous you are and to adopt safer driving practices.

In the diagram you can see a car backing out of a driveway.  Typically when you are at the edge of your property and have a fence (see photo below) blocking your view of the footpath you are able to see about 1.7 metres along the footpath.  Let us imagine that there is a child on a trike riding at 5 km/h just out of your line of sight.  How long  does it take them to travel that 1.67 metres?  The physics is quite easy.

5 km/h is 5000 metres in 60 x 60 seconds, ie about 1.4 m/s.  Putting this in the formula above means that it takes about 1.2 seconds for the child to travel that 1.67 metres. 

Now consider this. According to design guidelines for safe bicycle use 2.5 seconds must be allowed for someone to observe the danger, react, apply brakes and stop.  In other words, if you covered the distance from your driveway to the middle of the footpath, about 1 metre, in under 1.2 seconds you will almost certainly hit the child.  That is a speed of just 3 km/h!!!!!

Now consider who else is on the footpath, all legally:

  • Pedestrians 5 km/h
  • Joggers 5- 15 km/h
  • Kids on skateboards or scooters 10 km/h
  • Child on bicycle with small wheels, 10 km/h
  • Mobility scooter, 5-10 km/h
  • Me on my Trikke, 10 km/h
  • Postie on a bike 5-10 km/h.

For those going 10 km/h your speed needs to be just over 1.5 km/h to hit someone! That’s the legal people … but the Lime scooters at 20 km/h mean it all the more necessary to slow down.

So, before you do some damage here is what you can do:

  • Never back out of a driveway unless you really really must.  If you think you must because of the design of your driveway, change the design!
  • Cut back those hedges, remove some of that fence so that you can see further. [ City councils… please make a by-law to make this happen].
  • Always always always stop at the end of your driveway (BEFORE THE FOOTPATH) and toot a horn.  Then proceed very very slowly.

By the way, you are legally obliged to give way:

(1)
A driver entering or exiting a driveway must give way to a road user on a footpath, cycle path, or shared path (as described by clause 11.1A(1)).

Thank you for considering the physics of maiming a child, may you never find your self in such a terrible situation.

Regards,

Dr John Pickering

A typical driveway with almost non-existant visibility
 A typical driveway with almost non-existent visibility

———

Feature Image: Intangible Arts https://www.flickr.com/photos/intangible/ under Creative Commons Attribution 2.0 licence.