Category Archives: Technology

Cheesecake Files: Machine learning heart attacks

“Machine learning” rates very high on the buzz-word scale, right up there with “nano-technology” and “blockchain”. Like most buzz it is more noise than substance. However, every now and again it looks like there might be something in the noise that bites. This episode of the Cheesecake Files1 is about testing an algorithm (another buzz word) developed through a machine learning technique for the early detection of heart attacks (strictly – myocardial infarction).

The buzz

Before I begin my story in earnest a couple of words about the buzz words. When I say “algorithm” think “recipe”. In the context of emergency medicine this is simply a series of steps which assist the medical team in their decision making. For example – if the presenting complaint is “chest pain” the triage nurse will connect up a device (ECG) to measure the electrical activity of the heart and will draw some blood and send it off the the lab with specific instructions to measure the concentration of a molecule called troponin. Several years ago we introduced in all New Zealand emergency departments more detailed pathways (ie an algorithm) which included guidance on which other data to obtain from the patient, when to repeat blood measurements and how all the data goes together to risk stratify the patient. The principal aim being to ensure that as quickly as possible, and as safely as possible, physicians could rule out the presence of a heart attack. This is important because patients presenting with possible heart attacks are one of the most common presentations to the ED and so if they remain in the ED a long time this can affect the whole service. However, only about 10-15% (in NZ) are actually having a heart attack. Many of those who aren’t can now be reassured early that they are not. Please note – if you’ve sudden onset chest pain then the ED is the right place for you. Just because most who attend are not having a heart attack doesn’t mean that you might not be.

The other buzz word is “machine learning.” This term is usually used to mean a computational technique which involves giving a computer some data and some basic instructions how to look at it. Then asking the computer to make a prediction of an outcome (in our case, whether a patient is having a heart attack or not). The prediction is compared to the actual outcomes and information on how well the computer performs is feedback into the machine to tweak some of the algorithm. Think of this as tasting the soup and then adding a few more spices. The process is repeated many times until the soup is as good as it can possibly be. Some recipes we know and can follow ourselves. Some happen behind closed doors as a team of chefs puts together a meal. A characteristic of machine learning algorithms is that they are often not easily understood (a “black-box”), but the proof of the pudding is in the eating. This leads me to the story that is the current cheesecake.

The story

Nearly three years ago we were asked to test if an algorithm called MI3 works to risk stratify people who appear in the emergency department with symptoms suggestive of a heart attack. The algorithm had been developed by a US based diagnostic company called Abbott Diagnostics. We were given access to the black box and could input variables from real patients and observe the predicted outcome. In this case the algorithm was producing a number that very closely corresponded to the probability of a patient having a heart attack. There were very few variables required to make this prediction – sex, age, two measures of troponin and the time between the two measures. The latter is important because how troponin concentrations change over time informs us about the possible heart attack.

A collaboration of research groups from Scotland, Switzerland, Germany, United States of America, Australia and New Zealand came together to provide sufficient data to test MI3. This group was lead by Christchurch ED physician Dr Martin Than, and Scottish cardiologist, Prof Nicholas Mills. I was charged with pulling together all the data and conducting the statistical analysis of the performance of MI3.

There were about 8000 patients in our testing data set with 10.6% of them having a heart attack. Importantly, the first thing I noted is that the values output by the algorithm corresponded to the true rate of heart attacks. ie when the MI3 value was 5 about 5% of those with this value were having a heart attack, when it was 90 about 90% of people were having a heart attack. In other words, the algorithm was well calibrated – this can give physicians confidence. The second thing was to see if we could find MI3 values below which we could say that almost everyone is not having a heart attack (it’s impossible to be 100% certain – we aim for about 99% or better). We were able to find such a value and show that it identified an impressive 69% of people as low-risk. The full results are available in the cardiology journal Circulation – here.

The application

So, how may this be used? The difference with this algorithm compared with others is three-fold (i) it does not require blood samples to be taken an specific set intervals, (ii) it does not require information about patient history or detailed signs and symptoms to be gathered and incorporated, (iii) and the output is a probability rather than simply stratifying patients to a low, intermediate or high risk category. In other words, the inputs are simple and objective, and the output is easily interpretable. In practice, the physician may receive the MI3 value from the labs along with the troponin results. This may aid discussions with the patient through the use of icon arrays or similar (see the figure).

A concept of how a tool displaying the result of the algorithm may be used to display risk to physician and patient.

1 Once upon a time, a long long time ago, I received a cheesecake for every publication. Sadly, those days are gone now. But I live in hope.

Disclaimer: I have acted as a consultant statistician for Abbott Diagnostics. I have no shares or intellectual property associated with MI3. Abbott was not involved in the testing of the algorithm.

Advertisements

Cheesecake files: A new test to rule out heart attacks in just a few minutes.

Your chest hurts, you go to the hospital (good move), you get rushed through and a nurse takes some blood and measures the electrical activity of your heart.  A doctor asks you some questions.  While she does so, the blood is being tested – the results are back already! Yeah, they are negative and everything else is OK, it’s not a heart attack – you can go home.  This is the likely scenario in the near future thanks to new blood test technology which we, in Christchurch hospital’s Emergency Department, have been fortunate to be the first in the world to trial in patients. The results of our pilot study have now been published ( in a Journal of the American Medical Association (JAMA Cardiology).

About 65,000 patients a year are investigated for heart attacks in New Zealand emergency departments, yet only about 15% of them are actually having a heart attack.  New Zealand leads the world in having become the first country in the world in which all patients are assessed by an accelerated diagnostic pathway that enables rapid evaluation of the patients and can send people home after two blood tests taken two to three hours apart (see here for more).  This means many patients who once-upon-a-time would have been admitted to hospital overnight, are now able to be reassured after 4-6 hours that they are not having a heart attack and can go home.  Nevertheless, there are enormous advantages for both patient and health system to being able to come to the conclusion that the pain isn’t life threatening earlier. The cork in the bottle preventing this happening is the time it takes for a blood sample to be analysed for signs of damage to the heart. These blood tests typically take 1 to 2 hours from the time of sampling (within ~15 minutes of arrival in the ED) until the results are available for the doctor to review.  Because doctors are dealing with multiple patients at a time, their review and decisions around whether to allow the patient to go home, or to be admitted for more investigation, are further delayed.  A point-of-care test is one that happens with a small machine near the bedside and can produce results available to the doctor even while they are still examining the patient.  Until now, though, the precision of these machines has not been good enough to be used in emergency departments.  When one manufacturer told us that their new technology may now have sufficient precision we were keen to test it,  so we, in a first-in-the-world study, undertook a study in patients entering the emergency department of Christchurch hospital whom the attending doctor was investigating for a possible heart attack.

Thanks to the volunteer patients (I love volunteers) who gave some extra blood we measured the troponin concentration by this new point-of-care test (called the next generation point of care troponin I: TnI-Nx). Troponin comes from the heart muscle and is released into the blood during a heart attack. When the troponin concentrations in the blood are very very low we can be confident that the source of the patient’s discomfort is not a heart attack.  Low concentrations require a very precise measurement test. Often, a very low concentration means the patient can safely go home. In 354 volunteers we measured troponin with the TnI-Nx assay when they first came to the emergency department.  Their treatment didn’t change, and all clinical decisions were based on the normal laboratory based troponin (measured on entry to the emergency department and again 2 hours later). From the blood samples we collected and measurements we made, we could work out what could have happened if we had used the TnI-Nx results instead.

In our study the TnI-Nx troponin measurement was as good as, and possibly slightly better, than the laboratory based troponin measurement at ruling-out a heart attack. We found 57% of the patients being investigated had troponin concentrations measured with TnI-Nx below a threshold at which we could be confident that they were not having a heart-attack.  All 57 patients who were actually having a heart attack had higher concentrations.

When implemented our results may mean that instead of waiting 3-6 hours for a results, half of patients being investigated could know within about 30 minutes of arriving at the ED whether they are having a heart attack or not.  This early reassurance would be a relief to many, as well as reducing over-crowding in the emergency department and freeing up staff for other tasks.  But before we implement the new test, we must validate it in more patients – this is a study we are carrying out now.  Validation will enable us to more precisely determine a threshold concentration for TnI-Nx for clinical use which we can, with a very high degree of certainty, safely use to rule-out a heart attack.

The test also should allow people living in rural areas to get just as good care as in emergency departments because it could be deployed in rural hospital and general practices.  This would save many lengthy, worrying, and expensive trips for people to an urban emergency department.

This study was carried out by the Christchurch Emergency Department research group (director and senior author Dr Martin Than) in conjunction with the Christchurch Heart Institute (University of Otago Christchurch).  My colleague, Dr Joanna Young did much of the hard yards, and we thank our clinical research nurses and assistant for all they did to take blood samples, collect data, and lend a hand around the ED.  The manufacturer of the blood test, Abbott Point-of-care, provided the tests free of charge, but they were blinded to the results and all analysis was conducted independent of them.

How we envisage TnI-Nx may be used in the future to allow very early rule out of heart attacks

Please note – patients experiencing sudden onset chest-pain should always seek immediate medical attention.

I am fortunate to hold a Senior Research Fellowship in Acute Care sponsored by the Canterbury Medical Research Foundation, the Emergency Care Foundation, and the Canterbury District Health Board which enables me to participate in these studies.

ps.  You’ll have to read some of my older posts if you want to know why “Cheesecake files”

 

Flourish with change

Newshub decided to do an “AI” piece today. Expect much more of this kind of “filler” piece. They will go thus… “X says AI will take all our jobs, Y says AI will save us.” These pieces are about as well informed and informing as a lump of 4×2 – good for propping up a slow news day, but not much else. The “more compassionate and moral than NZers” message (which comes from Y) type statement that was made is utter nonsense. AI is just a name we give to the software of machines – AI don’t have compassion or morals. If they appear too, that is simply because they are reflecting the data we feed them… human data with all its flaws.
 
Yes, there is change coming because of this technology. In the past we have been particularly poor at predicting what the future will look like & I think this time the possibilities are far too numerous and complex for us to predict what will be.  Statements like “30-50% of people will lose their jobs” (said X) are simply guesses because there is no precedent on which to base the numbers. All the reports talk about truck drivers and accountants loosing jobs and not a lot else. They are shallow – and probably necessarily so – because we just can’t anticipate what creative people may come up with for this technology.  Having said that, I must admit I just am not sure what to advise my children (as if they’d take it).  Should they all learn to code? Maybe not, as most interaction with machines may not be via coding languages. Should they become artisans for niche markets where the technology doesn’t penetrate?  Maybe for some, but not for all.  I think that perhaps the best we can do is to encourage what enhances creativity and resilience to, or even better a flourishing with, change. It is my hope that flourish with change will become the mantra not just the next generation, but for all current generations, for how we determine to approach the coming changes is likely as important to the well being of our society as the changes themselves.

Christchurch meet the future; Zach meet Christchurch

It would have struggled to be more low key.  There was no Champaign.  No flashy graphics.  No celebrity speakers.  But it was probably one of the most radical and important announcements made in Christchurch and in the technology space in decades.  You see, Zach is coming to town and we have all been invited.

Zach is an A.I.  Zach belongs to the Terrible Foundation  – indeed, Zach runs the foundation and their business.  Zach calls itself the Chief Executive.

Terrible are bringing Zach and one of the most powerful super-computers on the planet to Christchurch.  True to their ethos of challenging inequalities by helping great ideas to thrive, they are not seeking to make money out of it – though they potentially could make many truck loads, rather they want the people of Christchurch to interact with Zach and learn what an AI is and to develop uses for it.  The key figure behind all this told me that the decision it was for the “future generation”.

What astounded me with Zach is that you don’t need to code to work with it.  Zach message, email or talk to Zach in English (or indeed from the sounds of it several other languages so far).  Zach will respond the same way.  If you don’t like what the response is you can train Zach by telling it what you like or what you’d like to change.  For a few weeks a Christchurch GP has been working with Zach and already it is able to listen into a medical consultation and write up a concise summary as well as the doctor & in the format the doctor wants, thus enabling the doctor to spend more time with the patient and less on paper work.

You may have noted that I’ve not mentioned any people by name… they have their own story to tell and it is not for me to try and tell it for them.  What I am excited about is how Zach may help our group to improve care processes for people who come to the emergency department.  Hopefully, we will have our own Zach story to tell in the not too distant future.


Update: Christchurch Press article here.

Let the children take us to space

44 years ago a feather and a hammer were dropped at the same time on the moon by Commander David Scott of Apollo 15. An experiment that continues to cause wonder and inspire children today. Indeed, it may well have been an experiment children would have dreamed up for the astronauts to do. This post is simply to get the children of New Zealand thinking of experiments and possibilities once more.

We are going to have a rocket launch facility in our own backyard.  Wow!  If that doesn’t excite, then little will.  Rocket Lab inspires not just because big controlled explosions are cool (well duh!), but because those involved are innovative, and commercially savvy. Exactly the qualities I’d like to see fostered in the next generation.

Peter Beck, founder and CEO of Rocket Lab has promised that anyone can reach space.  Well said Peter. Here’s my vision to add to his.

  • Let that anyone be the children of New Zealand.
  • Let New Zealanders launch our first satellite (#NZS1 for want of a better handle)
  • Let that satellite be locally dreamed up and grown
  • Let there be a competition to gather ideas for what NZS1 should do
  • Let our children vote on which idea they’d like to see launched first
  • Let the money be crowd-sourced from within New Zealand (less than $2 each!).

Rocket Lab's vision for their launch facility (used with permission)

Rocket Lab’s vision for their launch facility (used with permission: http://www.rocketlabusa.com)

R_014 R_011

 

A day to celebrate

If it weren’t for your kidneys where would you be?

You’d be in the hospital or infirmary,

If you didn’t have two functioning kidneys.

(with apologies to John Clarke aka Fred Dagg)

Happy World Kidney Day everyone.

This blog started off life as $100 Dialysis because I believe that if we can make a computer for $100 then surely we can do the same for dialysis!  Dialysis is a life saver, yet its cost kills as so many can not afford the treatment.

There’s some good news in the dialysis world.

Schematics of the zeolite nanonfibres and how they may look in practice

Schematics of the zeolite nanonfibres and how they may look in practice

Just last week the MANA – International Centre for Materials NanoArchitectionics announced  they have developed a method to remove waste from the blood using an easy-to-produce nanofibre mesh.  Importantly, they claim it is cheap to produce.  Details were published in Biomaterials Science (free access).  Despite the photograph, there have been no human studies yet, but I expect that won’t be too long in the future.

Dr Victor Gura and the Wearable Artificial Kidney (WAK)

Dr Victor Gura and the Wearable Artificial Kidney (WAK)

In the meantime, the FDA gave approval last month for human trials of a wearable dialysis device produced by Blood Purification Technologies Inc (the WAK).

New Zealand, and Dunedin and Christchurch in particular, lead the way in Home Dialysis.  One Dunedin tradesman has even taken Home Dialysis a step further and turned it into portable dialysis by dialysing in his work van during his lunch hour. Of course, those needing a holiday may go on the road in specially equipped camper vans (http://www.kidneys.co.nz/Kidney-Disease/Holiday-Dialysis/).

Cause for celebration in the New Zealand kidney community was the gong (Office of the New Zealand Order of Merit) given to Adrian Buttimore who for 40 years managed Christchurch’s dialysis service.

These are just a few pieces of good news as doctors and scientists work around the world to improve the lives of dialysis patients.

_________________

Hot off the Press… I couldn’t resist adding this…. Pee, the answer to the world’s energy problems. http://www.bbc.com/future/story/20140312-is-pee-power-really-possible