Fat Mate: Fat chance lost

I expect New Zealand tax dollars science spending to be better than what I read in the media this morning. Headlined in the Press was “Blenheim ‘fat mate’ loses 13.5kg in 8 weeks.”  The story was of someone on a trial of a locally produced diet supplement having lost weight.  So far nothing to peak my interest, but then I came across the statements “Satisfax was the result of $12 million research over four years with support from Crown Research Institute Plant & Food Research.” (Satisfax is the trademark)  and “Huge demand for the trial saw it expanded from 100 “fat mates” to 200.”  The second of the links goes to an October article in the Marlborough express which includes the statement that “The trial had been approved by the Health Ministry’s health and disability ethics committee and was partially funded by Callaghan Innovation.

So, your and my taxes are being spent by Callaghan Innovation on a trial of a diet pill the development of which received other tax dollars through Plant & Food. Worth a little more investigation.  The trial went through an ethics committee – big tick.  It was also (a little late) registered on the Australia New Zealand Clinical Trials network (here) – tick.

BUT, it fails miserably as an efficacy trial.

There is no control group.  ie the pill is not compared against a placebo. I can think of no practical reason why there was not a control group taking a placebo (randomised and blinded of course).  Instead, the trial just looks at the average change in weight change over eight weeks and tries to establish if this is non-zero.  Given that these people are doing something hoping to loose weight, there may well be an average loss of weight that has nothing to do with the pill. The press article suggests a biostatistician is going to somehow “account” for the placebo affect (something not mentioned on the trial registration).  I pity the biostatistician as this is involves trying to convince someone that a study run elsewhere with a placebo group, at a different time, under different circumstances could actually serve as a control for this study.

Incredibly, that is not the only major issue.  I read that part way through the trial the publicity was such that there was a demand from people to enter the trial and so they doubled the number of participants from 100 to 200.  Ahhhhhh….. this is classic introduction of bias and should never have been allowed.  Those extra 100 people are not the same as the first group… they have elevated expectations that may well bias the results.  Furthermore, it is always dangerous to talk about the trial efficacy part way through as this may influence the behaviour of those already in the trial.   Grrrrr….

In short – a chance lost and waste of Plant and Food & Callaghan Innovation funding.  There is hope though – a proper randomised controlled trial could be conducted.  But, I won’t be holding my breath.

ps.  The Marlborough Express and Press should be ashamed of such blatant product placement – diet pills on January 2nd are so cliche.  I wonder if it was the reporter or the company who initiated this piece?

5 thoughts on “Fat Mate: Fat chance lost

  1. Pingback: Is this being sold to people who care if it works? | Stats Chat

  2. Glenn Vile

    The Fat Mates trial was designed by clinical trial specialists to generate information about the Satisfax® capsules that would help Tuatara Natural Products plan a larger and longer double blind, cross over, placebo controlled trial. While the Fat Mates trial did not include a placebo controlled treatment arm the weight loss achieved by most of the participants who complied with the Satisfax® dose recommendation was so great that we got our independent bio-statistician to compare the weight loss achieved by these Fat Mates with the weight loss achieved with placebo treatments in other similar, published, weight loss trials.

    Our bio-statistician concluded that the average weight loss due to the placebo effect in those trials was less than 1.0 kg, much less than the average weight loss of 2.9 kgs of the Fat Mates who complied with the recommended dose during the Satisfax® trial.

    We will use this information to proceed with the next clinical trial, but in the meantime we were so excited the weight loss achieved by most of our Fat Mates was much greater than the placebo effect seen in other weight loss clinical trials that we decided to launch the product so that anybody who is overweight can try Satisfax® for themselves.

    The trial recruitment was increased to 200 applicants prior to the trial commencing, so expectations of all participants would have been the same, and would not have not introduced bias as mistakenly concluded by Mr Pickering.

    Current strategies to stem the obesity epidemic have not been effective. The research that has been conducted by Plant and Food Research and supported by Government funding, including Callaghan Innovation, has identified a physiological mechanism and potential treatment that our initial trial has shown to be extremely effective in some overweight people. The full trial will hopefully confirm Satisfax® as an important tool in the fight against the obesity epidemic.

    1. Mark Hanna

      Hi Glenn, I’m glad to hear that this trial was designed to help you plan a larger and more rigorous trial, and that you still intend to conduct such a trial. This is precisely how preliminary research such as this should be used. However, I also think that in your excitement you may have overstepped yourself in using this preliminary research to promote the product in the media.

      For example, I’m not sure that you’re justified in saying that this trial “has shown [Satisfax to be extremely effective in some overweight people]”. After all, you didn’t conduct a rigorous trial in overweight people that showed it to be extremely effective. You conducted a preliminary trial on people of varied weights that showed varied results. If you cherry pick your data, for example by selecting only those who lost the most weight during the trial period, you cannot draw reliable conclusions.

      I’m sure that comparing the data gathered in the “Fat Mates” trial to placebo data from other trials will be better than making no comparison at all, but there are still sources of bias in this approach and you should be ready to acknowledge its limitations. For example, all participants in the “Fat Mates” trial would have known they were receiving the experimental treatment, whereas participants in participant-blinded and placebo-controlled trials would not have known whether they would receive the experimental treatment or the placebo.

      Basically I’m glad the research is being done, but I think the promotion of this product in the media is getting too far ahead of the research.

  3. Pingback: When Marketing Trumps Science | Honest Universe

Got something to say? Don't be shy

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s