Sunday, March 29, 2020

Raoult critique

Coronavirus COVID-19 pandemic is about to be stopped by a stroke of a French “genius” with a history of publishing manipulated data. The charismatic Didier Raoult, director of the Research Unit in Infectious and Tropical Emergent Diseases (URMITE) in Marseille has found a cure: the humble chloroquine, cheap unpatented substance used to treat malaria and autoimmune diseases lupus and rheumatism. The substance so far failed in all antiviral therapies, but this didn’t prevent Raoult from deciding that chloroquine can cure corona virus infections, serious side effects notwithstanding. To prove that, Raoult treated 26 patients at his institution with the derivative hydroxychloroquine, alone and in combination with the antibiotic (meaning antibacterial!) drug azithromycin. The study was not randomised, ethically approved only after it already began, and it was not really controlled: the 16 control patients were treated in different clinics.

After some adjustments (patients removed, data points guessed), a preprint was published simultaneously with a paper in a peer reviewed journal Raoult basically controls…

It is the opposite of proper clinical research practice. The trial started somewhere in March, likely before an ethics vote was applied for. But already on 11 February 2020 Raoult and colleagues decreed which drug can prevent and cure COVID-19…

As Elisabeth Bik discussed, the paper was peer-reviewed in less than 24h, it probably helped that the journal’s Editor-in-Chief is both the paper’s coauthor and Raoult’s IHU subordinate Jean-Marc Rolain. Bik also listed other problems with that paper. Like this:

“In the EU Clinical Trial Register page, the study was described as evaluating PCR data on Day 1, Day 4, Day 7 and Day 14. However, the study show the data for Day 6, which is different than planned. Why did the authors not show the results on Day 7? Did the data on day 7 not look as good?”…

In fact, the authors never showed the results of day 14 either. They also refused to share their secondary endpoint data, namely “the clinical effectiveness of treatment on time to apyrexia, normalization of respiratory rate, and average length of hospital stay and mortality”. Basically, it is none of anyone’s business to know if the therapy had any clinical benefit for the patients…

Even then, additional tricks were apparently needed. Viral load in control patients was analysed by quantitative RT-PCR and a bit too liberally, as Bik explains:

“Of particular note, control patients 6 and 8-16 appear to have been analyzed differently. Their Day 0 PCR values are not given as CT values (the number of cycles after which a PCR becomes positive, the lower the number, the more virus is present) but as POS/NEG, suggesting a different test was used. […] Several patients in the control group did not even have a PCR result on Day 6, so it is not clear how they were counted in the Day 6 result.”..

Some control patients were tested only every second day, afterwards guessed to be positive, some were not tested at all on day 0, as a PubPeer commented noted. And then the authors simply changed the results in control patients between the “in press” and the final paper version, as another PubPeer user observed…

Another PubPeer user re-ran the analysis, because “an important number of non-treated patients were not tested by PCR (ND)“. Once the ND values were disregarded, there wasn’t any more significant difference between controls and chloroquine-treated patients.

Not only control group had to be adjusted: the treatment group somehow lost 6 patients, as Bik discussed:

“Although the study started with 26 patients in the HQ or HQ+AZ group, data from only 20 treated patients are given, because not all patients completed the 6-day study. The data for these 20 patients looks incredibly nice; especially the patients who were given both medications all recovered very fast.

What happened to the other six treated patients? Why did they drop out of the study? Three of them were transferred to the intensive care unit (presumably because they got sicker) and 1 died. The other two patients were either too nauseous and stopped the medication, or left the hospital […] So 4 of the 26 treated patients were actually not recovering at all.”

It seems, the authors simply removed inconvenient patients from the analysis before publishing their study. They left online their previous evaluation though, which showed a slightly different result.
Statistical and ethical problems with that Gautret et al 2020 study were addressed extensively in this preprint by Dahly, Gates & Morris zenodo 2020. The authors also reject Raoult’s claims about previous positive results from China:

“In the background of their paper, Gautret et al referred to “an early clinical trial conducted in COVID-19 Chinese patients, [which] showed that chloroquine had a significant effect, both in terms of clinical outcome and viral clearance, when comparing to control groups”. There were two citations for this claim. The first was a letter7 that doesn’t report any trial data, but instead refers to a conference held in China in February, during which participants (“experts from government and regulatory authorities and organizers of clinical trials”) seemingly agreed that chloroquine was an efficacious treatment for COVID-19. The second cition (also included in the aforementioned letter) refers to a number of clinical trials registered in China, though many of these have now been suspended or closed, while the remaining trials are still recruiting (per their entries on http://www.chictr.org.cn as of March 21, 2020). Hence there are, to our knowledge, no other published reports of clinical trials testing the efficacy of chloroquine for COVID-19 treatment.”

BUT WHAT IF RAOULT IS RIGHT AND IT WORKS?????

I hear you ask that. After all, Raoult is a star of French science: he publishes a scientific paper “nearly every work day”, thanks to 800 employees working under him, as a blog post mentioned, and he is so important that he named two bacteria species after himself: Raoultella planticola and Rickettsia raoultii.

Well, Elisabeth Bik found some very bad data fakery in a 15 year old Raoult-co-authored paper…
Indeed, problems in a paper about a mouse model for typhus got his lab in hot water in 2006. A reviewer for Infection and Immunity, a journal published by the American Society for Microbiology (ASM), discovered that four figures in a revised manuscript were identical to figures in the original manuscript, even though they were supposed to describe a different experiment.

In letters to ASM, made available by Raoult, second author Christian Capo and last author Jean-Louis Mège, a group leader, accepted “full responsibility” for the problem, which they said involved only two figures. Capo, in his letter, wrote that he had made an innocent mistake; Mège wrote that Capo had subsequently failed to show the revised manuscript to other authors, who were on vacation, before resubmitting it. But after consulting its ethics panel, ASM banned all five authors, including Raoult, from publishing in its journals for a year. “We are not entirely comfortable with the explanation provided,” ASM officials wrote to Mège. “Misrepresentation of data … is an affront to the ethical conduct of scientific inquiry.”

Capo and Mège accepted the decision, but Raoult wrote ASM that he wasn’t at fault and that the “collective punishment” was “very unfair.” He appealed the ban, also on behalf of two other co-authors, but lost. Furious, he resigned from the editorial board of two other ASM journals, canceled his membership in the American Academy of Microbiology, ASM’s honorific leadership group, and banned his lab from submitting to ASM journals, in which he had published more than 230 studies. His name has been on only two ASM journal papers since, both published in 2010. To clear his name, Raoult sent his ASM correspondence to French colleagues in 2007, along with a letter defending himself. “If I had been in the United States, I would have sued,” he wrote.”…

Raoult did not like Science coverage of his genius. He made the journal publish an Erratum, where he even wrongly accused the author of shilling for Danone:

“IT WAS AN HONOR TO HAVE MY PROFILE PUBlished in Science (“Sound and fury in the microbiology lab,” C. Mary, News Focus, 2 March, p. 1033). However, I was surprised that 20% of the article is devoted to the American Society for Microbiology (ASM) story, in which I was a collateral victim of a collective sanction (there has been no collective liability in France since World War II). I did not manage the paper and did not even check the last version. The mistake by C. Capo consists of a single figure inversion (not four, as stated in the Science profile). This paper has since been published (1). In January 2007, I was awarded one of the highest ASM honors—the ICAAC lecture—thus clearing doubts about my scientific integrity.”

Now, the extra bit of irony is that Raoult is so full of himself that he published in 2018 an opinion piece to teach France some… research integrity…

Now consider this. Raoult’s past papers show falsified data, which even resulted in his ban by ASM for one year, to which Raoult responded with threats of lawsuit. He is a patriarchal control freak and a misogynous bully who violently punishes all disagreement and uses threats against whistleblowers and victims to achieve compliance. He is pathologically resistant to criticism and believes to be infallible and omniscient:.. Raoult’s new study on chloroquine as the cure for COVID19 is obviously flawed, at best.

Should we really trust his claims and put our all lives in his hands?

3 comments:

  1. Unfortunately, there are many potential problems with the way the data and the peer review process were handled. The discussion is still ongoing on PubPeer, with 2 posts on either the preprint (40 comments as of today) or the published version (3 comments)...

    So 4 of the 26 treated patients were actually not recovering at all. It seems a bit strange to leave these 4 patients who got worse or who died out of the study, just on the basis that they stopped taking the medication (which is pretty difficult once the patient is dead). As several people wrote sarcastically on Twitter: My results always look amazing if I leave out the patients who died, or the experiments that did not work...

    The paper was submitted on 16 March and accepted, presumably after peer review, on 17 March. Assuming the paper was indeed peer reviewed within 24h, that seems incredibly fast. But there might be a reason why this process was so fast.

    One of the authors on this paper, JM Rolain, is also the Editor in Chief of the journal in which the paper was accepted, i.e. the International Journal of Antimicrobial Agents. This might be perceived as a huge conflict of interest, in particular in combination with the peer review process of less than 24h.

    This would be the equivalent of allowing a student to grade their own paper. Low and behold, the student got an A+!

    The journal should make the peer reviews (usually reports from 2 or 3 other scientists who are not authors on the paper or work at the same institutes) publicly available, to prove that a thorough peer-review process indeed took place...

    Obviously, in the case of a viral pandemic there is a great need for new results to become available as fast as possible. It is understandable that the scientific peer review process might be a bit less polished in this case than for other science papers.

    However, the preprint was already publicly available for all, so it does not make sense that this peer review was done in such a rushed manner. It would have been better if the raw data were shared by preprint, with a more carefully written and analyzed, peer-reviewed version being published a couple of weeks later.

    https://scienceintegritydigest.com/2020/03/24/thoughts-on-the-gautret-et-al-paper-about-hydroxychloroquine-and-azithromycin-treatment-of-covid-19-infections/

    ReplyDelete
  2. The lack of randomization limits the conclusions that should be drawn from this study.

    To conclude that the differences in the outcomes observed between the groups receiving different treatment regimens are in fact due to differences in treatment, we need to be able to confidently assert that the groups shared a similar level of baseline risk, prior to treatment.Randomization is of course our best tool for creating such comparable groups, but not applied in this study.This limitation is amplified by the fact that control patients were either recruited from other medical centres, or were patients that refused consent for the active treatment (more on this below). That said, control patients, despite experiencing worse outcomes, were younger onaverage, less likely to be male, and had a more favourable symptom profile, perhaps suggesting lower baseline risk (note: that these differences were not “significant” with a p < 0.05 does imply that they don’t matter with respect to interpretation outcomes). However, it’s entirely plausible that there could be other, unobserved factors leading to a control group with higher baseline risk. Unfortunately, there was no other information about the characteristics of patients, or about the clinicians and medical centres where they were treated, that would help us better judge the comparability of groups.

    Inappropriately included control patients

    Following from the points above, it’s important to note that you would not normally include someone in a trial unless they consented to receive the treatment being tested. That was not the case in this study. Instead, control patients included those who refused consent for the active treatment regimen. The fact that patients who didn’t give such consent is a red flag, not just for the interpretation of the trial but its ethics. We also think it is relevant to note that this is a treatment that the authors have been promoting in the months leading up to this study.

    https://zenodo.org/record/3725560#.XoDTI4hKjIW

    ReplyDelete
  3. Catherine Mary. Sound and Fury in the Microbiology Lab. Science 02 Mar 2012:Vol. 335, Issue 6072, pp. 1033-1035. DOI: 10.1126/science.335.6072.1033

    Summary
    At 59, Didier Raoult is the most productive and influential microbiologist in France, leading a team of 200 scientists and students at the University of Aix-Marseille. He has discovered or co-discovered dozens of new bacteria, and in 2003, he stunned colleagues with a virus of record size, dubbed Mimivirus, the first member of a family that sheds an intriguing new light on the evolution of viruses and the tree of life. Controversial and outspoken, Raoult last year published a popular science book that flat-out declares that Darwin's theory of evolution is wrong. And he was temporarily banned from publishing in a dozen leading microbiology journals in 2006. Scientists at Raoult's lab say they wouldn't want to work anywhere else. Yet Raoult is also known for his enmities and his disdain for those who disagree with him.

    ReplyDelete