Scott Alexander does a deep dive into the studies of effectiveness of ivermectin in treating COVID-19 and he finds, remarkably enough, that there is some evidence that it does, indeed, have some positive effect even if if isn’t the panacea its most ardent supporters may claim. Here’s an interesting snippet from his conclusion:
If you have a lot of experience with pharma, you know who lies and who doesn’t, and you know what lies they’re willing to tell and which ones they shrink back from. As far as I know, no reputable scientist has ever come out and said ‘esketamine definitely works better than regular ketamine’. The regulatory system just heavily implied it.
I claim that with ivermectin, even the people who don’t usually lie were saying it was ineffective, and they were saying it more directly and decisively than liars usually do. But most people can’t translate Pharma → English fluently enough to know where the space of “things people routinely lie about and nobody worries about it too much†ends. So they incredibly reasonably assume anything could be a lie. And if you don’t know which statements about pharmaceuticals are lies, “the one that has dozens of studies contradicting it†is a pretty good heuristic!
If you tell these people to “believe Scienceâ€, you will just worsen the problem where they trust dozens of scientific studies done by scientists using the scientific method over the pronouncements of the CDC or whoever.
So “believe expertsâ€? That would have been better advice in this case. But the experts have beclowned themselves again and again throughout this pandemic, from the first stirrings of “anyone who worries about coronavirus reaching the US is dog-whistling anti-Chinese racismâ€, to the Surgeon-General tweeting “Don’t wear a face maskâ€, to government campaigns focusing entirely on hand-washing (HEPA filters? What are those?) Not only would a recommendation to trust experts be misleading, I don’t even think you could make it work. People would notice how often the experts were wrong, and your public awareness campaign would come to naught.
His final conclusions at the end of a very long post are interesting:
- Ivermectin doesn’t reduce mortality in COVID a significant amount (let’s say d > 0.3) in the absence of comorbid parasites: 85-90% confidence
- Parasitic worms are a significant confounder in some ivermectin studies, such that they made them get a positive result even when honest and methodologically sound: 50% confidence
- Fraud and data processing errors are of similar magnitude to p-hacking and methodological problems in explaining bad studies (95% confidence interval for fraud: between >1% and 5% as important as methodological problems; 95% confidence interval for data processing errors: between 5% and 100% as important)
- Probably “Trust Science†is not the right way to reach proponents of pseudoscientific medicine: ???% confidence
There are a couple of larger issues here. Science is global. How much should we credit studies that aren’t performed in the United States? There may be confounding factors in studies performed in South America, South Asia, or East Asia which physicians in those regions would understand instinctively but which may be quite opaque to us. That doesn’t just pertain to ivermectin. Hydroxychloroquine (HCQ) was used early on in the pandemic by South Korean physicians with extensive anecdotal support and to the best of my knowledge no one has successfully explained that.
Maybe it can all be summarized as “medicine is complicated”.
With US scientists and engineers succumbing to wokeness, I don’t think “performed in the US” is any guarantee of good quality, reliable results. The replication problem is rife (standard? universal?) in the so-called social sciences, but it is a major problem in any discipline related to biology, especially medical and environmental sciences. In fact, out right fraud is common in medicine and environmental science.
The only clean disciplines are mathematics, physics (other than fusion), and chemistry. Those people make mistakes, but they rarely cheat.
https://www.c-span.org/video/?c4930160/user-clip-dr-pierre-kory-senate-hearing-ivermectin-100-cure-covid-19
Given the low cost and the almost non-existent serious side effect profile of ivermectin it seems that not at least trying it – rather than punishing physicians – is morally reprehensible.
Geez. I read all of those, plus a lot more and told you pretty much the same thing. When studies conflict like this it either doesnt work or if it has an effect it will be small.
More substantially I disagree with his take on meta-analysis a bit. Combining a bunch of badf studies does not give you a good one. You are very dependent upon the aggregator placing similar studies into the meta-analysis. In the case of this one the aggregator put in studies that were clearly fraudulent while placing warnings about RCT studies that show it did not work (ivermectin).
So in his individual study review he really comes up with only 3 of any merit. Small studies are useless other than making you think you need a larger one**. Of those 3, two showed no effect and one showed some small effect. That said covid is not exactly a rare disease. Before we completely rule it out I would like to see some truly large, well done studies.
The worms idea is kind of cool. One of our ID people had floated that a while ago. Could be true. I mostly just think that these studies are pretty crappy.
“How much should we credit studies that aren’t performed in the United States?”
Depends. We know that some places like Egypt are notorious for bad studies. Much of the mideast and South America have issues. Literature from most European countries, Australia, Canada and Japan are generally better done. Hit and miss with China. Even then there can be issues. There were a couple of bad groups in Italy 25-35 years ago publishing bad literature.
“Maybe it can all be summarized as “medicine is complicatedâ€.
It is, but we are also dealing with a lot of instant experts. We now have tons of people , including at your site, who have no experience reading medical literature, none of the analytical skills, none of understanding of statistics to know what constitutes a good study vs a bad one*. On top of that it is clear that these “experts” largely dont even read the studies they cite. Some other “expert” has told them the studies are good. The fact is that if something works we notice it almost right away. Day after day and night after night spent with dying people and you really want to find something that helps. We didnt see it with Ivermectin (or HCQ for that matter).
Of note, our ID people announced this week we will stop using ivermectin. They think the literature is now strong enough to not bother catering to people who want it.
* Maybe I misread it, just had eyes dilated, but I think he accepted as a good study one with a p value of 0.3 and another with 0.03. The old convention, some still follow, was a p value of less that 0.05. If you understand what a p value is I think you are much happier with 0.01 at a minimum or better yet 0.001.
** One of the things I teach med students is that if you have a very common disease or problem, like HTN or diabetes, and someone publishes a study with a very small number of subjects that should be a red flag. Why are they only studying 30 people when we have 30 million people with that disease? Dont make any major decisions based upon such a study alone.
Steve
Steve
I thought that was particularly insightful.
The Right To Try Act was passed by the former administration. Why is it then, when people ask for ivermectin citing this law, hospitals still refuse to give it to them? Like the infamous HRC quote, “What difference does it make!â€
Also, Dr. Tyson in the Imperial Valley area of CA, has used ivermectin successfully on over 6000 patients in the early stages of the virus, without a single death. On the ground physicians dealing with COVID are also using antiviral protocols on people and saving lives. I think it’s medical malfeasance to block those who either prescribe or request these alternate ways to medically mediate this virus.
“Given the low cost and the almost non-existent serious side effect profile of ivermectin it seems that not at least trying it – rather than punishing physicians – is morally reprehensible.”
Is it morally supportable to give someone a drug for which there is no sound medical evidence, especially when there are other drugs/interventions which do have evidence based success?
Steve
“Is it morally supportable to give someone a drug for which there is no sound medical evidence, especially when there are other drugs/interventions which do have evidence based success?â€
What is so arbitrary in that question is your interpretation of “sound evidence.†For literary thousands of physicians there is “sound evidence†supporting prescribing specific dosages of ivermectin or HCQ, along with a cocktail of auxiliary supplements, in lieu of a vaccine that some say is riskier to take than actually getting the virus.
“some say”
Some say if you have good evidence you should publish it.
Steve