More hype over new cholesterol pill ‘bempedoic acid’?
There has been a lot of buzz around a new cholesterol-lowering drug called bempedoic acid. In the US, the drug is approved for people who are intolerant to statin therapy.
Earlier this year, a major trial published in the New England Journal of Medicine of over 14,000 statin intolerant people, found bempedoic acid reduced cardiovascular events (composite of cardiovascular death, nonfatal MI, nonfatal stroke, or coronary revascularisation) by 13% (relative risk) or 1.6% (absolute risk).
The trial had a mix of primary prevention (30%) and secondary prevention (70%) subjects, and the authors were interested to know if either subgroup benefited more from bempedoic acid.
So recently, they carried out a subgroup analysis and published the findings in the Journal of the American Medical Association (JAMA).
The findings
The journal article only disclosed data on the primary prevention group – the secondary prevention group was excluded from the publication.
In the primary prevention group, bempedoic acid reduced cardiovascular events (i.e. cardiovascular death, nonfatal MI, nonfatal stroke, or coronary revascularisation) by 30% (2.3% absolute reduction) compared to placebo.
The authors then carved up the data further to show bempedoic acid reduced cardiovascular death by 39% (1.3% absolute reduction) and all-cause death by 27% (1.6% absolute reduction) compared to placebo.
The size of the effect, which rivals even some of the most effective medications, immediately raised suspicions since the main trial showed no such benefit in these endpoints.
Nonetheless, over 80 news outlets covered the story, including interviews with the lead author of the study Steve Nissen, a cardiologist from Cleveland Clinic.
Nissen told reporters that the new results were “frankly striking” and a "wake-up call for the medical community that we need to pay far more attention to high-risk primary prevention patients."
Critics push back
Despite the enthusiastic press coverage, notable researchers were quick to point out flaws in the analysis.
- Subgroup analyses
John Mandrola, a cardiac electrophysiologist at Baptist Health in Kentucky, was particularly critical of the methods used in the analysis.
“I'm going to try and stay calm through this,” said Mandrola. “I learnt very early on in my career that you cannot slice data the way they did.”
“You could excuse this if they were junior researchers who didn't know better,” he said, “but the authors clearly know it's a dubious analysis. The JAMA editors know it's dubious, and yet there it is, published and promoted in media.”
Mandrola said normally trials are designed with a certain number of participants in order to sufficiently power the study to show statistical significance.
“But when you take a subgroup, you're just taking a small slice of the pie, and if you're cutting it up, you will have a smaller number of events, and it's going to be more way more susceptible to [statistical] noise,” said Mandrola.
“It's hard not to be cynical about the research profession, when you see stuff like this in the highest-level journals,” he added. “I’m really worried about the state of adjudication of science.”
Vinay Prasad, a haematologist oncologist at San Francisco General Hospital agreed with Mandrola saying, “It's very rare for a top journal to publish a subgroup analysis of a paper because it's considered low credibility. I think it's shocking.”
Prasad added, “Another way to think about it is if you get a Jackson Pollock painting where he splattered paint all over the canvas. And if you allow me - after the fact - to cut out the part that's green, I can cut out pieces that look whatever colour you want, it doesn't mean it's a real signal.”
Rita Redberg, cardiologist, and former editor of JAMA Internal Medicine, agreed.
“If you want to show the benefit of a drug in healthy people in primary prevention, then you should do a trial in that population. I don't think doing a subgroup analysis is reliable,” said Redberg.
“If you have enough subgroups, you're going to find some subgroups where it will look like the drug works,” added Redberg.
In fact, Redberg pointed to the ISIS-2 study which famously demonstrated the fallibility of subgroup analyses.
ISIS-2 was a large trial that reported the benefits of aspirin on heart attacks. But during the peer-review process, The Lancet insisted the researchers divide the patients into subgroups and detail which ones had benefited and which ones had not.
The researchers regarded it as “bad science” but decided to comply. Cheekily, they included a subgroup analysis based on participants’ astrological star sign and found only Geminis and Librans did not benefit from aspirin.
To this day, the study serves as a stinging reminder of why subgroup analyses are scientifically dubious and should require confirmation with a properly designed RCT of that particular subgroup.
- Missing data
The alarm was raised when the authors did not publish the secondary prevention results in the same paper.
James McCormack, a professor in the Faculty of Pharmaceutical Sciences at the University of British Columbia said, “There was not a single mention of the secondary prevention group in the JAMA subgroup analysis.”
“If you decided to do a subgroup analysis on a large trial which included different sexes for example, why would you only publish data on the male subgroup and exclude the female subgroup? It doesn’t make sense,” said McCormack.
Nissen responded to the criticism saying that “including both subgroups would have generated a manuscript much too large for a single paper” and that “additional presentations and manuscripts describing the secondary prevention findings are nearing submission.”
McCormack did not accept Nissen’s explanation, saying it was a “ridiculous” excuse.
“It would’ve been easy to present both subgroups in the same paper.” said McCormack. “It should have been picked up in peer-review, but then again, peer-review is not perfect.”
I asked Nissen if the peer-reviewers had requested the secondary prevention group data before publication, but said he could not answer my query “due to the strict ethical rules about the confidentiality of peer review.”
In the absence of this data, McCormack did his own analysis by subtracting the primary prevention group from the overall data to get results for the secondary prevention group.
By his calculations, McCormack estimated that the drugs benefit in reducing cardiovascular events was lost in the secondary prevention group, which he says, doesn’t add up.
“It’s really strange a drug would work in primary prevention, and not work in secondary prevention because in secondary prevention people are at a higher risk of heart attacks,” said McCormack.
Even Gregory Curfman, executive editor of JAMA raised doubts about the findings of Nissen’s analysis during an interview with Dhruv Kazi, a cardiologist at Beth Israel Deaconess Medical Center in Boston.
Curfman said:
“It’s puzzling at least to me, that there would be a reduction in cardiovascular mortality and overall mortality in the primary prevention subgroup, but not in the overall trial of all the patients, suggesting that there might have even been an increase in these mortality endpoints in the secondary prevention patients.”
Kazi replied:
“I agree with you. This is somewhat conflicting to see the signal of pretty substantial benefit in mortality in the primary prevention arm versus no evidence of benefit in the overall trial and possibility even of harm in the secondary prevention arm I think that this temptation to think that bempedoic acid is somehow more effective in primary prevention than in secondary prevention and may even be more effective than statins is one that we should forego. I think that conclusion would be incorrect.”
- Benefits not mediated via cholesterol-lowering
Finally, McCormack said bempedoic acid reduced LDL-cholesterol to an equivalent degree in both the secondary and primary prevention groups yet, the cardiovascular outcomes for each group were not equivalent. This suggests that if there is an effect from bempedoic acid, it’s not because of its ability to lower cholesterol.
When I asked Nissen to explain the paradox he simply replied, “We were careful not to make any claim about the mediation of the results.”
McCormack said, “I think this puts Nissen in a corner. He does not address this in the paper because it requires an explanation that goes against the whole LDL hypothesis which is difficult for some researchers to accept.”
“I’m not interested in whether a drug lowers cholesterol, I’m only interested if it lowers cardiovascular outcomes. There are a number of medications that lower cholesterol, but they don’t change cardiovascular outcomes,” added McCormack.
Researchers now await Nissen’s upcoming paper which they hope will give more context to the latest controversial analysis.
Original version published on Broken Science Initiative
Also, see previous article on original trial of Bempedoic Acid
They will do anything to sell their drugs.
The pitfalls of subgroup analysis was demonstrated in the Bangladesh mask study which showed a larger variance in outcomes (Covid cases) due to differences in mask colour than there was with mask versus no mask groups.
Big Pharma is big business.
Vertical integration ("owning" the universities, researchers, medical journals and the media which publish the "findings" without any scrutiny) is the favoured business model.