More Controversy Over Major Cardiology Clinical Trial

Think about this: A new article reports that a major NIH-funded trial runs into trouble. The article raises all sorts of fundamental questions about our ability to perform meaningful clinical research. But instead of expressing concern about these legitimate problems, medical leaders ignore these questions and instead focus their ire and criticism on the article authors, saying it is improper to attack the trial without first checking with the trial leaders.

The ISCHEMIA trial was designed to answer fundamental questions about an important topic: the diagnosis and treatment of stable angina.  As the trial was nearing completion last year the investigators changed the primary endpoint, which had been the relatively “hard” combination of cardiovascular death and MI. This hard endpoint had been a powerful selling point of the trial. But because ISCHEMIA had trouble recruiting patients— more on that later— they were only able to enroll 5,179 patients instead of the planned 8,000. Because of this change the investigators believed they would be unable to accumulate enough endpoint events to achieve a useful result. To salvage the trial they added the “soft” endpoints of hospitalization for unstable angina or heart failure to the hard endpoints of CV death and MI.

As I reported earlier, the change became public, and controversy ensued, with the publication of an article by Christopher A. Rajkumar and colleagues led by Darrel Francis at Imperial College, London in Circulation: Cardiovascular Quality and Outcomes. They accused the ISCHEMIA investigators of “moving the goalpost.” They pointed out that because patients and doctors in ISCHEMIA were aware of their treatment assignment their subsequent treatments— including things like the new hospitalization for unstable angina endpoint– were likely to have been influenced by that knowledge. This, they argued, meant that the trial would be much less likely to provide a definitive answer to an extremely relevant clinical problem.

The Francis paper was instantly attacked by the ISCHEMIA investigators and other members of the cardiology establishment. The NIH’s Yves Rosenberg expressed “utter stupefaction and profound disappointment” over “grave inaccuracies and mis-statements.” over the paper. One ISCHEMIA investigator, Sripal Bangalore (NYU), immediately dismissed the article for containing “factual errors,” but didn’t offer any details.

The attacks did not focus on the main points of the Francis paper. Instead the attacks focused exclusively on how the endpoint change was characterized in the Francis paper. “On January 17, 2018, over 99% of the way through the recruitment period, an amendment was made to the clinicaltrials.gov website,” the UK authors wrote.

“We must reflect on whether there is any meaning to the term primary end point if these goalposts can be moved at will after a trial has begun recruiting—and especially if a very prominent trial does so shortly before reporting its result,” the British authors wrote. It was this assertion, that the endpoint change had been made capriciously or irresponsibly, which the ISCHEMIA investigators and supporters objected to most.

In short order several important details about the change to ISCHEMIA emerged. The trial leaders publicly revealed for the first time that the endpoint change had been based on a plan outlined in the trial’s original protocol. An Independent Advisory Panel with access to aggregate outcome data, but not outcome data by treatment group, recommended the change, which was approved by the NHLBI in June of 2017. In January 2018 the ClinicalTrials.Gov website was updated to reflect the change. However, there was no public record or explanation for the change.

The ISCHEMIA supporters blame Francis et al for not fact checking their paper with them. “If we had been given the opportunity to vet the information about the trial included in the manuscript prior to the article’s publication, we might have prevented the dissemination of misinformation,” wrote ISCHEMIA trial leaders Judith Hochman (NYU) and David Maron (Stanford) in a published comment in response to the paper.

This of course is true, but it also represents an attempt to deflect responsibility and places an impossible burden on trial critics. In the interests of full transparency, the investigators— of a publicly funded trial, it should be emphasized– could and should have taken the proactive opportunity to announce and explain this major change in their trial. An explanation of the change on ClinicalTrials.Gov or a published statement or press release could have entirely prevented this mixup.

One typical response to the Francis article came from Ajay Kirtane (Columbia University). He expressed grave concern about the publication of the Francis letter, saying that “damage has already been done” and compared its publication, astonishingly, to “current day political affairs.” “We do need to show that we are better than this as a profession going forward,” he tweeted.

He echoed his Columbia colleague Gregg Stone(Columbia University), one of the trial leaders, who tweeted that the journal editors should not have published the “inflammatory/controversial” article without a rebuttal “unless the goal is to emulate the Inquirer.”

Kirtane and Stone were gravely disturbed by the perceived absence of respect paid to the ISCHEMIA leaders but they expressed no concern whatsoever about the larger questions raised by the fact that a major $84 million NIH trial had been weakened or compromised.

The entire episode can be seen as a perfect example of the establishment circling the wagons in a desperate attempt to defend itself from the threat of an attack from those perceived to challenge the status quo.

Francis fired back on Twitter with a devastating response, first noting that for all Kirtane’s and Stone’s insistence on paying due respect to the trial, their home institution, Columbia University, had only enrolled 3 patients in the trial. Kirtane and Stone, in other words, paid far more respect to ISCHEMIA through their words than through their actions.

Francis then went on to enumerate instances in which the trial investigators publicly cited the original hard endpoints as a chief strength of the trial. He even posted a video taken from the ISCHEMIA trial website in which Stone repeatedly cited the hard endpoints as a key indicator of the trial’s strength and integrity. (The video has now been removed without comment or explanation from the ISCHEMIA website.) Here are Stone’s own words in the video:

…as long as there is no difference in hard outcomes, death or myocardial infarction, I believe that revascularization for stable ischemic heart disease is going to be relegated to primarily patients who have failed an initial medical approach.

Francis also blasted critics demanding an immediate correction by pointing to the absence of any similar demand to the many publications from the ISCHEMIA leadership that highlighted the original primary endpoint as a key feature of the trial— without any mention of the alternative plan to expand the endpoint.

One thing needs to be clear: the attacks on the Francis paper deflect attention from the importance and significance of the change in primary endpoint of ISCHEMIA, a change which will significantly impact our perception of the trial and its ability to provide meaningful information. I don’t want to suggest that the trial will now be meaningless, but this change does significantly diminish the trial’s importance. And yet the investigators and defenders of the trial have chosen not to address or explain this key point, though the change was made back in June 2017.

The ISCHEMIA investigators have also chosen not to address a host of other important questions raised by this episode, including questions about transparency, preregistration and prespecification, and the role of ClinicalTrials.Gov.

The controversy should also bring attention to another little discussed but glaring problem, which is the commitment of the cardiology community to support research that raises legitimate questions about common clinical practices. This is, in the current lingo, an existential problem for all of cardiology, not just interventional cardiology, and yet there is very little public discussion about it. As this episode demonstrates, the leaders of cardiology are far more inclined to question the propriety of junior authors criticizing established investigators than they are to address fundamental questions about their profession.

Along these lines, it should be acknowledged that it has long been apparent to nearly everyone in the field that ISCHEMIA would have trouble enrolling patients. I wrote about this problem back in 2009, before the trial even started, and then again in 2015. These articles expose only the tip of a much bigger iceberg.

Regarding this part of the story the ISCHEMIA investigators can be seen as both sympathetic victims and guilty assailants. Hochman and others worked hard, using videos and social media and speaking engagements, to persuade the cardiology community to enroll patients in the trial. On the other hand, they were obviously unable to respond effectively to the resistance they encountered.

No where is this tension within the cardiology community more apparent than the fact, mentioned earlier, that Columbia University, the home institution of co-principal investigator Gregg Stone, only enrolled 3 patients in the trial. In their responses to the Francis paper the ISCHEMIA investigators portray themselves as victims. I am reminded of the old story about the defendant convicted of murdering his parents who then throws himself on the mercy of the court on the grounds that he is an orphan.

Related Reading:

• Cardiology World Erupts Into Controversy Over Change In Major Clinical Trial
• Waiting For ISCHEMIA: Why Won’t Cardiologists Enroll Patients?
• What If PCI Is Just A Sham?
• ORBITA Trial Puts Interventional Cardiologists On The Defensive
• In Defense Of ORBITA