Recent Blow Ups Spark Call For Overhaul Of Clinical Trials


–Trial leaders say they need to get down into the weeds of trial details.

Large international trials are under fire. In recent weeks, as I’ve reported, serious questions have been raised about three major heart failure trials. There is no reason to believe these are the only trials about which questions will be raised or that the general issue of trial reliability will go away any time soon. Further, there is no reason to believe the concerns are limited to heart failure trials or even cardiology trials in general. Instead, these questions almost certainly apply to many large international trials in many different clinical areas.

In response to these concerns clinical trialists are calling for a major overhaul of the way clinical trials are performed and overseen. Safeguards and controls that existed in the past are no longer present, having been removed in the rush to streamline the process and reduce the expense of performing trials, they say. The current system has degenerated, and faith in results can not be taken for granted as in the past.

“There has been a great deal of internal discussion about reform, and interestingly, an unusually high degree of consensus,” said Milton Packer (Baylor University). “Amazingly, this has not been an issue that generates division among clinical investigators; it has actually generated an incredible amount of unity.” Packer promised: “Rely on us to meet together and fix this. We will.”

Immediate Concerns

The immediate concerns about clinical trials were sparked by the TOPCAT investigators, who uncovered compelling evidence that a large number of patients enrolled in Russia never took the study drug, thereby, obviously, undermining the reliability of the trial. Then a new analysis of the recent TRUE-AHF trial found strong evidence that large numbers of patients from Eastern Europe were ineligible for the trial and should not have been enrolled. Finally, the recent presentation of the main results of the RELAX-AHF-2 trial raises the possibility that yet another trial may have been compromised.

John Teerlink (UCSF), the principal investigator of the RELAX-AHF-2 trial, emphasized in an interview that he has not found a “smoking gun” indicating serious misconduct. “We’ve only had a month to analyze the data,” he said, and the findings are “still very preliminary.”

But Teerlink also indicated that there were troubling findings in the trial that are difficult to understand, and that these findings at least raise the possibility that misconduct may have taken place.

Teerlink’s biggest area of concern in RELAX-AHF-2 is a much lower than expected event rate in the trial. This leads to the possibility that, like TRUE-AHF, RELAX-AHF-2 did not enroll patients who truly had acute decompensated heart failure. Teerlink explained that despite the fact that patients in the new trial were even sicker than patients in the previous first RELAX-AHF trial, as clearly indicated in the higher nt-proBNP levels in the second trial, the event rates were much lower than in the earlier trial.

Even if misconduct didn’t occur in RELAX-AHF-2, the fact that it is being considered, and the fact that misconduct almost certainly occurred in both TOPCAT and TRUE-AHF, serve as an indication that faith in clinical trials requires strengthening. Teerlink acknowledged that, his own trial aside, there are important problems in the conduct of large international trials.

[Update, May 15– Dr. Teerlink, who was co-PI with Prof. Marco Metra of the RELAX-AHF-2 trial, has responded that his views are not accurately reflected here. He wants to emphasize that there was no indication of misconduct in RELAX-AHF-2 and that the trial had multiple layers of oversight. He specifically stated that there was no evidence that the problems that occurred at some international sites in the other trials had also occurred in RELAX-AHF-2. “I did not and do not believe that there was any significant clinical trial misconduct in RELAX-AHF-2. To suggest otherwise is an insult to the hundreds of people who worked to insure the integrity of the trial,” said Teerlink. “However, proper monitoring and oversight of sites in international trials is a major issue that needs to be addressed by the clinical trial community.”]

International Perspective

Academic investigators appear to be united in their support for greater oversight and involvement.

Clinical trial experts believe a basic underlying reason for the problems is the movement of clinical trial sites from the US and western Europe to countries and cultures less familiar with the principles of scientific research, including some but by no means all sites in Russia, eastern Europe, and Asia.

Teerlink, echoing the remarks of others, said that there are “no incentives for US and European researchers to participate in trials and enroll patients.” In the US and western Europe the financial incentives to enroll patients in clinical trials are not strong. By contrast, enrolling large numbers of patients in less developed countries can be lucrative, Teerlink said. “Let’s just say 10,000 US dollars goes a lot further in Bulgaria than it does here.” To illustrate the issue Teerlink pointed out that dedicated research nurses are required for participation in research in the US, “we can’t use our regular nurses for research.” In Russia, by contrast, trial payments can go directly to the physician, who may have tremendous discretion about how the money is spent.

G. Michael Felker: (Duke University) expressed support for clinical trials as “a global enterprise” but that it requires “substantial attention” so that “the process of care and the protocol is followed in a consistent way across the globe.”

Felker talked about the pressure to rapidly enroll large numbers of patients in clinical trials. “It’s important that all the participants are aligned toward the same goals, including enrolling the right patients and ensuring high data quality.” Too often, he said, “quantity of enrollment is prioritized over quality.” Felker described a disturbing trend that he has observed during meetings of the leadership of some clinical trials. “They’ll say enrollment is going well, we’re enrolling rapidly, but that shouldn’t be the only metric.” He said there needs to be a far greater focus on quality.

Change will require “getting down into the weeds, so we know more about what patients are being enrolled.” When problems are identified “it’s important to intervene early— because it doesn’t take many patients to dilute any benefit, as we saw in TOPCAT and maybe in TRUE.”

“The key is to have substantial involvement of the academic leadership and not just turn it over to third parties like CROs which may not have the same priorities,” said Felker. But first, he states, there needs to be a general recognition of the problem: “solving any problem means acknowledging what the issue is.”

Chris O’Connor (Inova Heart and Vascular Institute) agreed with Felker, his former Duke fellow. O’Connor has been a leader of efforts in the Heart Failure Society of America and elsewhere to identify the problems and propose solutions. One way to ensure that the proper patients are being enrolled and that they are being treated appropriately is for academic leaders to monitor drug levels, adverse events, and biomarker levels “to make sure there’s no deviation of standards.” To avoid imbalance or other problems they can also consider capping enrollment by region, he suggested.

“We need standardization of the conduct of clinical trials,” said O’Connor.

Further, participating in clinical research “should be part of the training of fellows.” Unfortunately, he noted, “site-based research is not rewarded in academia. You get no reward for enrolling patients at a site.” Another idea is for medicare to offer reimbursement to doctors for participating in clinical trials.”

Degeneration of Standards

In an extensive and wide-ranging interview Packer talked about the problem using the perspective of his long experience in clinical trials. As a veteran of clinical trials dating back to the 1980s Packer talked about the differences between trials performed a generation ago and trials performed today. Like others, Packer supports the globalization of clinical research, but he also believes that without greater oversight this trend is a recipe for disaster. Going forward, the key to resolving these problems and restoring the integrity of clinical trials is rigorous monitoring and oversight of trials, with academic leadership playing a much expanded role overseeing trial operations.

Following the interview Packer sent the following summary of his perspective:

“The 1980s through the early 2000s represented the halcyon period of clinical trials, at least with respect to the development of cardiovascular drugs and particularly with respect to drugs for heart failure. In those days (as Dr. Marc Pfeffer has aptly observed), the principal investigators personally knew nearly all of the investigators. The operations of the trials were carried out by highly qualified people (whether employed by industry or government) who worked full time to achieve and maintain the highest quality control. There were layers and layers of oversight, including having the entire database maintained by an independent third party. David DeMets (University of Wisconsin) and I developed such a model years ago, and it worked extremely well. In 2004, I became a department chair and took a leave of absence doing clinical trials. During the past 13 years, I have been the principal investigator of only two trials: PARADIGM-HF and TRUE-AHF. The PARADIGM-HF trial was carried out using the “old traditional model”; operations were carried out by internal full-time staff, and there was enormous oversight. The Executive Committee continuously reviewed metrics of quality control, and the database was distributed to many independent third parties. In contrast, the TRUE-AHF trial was carried out according to a new model, a model that has only evolved within the past decade but now has become the prevalent model especially for trials that have limited resources. According to this new model, the operations of the trial are outsourced to a private for-profit contract research organization, which no longer interacts with the Executive Committee and often is not even responsive to the sponsor; this organization often controls the database and does not allow any outside independent access. The major measure of trial success is the rate of patient recruitment, and the acquisition and assurance of high-quality data is often a secondary consideration. This new model was also used in the TOPCAT trial; in fact, when the leadership of that NIH-sponsored trial asked to be involved in operations and quality metrics on a regional basis, their request was denied. So in the last decade, the conduct of major clinical trials has changed. Some trials are still done in an old-fashioned way, but in many instances, large-scale trials have become cheaper but far less reliable. In many cases, there is much less oversight; there is no independent supervision of operations; and database access is limited. The ability of the Executive Committee to do its job is often restricted. The goal is to finish the trial rather than obtain a valid answer. This is really unfortunate, but I guess you get what you pay for.”


  1. dearieme says

    But where is the moral agent in this story? The nature of trials didn’t change just because a wizard waved his wand. Which people, which institutions, did this?

  2. Wonderful post! We will be linking to this particularly great post on our website.

Speak Your Mind