–Doctors who got famous for doing clinical trials resist changes to the clinical trial system.
It should probably come as no surprise that hundreds of clinical trial investigators whose positions and livelihoods depend largely on the existing clinical trial system have expressed great reluctance and annoyance at a proposal that could radically shake up that system.
As I recently reported, two large groups of well known and highly influential clinical trial investigators, representing hundreds of investigators around the world, wrote in the New England Journal of Medicine about their grave concerns about data sharing.
They say that they actually favor data sharing, or at least its better aspects, but their support for data sharing is half-hearted at best. A careful reading of their articles makes clear where their hearts and values really lie. Clearly they are far more concerned about disruptions of the system from which they have benefited than they are about the faults and weaknesses of that system. So in their papers they call attention to potential or theoretical dangers of the data sharing proposals but they complete ignore the all too obvious fact that the current system is unsatisfactory and on the edge of failure. In this they are not unlike legislators who propose drastic voting restrictions, claiming that they are addressing or preventing a serious problem. But of course the underlying motivation is clearly quite different.
This doesn’t mean that there aren’t legitimate problems with data sharing, and these problems could and should have been addressed within the context of a much broader and enthusiastic support for data sharing. But this is not the case. The papers suggest that these clinical trial leaders may grudgingly come to accept data sharing– they are nothing else if not practical and pragmatic– but they don’t support the concept and they will do everything in their power to weaken the proposals and mitigate the damage to themselves and their entrenched interests.
Nowhere is their perspective more clear than in their efforts to extend the length of time after initial publication of a trial when the data must be shared from six months in the original proposal to two to as long as five years in some cases. It seems clear that these delays would almost completely weaken the point of the original proposal. There’s simply no good reason why the trial investigators can’t perform these analyses faster than newcomers.
I think it’s also worth noting that this is very likely a spurious fear. I don’t think a lot of outside researchers are going to try to jump in and act like the “data parasites” they seem to fear. Clearly if the original investigators act in a reasonable amount of time they will have an enormous advantage. My sense is that the problem of competing analyses will occur only when the original investigators are being dilatory or are otherwise acting in a dubious manner, such as when a secondary paper is either delayed or rushed into publication, or when it is written in a way to invite controversy. Often these accusations arise when these action appear to favor the interest of the sponsor or the intellectual position of the authors. In such cases the threat of an outside analysis may well have a beneficial impact on the entire process.
Like many elites, the trial investigators think that they have a monopoly on the truth and that data interpretation outside their trusted hands will be tainted and lead to harmful consequences. The ACCESS CV authors write:
Attempts to validate trial findings made by persons who are either unfamiliar with the data set structure or inexperienced in the analysis techniques could create apparent discrepancies where none exist, potentially alarming the public and hindering rather than advancing science. The problem may be compounded by publication bias, which may lead to undue focus on findings that seem to differ from those of the original analysis.
But there is absolutely no mention or recognition of the fact that these dangers already exist and proliferate in the current system. Think of all the cases of medical reversal in the last decade or so. We need more not fewer analyses of data. Why do these investigators have so little faith in the scientific process outside their own control? Of course there may well be some poor quality publications, but that’s how the scientific process works. One would assume that these investigators and others will respond appropriately and let the court of scientific opinion decide. The trial investigators can’t serve as judge, jury, DA, and defense lawyer all at once. That’s not how it works.
The best example of the importance of independent analysis is the extraordinary reviews performed by FDA reviewers. These often differ in significant and important ways from the original analysis published by the investigators. But in general the FDA performs these analyses on the big phase 3 and 4 pivotal trials. Most trials never undergo similar scrutiny. Now imagine what outside researchers might turn up when given a chance to independently analyze earlier exploratory trials, since these are the trials that are far more likely to be subject to abuse and distortion. It is a small but telling detail that the ACCESS CV authors mention the “confirmation of results” as a benefit of data sharing but not the refutation or correction of wrong or mistaken results.
I am also sorrowed by the obsession over academic credit in the articles. I have never heard anyone actually defend the current system of authorship, and all the related gamesmanship and nonsense involving citations and impact factor. Everyone I’ve ever spoken with agrees that the whole system is broken and sick. Why should we allow a really good proposal to founder on the basis of defending a broken system? Instead, let’s use the new proposal as a wedge to begin reform of the utter madness of the authorship/journal/academic credit system.
Further, if the authors really had an interest in preserving the purity of the publication process they would have long since expressed far more concern about slice and dice publications and guest authorship. But about these issues they have been silent. Authorship is a legitimate issue but it’s part of a different but related and equally important discussion.
One final point. I’ve previously speculated that the most important benefit of sharing data is the effect that it will have on the original investigators. I don’t think we should expect– at least right away– a whole bunch of radical revisions of the literature. But I suspect that the investigators will start to behave differently and better with the underlying knowledge that it is at least feasible that their data will be examined by outsiders.
Speak Your Mind