Guest Post: After an Unprecedented Request for a Retraction, A Close Look at the Data

Editor’s Note: The following guest post is published with the permission of its author,  Edward J. Schloss, MD, (Twitter ID @EJSMD) the medical director of cardiac electrophysiology at Christ Hospital in Cincinnati, OH.

After an Unprecedented Request for a Retraction, A Close Look at the Data

by Edward J Schloss MD

Last week, St. Jude Medical took the unusual step of requesting a retraction of an article accepted for publication in Heart Rhythm Journal.  In this article Deaths Caused by the Failure of Riata and Riata ST Implantable Cardioverter-Defibrillator Leads, Dr. Robert Hauser et al analyzed patient deaths in the FDA MAUDE database associated with St. Jude Medical and Medtronic ICD leads.  This week, Dr. Douglas Zipes, Heart Rhythm editor, declined that request. (Please see my previous Cardiobrief coverage of Hauser’s article.)

In a company news release, St. Jude accused Hauser’s group of “inaccurate facts and biased analysis.”  The sub-headline reads:  “Research undercounted and excluded MAUDE data reports for Medtronic resulting in substantial factual errors.”

St. Jude’s most prominent concern revolves around the number of reported deaths in the analysis of Riata/Riata ST leads and the comparison lead, Medtronic Quattro Secure.  St. Jude wrote: “using the same search criteria outlined in the manuscript, the company has identified that Dr. Hauser’s research substantially undercounted total deaths in the MAUDE database for Quattro Secure.”  St. Jude counts 377 deaths compared to Hauser’s 62.

In an effort to clarify the reasons for this discrepancy, I undertook an independent analysis of the facts available.  My sources are the original Hauser article and the PDF file from St. Jude listing the details of their database search.  I then examined the methodology of both database searches to compare consistency of data entry.  Finally, I performed an independent review of all MAUDE database entries supplied by St. Jude and attempted to classify the 377 deaths Quattro Secure deaths identified by St. Jude into Hauser’s defined categories:  lead-related, indeterminate, and not lead-related, using the methods he described in his paper.

I hypothesized that differences in search methodology may explain the large discrepancy in numbers of deaths found by Hauser and St. Jude.

Results

The Hauser study queried the MAUDE data using the “simple search feature” entering term “Quattro Secure death.”  Results were “refined” to include only model 6947 leads.  Finally, “Reports were excluded from the study if there was no known lead problem or allegation of lead malfunction, and if a returned product analysis found no anomalies that were not caused by the explant procedure.”  With this methodology, the query returned 62 Medtronic Quattro Secure deaths.

The St. Jude data collection methodology is discussed on page one of the PDF document.  The “simple search” function specified by Hauser is not used.  Rather in the standard search fields, individual entries are made for Brand:  Quattro, Manufacturer:  Medtronic, and Event:  Death.  Results were then refined to model 6947 excluding duplicates.  Also in contrast to the Hauser data, no pre-specified exclusions were indicated. This query listed 377 Medtronic Quattro Secure deaths.

Hauser categorized his 62 deaths as 5 (8%) lead-related, 25 (40%) indeterminate, and 32 (52%) not lead-related.

St. Jude did not categorize the 377 deaths. My individual analysis found 7 (2%) lead-related, 129 (34%) indeterminate, and 241 (64%) not lead-related. (Lead-related death MDR Identifiers were 71402, 1438305, 1523151, 1760409, 1944512, 2016463, and 2025526.) I have asked St. Jude for their own analysis of the data into these categories, but have not received a reply.

Discussion

My review confirmed significant methodological differences in the database query specified by Hauser and St. Jude that may explain the differences in the numbers of deaths reported.  This stands in contrast to St. Jude’s assertion that they used “the same search criteria outlined in the manuscript.”

First, St. Jude does not use the “simple search” applied by Hauser, but rather a standard search with several individual field entries.  The substance of the search terms appears similar, but it is possible that database output would differ.  It is not clear why St. Jude chose to analyze in this fashion as Hauser’s methods were clearly stated in his manuscript.  I did not attempt to compare these two search methodologies, so it is not clear how much this difference contributed to the output discrepancies.

Second, St. Jude does not perform any specified exclusions from the output of the original database query.  As discussed above, Hauser indicated that prior to data analysis, reports were excluded  “if there was no known lead problem or allegation of lead malfunction, and if a returned product analysis found no anomalies that were not caused by the explant procedure.”  By leaving these types of reports in the data set, one would expect to see higher numbers of indeterminate or non-lead related deaths in the St. Jude data.

In my individual analysis of the St. Jude supplied reports, I found only a modest increase in lead-related deaths, from 5 to 7.  On the other hand, indeterminate deaths increased greatly, from 25 to 139.  Likewise, not lead-related deaths increased from 32 to 241.

Examination of the MDR identifiers of the 7 lead-related deaths in my dataset showed that 4 of these were common to Hauser’s group of 5.  I classified 1 of the Hauser lead-related deaths as not lead-related.  I classified 1 of Hauser’s indeterminate deaths as lead-related.  Lastly, 2 of my lead-related deaths were not seen in the Hauser data.  The mode of lead-related death in all 7 patients was similar to what was seen in Hauser’s data– conductor failures and sensing issues.  There were no can abrasions or documented insulation defects, as were commonly seen in the Riata/Riata ST deaths.

My findings support the hypothesis that additional deaths in the St. Jude data set can be explained by methodological differences in data collection.  The additional St. Jude data net only a 2 patient increase in lead-related deaths and actually reduce the percentage of these deaths from 8% to 2%.

There are significant limitations to any analysis of MAUDE data.  The quantity and relevance of data presented is highly inconsistent.  Many cases have insufficient information to draw any meaningful conclusions about lead performance and are therefore classified as indeterminate.  Categorization is subject to variable interpretation.  I used a highly specific standard when categorizing deaths as lead-related.  This will, by necessity limit the sensitivity of this category (i.e. some lead-related deaths are missed).  In so doing, I attempted to replicate Hauser’s methods.  MAUDE data entry is voluntary, so comparing datasets to one another should be approached with great caution.  One should note that most patients with ICDs die without ever having their data entered into MAUDE.

St. Jude’s retraction request contained a variety of other complaints that are outside the scope of this article.  These include questions of the validity of the quantitative comparisons drawn between numbers of deaths for both vendors’ leads.  St. Jude also suggested alternative leads to compare to Riata/Riata ST, such as Medtronic Fidelis, which is also on recall.

Summary

Methodological differences explain the wide disparity in number of deaths reported from analysis of MAUDE by Dr. Hauser’s group and St. Jude Medical.  The uncounted deaths in Dr. Hauser’s analysis were almost entirely indeterminate or not lead-related, and likely were excluded by Dr. Hauser’s differing collection methods.  In both datasets, a low number of lead-related deaths were observed.

Edward J. Schloss MD

Medical Director, Cardiac Electrophysiology

The Christ Hospital

Cincinnati, OH

Dr. Schloss has research relationships with Biotronik, Boston Scientific, Medtronic and St. Jude Medical.  He has consulting relationships with Boston Scientific and Medtronic.

 A word about potential conflicts of interest:  I wish to clearly state that my work on this project has been done without industry guidance or financial support.  I have not been subjected to a “whisper campaign,” nor has “competitive marketing” influenced my motives.  I have asked for, and been given valuable information from all major device vendors, but have never been solicited or lobbied.  I am driven by my desire to provide high quality patient care and educate others about this complicated subject.