–Defying conventional wisdom Ioannidis says we need more negative trials to clear away all our misconceptions about nutrition
Our scientific understanding of nutrition is so poor that we need negative randomized controlled trials just to help clear up all the errors and misconceptions that muddy the field, writes John Ioannidis (Stanford University), the well-known critic of medical evidence, in an editorial in American Journal of Clinical Nutrition
Large, well-performed randomized trials in nutrition are rare, but many nutrition experts are unconcerned. “Influential epidemiologists have expressed skepticism about randomized trials for nutrition or life-style interventions in general,” he notes. “Nonrandomized observational studies have been favored because people change their choices; thus, crossover, cross-in, withdrawals, and poor adherence are considered impediments for clinical trials, and the estimation of effect sizes by intention-to-treat may misrepresent the mechanistic effects.”
But it is essential to perform randomized trials for precisely the same reasons that they are difficult, he writes. “Epidemiologic studies are just too unreliable to capture subtle treatment effects whenever noise due to confounding and other biases exceeds plausible signals. For most clinical outcomes (as opposed to physiology or surrogate outcomes), the magnitude of the treatment effects of nutritional interventions is probably small or modest.”
Ioannidis also insists that information from mechanistic trials should not be blindly applied without careful attention to their practical effect: “If an intervention is mechanistically efficacious but cannot be used long term, it is practically the same as if it lacked efficacy to start with.”
Large trials are required because most nutrition trials will have a modest effect and unknown adherence, he says. “Chasing small or modest effects with underpowered, small studies is an ideal recipe for getting both false-negative and even more false-positive results. Long-term follow-up is necessary both for increasing the power for patient-relevant clinical events as well as for making the results relevant for real life.”
Negative trials should not be viewed as a bad thing, Ioannidis argues. Trials that show two interventions are equivalent “mean that different nutrition choices lead to equally acceptable outcomes. This is great news: it is better to have many equally good choices on what to eat than just one.”
Further, since observational epidemiology often leads to the belief that specific nutrients or diets are effective, a negative trial actually increases our “information gain” and will point the way for researchers to begin to design better studies more likely to achieve a useful end.
Ioannidis concedes that randomized trials are expensive and difficult to perform, but ultimately they are necessary. Observational studies may be “cheaper to perform” but they don’t yield “useful information.” Instead of the multitude of observational studies and small randomized trials, Ioannidis suggests that “the same resources might suffice to conduct a smaller number of large-scale, long-term, fully registered, and transparent rials that would be fully and appropriately reported.”
At that point, Ioannidis writes, “I wouldn’t be disappointed to also see some informative ‘positive’ results.”
Amen!
Because of the time frame of atherosclerosis the best candidates for study are those with severe disease, as done in Essestyn’s small cohort.