I’m commenting on this post at NewtonBlog because it ties in so neatly with a few things I’ve written lately. Ross Pomeroy reports on a Mayo Clinic study on why nutrition and diet studies are so lousy:
Every five years, the Dietary Guidelines Advisory Committee, composed of nutrition and health experts from around the country, convenes to review the latest scientific and medical literature. From their learned dissection, they form the dietary guidelines.
But according to a new editorial published in Mayo Clinic Proceedings, much of the science they review is fundamentally flawed. Unlike experiments in the hard sciences of chemistry, physics, and biology, which rely on direct observational evidence, most diet studies are based on self-reported data. Study subjects are examined for height, weight, and health, then are questioned about what they eat. Their dietary choices are subsequently linked to health outcomes — cancer, mortality, heart disease, etc.
That’s a poor way of doing science, says Edward Archer, a research fellow with the Nutrition Obesity Research Center at the University of Alabama, and lead author of the report.
“The assumption that human memory can provide accurate or precise reproductions of past ingestive behavior is indisputably false,” he and his co-authors write.
Two of the largest studies on nutritional intake in the United States, the CDC’s NHANES and “What We Eat,” are based on asking subjects to recall precisely what and how much they usually eat.
But despite all of the steps that NHANES examiners take to aid recall, such as limiting the recall period to the previous 24 hours and even offering subjects measuring guides to help them report accurate data, the information received is wildly inaccurate. An analysis conducted by Archer in 2013 found that most of the 60,000+ NHANES subjects report eating a lower amount of calories than they would physiologically need to survive, let alone to put on all the weight that Americans have in the past few decades.
Please note a few things. First, testimony is lousy evidence. I agree with that. However, the conclusion about that should not be that, since people’s self-reported “dietary choices” don’t conform to the simple thermodynamics theory of diet (calories in minus calories expended through exercise equals net weight gain or loss), people must be lying. The conclusion should be that better studies are needed, ones that don’t rely on self-reporting. It’s possible, even likely that the simple thermodynamics theory is wrong. For example, if the simple thermodynamics explains 100% of weight loss or gain, how do you explain the frequently observed weight loss after fecal transplant?
Both weight loss and weight gain have been reported. Transplants from thin people seem to lead to weight loss and from fat people to weight gain. But, too early to say for sure I think. Not many large scale prospective studies, which is almost always the case in diet studies. Not blinded, too small, often retrospective. When one sees a diet study one should always assume it is flawed until proven otherwise.
Steve