Many Diet Studies are Based on Poorly Reported Data

What did you eat yesterday and can you remember the quantity? How about the day before or last week? If you’re scratching your head trying to remember, don’t worry it’s not a sign of memory loss. The fact is that most of us can’t recall from day to day what we eat.

So you may be surprised to learn that for decades researchers have been collecting data, which go on to form the basis of dietary recommendations, using error ridden self reports of food intake. Many studies ask participants to recall their food intake from the last 24 hours to the last 12 months, though periods of between 2 and 4 years are also often used.

The knowledge that these methods of data collection are flawed is nothing new and yet they continue to be used in studies. However, a new paper published in the International Journal of Obesity entitled Energy Balance Measurement: when something is not better than nothing takes aim at their use in no uncertain terms

“We argue here that it is time to move from the common view that self-reports of EI (energy intake) and PAEE (physical activity energy expenditure) are imperfect, but nevertheless deserving of use, to a view commensurate with the evidence that self-reports of EI and PAEE are so poor that they are wholly unacceptable for scientific research in EI and PAEE.”

The paper mentions the often used NHANES study, which while collecting much useful data relies on self reporting questionnaires that have ‘been repeatedly shown to be seriously flawed.’

The authors of the paper found that self-reported measures invariably underestimate energy intake by hundreds of calories per day. With the NHANES data underreporting the EI by as much as 800 kcal per day.

Many researchers have suggested ways around this by excluding any data that falls below a threshold limit, which would be considered impossible to survive on, but such an approach is too simplistic and assumes that only a subset of the population under reports when, in fact, under reporting varies with such factors as the type of food, age, gender, education, BMI and multiple other factors.

In a press statement lead author, David Allison, spelled it out

“Using these self-reported values, researchers had concluded for years that individuals with more body mass were eating less food than thinner individuals, which is counterintuitive. Indeed, when using objective, scientific measurements instead of self-reported measurements, it turns out that larger people on average eat more. Using the flawed self-reported values led to incorrect conclusions about physiology and the etiology of obesity, despite the large quantities of flawed data collected.”

In another study 45% of pregnant women under reported their energy intake with many reporting a caloric intake of slightly more than half of what would be considered a normal intake, 1194kcal vs 2125kcal.

Despite this and the fact that the limitations of using self reported data are well known and documented many studies and papers make bold claims and recommendations on conditions from cancer and chronic kidney disease to Alzheimer’s and heart disease, which are often readily employed in clinical settings. This is in spite of the fact that many studies often reach different conclusions about the roles of various foods and energy intakes because they are based on this flawed data.

The key point here being that even though these methods may be cheap and easy to perform using such inaccurate methods can only lead to inaccurate conclusions.

What’s needed the authors argue are more objective ways of measuring both energy intake and energy expenditure

“Improvement have been made in the development of more objective tools tools for measuring PAEE”, think pedometers, heart rate monitors and the slew of cheap health trackers, which aren’t perfect but are much better than self reporting. With regards to measuring energy intake the paper suggests a number of measures that are still in their infancy; digital photography, chewing and swallowing monitors and wrist detectors that count plate to mouth motion.

The short paper which was co-signed by 37 obesity researchers from around the world can be read here

Got a question or maybe something to add? Leave a comment and let me know!