Advertisement
Mayo Clinic Proceedings Home

The Validity of Self-reported Dietary Intake Data: Focus on the “What We Eat In America” Component of the National Health and Nutrition Examination Survey Research Initiative

  • Brenda M. Davy
    Correspondence
    Correspondence: Address to Brenda M. Davy, PhD, RD, Department of Human Nutrition, Foods and Exercise, Virginia Polytechnic Institute and State University, 221 Wallace Hall, Blacksburg, VA 24061.
    Affiliations
    Department of Human Nutrition, Foods and Exercise, Virginia Polytechnic Institute and State University, Blacksburg, VA
    Search for articles by this author
  • Paul A. Estabrooks
    Affiliations
    Department of Human Nutrition, Foods and Exercise, Virginia Polytechnic Institute and State University, Blacksburg, VA
    Department of Family and Community Medicine, Carilion Clinic, Roanoke, VA
    Search for articles by this author
      The article by Archer et al
      • Archer E.
      • Pavela G.
      • Lavie C.
      The inadmissibility of What We Eat In America and NHANES dietary data in nutrition and obesity research and the scientific formulation of national dietary guidelines.
      in the current issue of Mayo Clinic Proceedings questions the validity and value of memory-based dietary assessment tools when investigating the relationship between dietary intake and health outcomes. In doing so, the authors cast doubt on the evidence base used by the US government’s 2015 Dietary Guidelines Advisory Committee (DGAC) and its actions to improve public health.
      Dietary Guidelines Advisory Committee
      Scientific Report of the 2015 Dietary Guidelines Advisory Committee.
      Archer et al
      • Archer E.
      • Pavela G.
      • Lavie C.
      The inadmissibility of What We Eat In America and NHANES dietary data in nutrition and obesity research and the scientific formulation of national dietary guidelines.
      advocate that dietary recall cannot be used to establish relationships between dietary intake and health. They suggest Popper’s criteria
      • Popper K.R.
      The Logic of Scientific Discovery.
      for scientific inquiry—independently observable, measurable, falsifiable, valid, and reliable—as the standard by which to judge recall-based assessments. In this editorial, we provide empirical examples that suggest that, by Popper’s criteria, recall measures can be scientifically sound. We also highlight a different perspective that values multiple forms of evidence to determine the scientific appropriateness of measurement instruments, including predictive validity, sensitivity to change, feasibility, and actionability,
      • Estabrooks P.A.
      • Boyle M.
      • Emmons K.M.
      • et al.
      Harmonized patient-reported data elements in the electronic health record: supporting meaningful use by primary care action on health behaviors and key psychosocial factors.
      as well as the goal and design of a given study.
      Attempting to develop recommendations to improve health is a complex enterprise because of the interactive nature of genetics, environmental factors, and individual behavior; however, one thing is clear—behaviors matter.
      • Fisher E.B.
      • Fitzgibbon M.L.
      • Glasgow R.E.
      • et al.
      Behavior matters.
      Tobacco use, physical inactivity, excessive alcohol consumption, and a poor diet are the leading preventable causes of death in the United States.
      • Mokdad A.H.
      • Marks J.S.
      • Stroup D.F.
      • Gerberding J.L.
      Actual causes of death in the United States, 2000.
      The body of research that contributed to these findings includes a variety of scientific approaches that range from retrospective and prospective epidemiological studies to randomized controlled trials. One consistency across scientific inquiry and behavioral domains is that participant recall has been used as a representation of behavior, by Archer et al
      • Archer E.
      • Pavela G.
      • Lavie C.
      The inadmissibility of What We Eat In America and NHANES dietary data in nutrition and obesity research and the scientific formulation of national dietary guidelines.
      and others (for examples, see references
      • Leitzmann M.F.
      • Park Y.
      • Blair A.
      • et al.
      Physical activity recommendations and decreased risk of mortality.
      ,
      • Lee D.C.
      • Pate R.R.
      • Lavie C.J.
      • Sui X.
      • Church T.S.
      • Blair S.N.
      Leisure-time running reduces all-cause and cardiovascular mortality risk.
      ,
      • Hardee J.P.
      • Porter R.R.
      • Sui X.
      • et al.
      The effect of resistance exercise on all-cause mortality in cancer survivors.
      ,

      Drenowatz C, Jakicic JM, Blair SN, Hand GA. Differences in correlates of energy balance in normal weight, overweight and obese adults [published online ahead of print April 8, 2015]. Obes Res Clin Pract. http://dx.doi.org/10.1016/j.orcp.2015.03.007.

      ).
      Rather than being a “well-kept open secret,”
      • Archer E.
      • Pavela G.
      • Lavie C.
      The inadmissibility of What We Eat In America and NHANES dietary data in nutrition and obesity research and the scientific formulation of national dietary guidelines.
      the limitations of self-reported dietary and physical activity assessment methods are well recognized and acknowledged by those utilizing these methods.
      • Leitzmann M.F.
      • Park Y.
      • Blair A.
      • et al.
      Physical activity recommendations and decreased risk of mortality.
      • Lee D.C.
      • Pate R.R.
      • Lavie C.J.
      • Sui X.
      • Church T.S.
      • Blair S.N.
      Leisure-time running reduces all-cause and cardiovascular mortality risk.
      • Hardee J.P.
      • Porter R.R.
      • Sui X.
      • et al.
      The effect of resistance exercise on all-cause mortality in cancer survivors.

      Drenowatz C, Jakicic JM, Blair SN, Hand GA. Differences in correlates of energy balance in normal weight, overweight and obese adults [published online ahead of print April 8, 2015]. Obes Res Clin Pract. http://dx.doi.org/10.1016/j.orcp.2015.03.007.

      However, the predictive validity of dietary recall instruments has been repeatedly documented using dose-response and other predictive methods.
      • Hu F.B.
      • Stampfer M.J.
      • Manson J.E.
      • et al.
      Dietary fat intake and the risk of coronary heart disease in women.
      • Satija A.
      • Yu E.
      • Willett W.C.
      • Hu F.B.
      Understanding nutritional epidemiology and its role in policy.
      Still, as with many types of scientific and clinical measurements such as blood pressure measurement,
      • Pickering T.G.
      • Hall J.E.
      • Appel L.J.
      • et al.
      Recommendations for blood pressure measurement in humans and experimental animals, Part 1: Blood pressure measurement in humans; a statement for professionals from the Subcommittee of Professional and Public Education of the American Heart Association Council on High Blood Pressure Research.
      the accuracy of dietary recalls can be substantially improved by using validated protocols (eg, multiple 24-hour recalls obtained using the Automated Multiple-Pass Method)
      • Moshfegh A.J.
      • Rhodes D.G.
      • Baer D.J.
      • et al.
      The US Department of Agriculture Automated Multiple-Pass Method reduces bias in the collection of energy intakes.
      and well-trained research personnel.
      • Pickering T.G.
      • Hall J.E.
      • Appel L.J.
      • et al.
      Recommendations for blood pressure measurement in humans and experimental animals, Part 1: Blood pressure measurement in humans; a statement for professionals from the Subcommittee of Professional and Public Education of the American Heart Association Council on High Blood Pressure Research.
      • Moshfegh A.J.
      • Rhodes D.G.
      • Baer D.J.
      • et al.
      The US Department of Agriculture Automated Multiple-Pass Method reduces bias in the collection of energy intakes.
      The Automated Multiple-Pass Method for collecting dietary recalls has been validated, using the criterion standard doubly labeled water technique as a biomarker of total energy expenditure (and thus, energy intake, if body weight is stable during the measurement period).
      • Moshfegh A.J.
      • Rhodes D.G.
      • Baer D.J.
      • et al.
      The US Department of Agriculture Automated Multiple-Pass Method reduces bias in the collection of energy intakes.
      Using this approach, the overall underreporting of energy intake was 11% when compared with total energy expenditure determined by doubly labeled water. In normal-weight individuals, the degree of underreporting was less than 3%, and a higher degree of underreporting was noted among overweight and obese individuals.
      • Moshfegh A.J.
      • Rhodes D.G.
      • Baer D.J.
      • et al.
      The US Department of Agriculture Automated Multiple-Pass Method reduces bias in the collection of energy intakes.
      Archer et al
      • Archer E.
      • Pavela G.
      • Lavie C.
      The inadmissibility of What We Eat In America and NHANES dietary data in nutrition and obesity research and the scientific formulation of national dietary guidelines.
      chose to frame the argument of independent observability, measurability, reliability, and falsification within the context of memory rather than the behaviors that those memories are intended to reflect. Our perspective is that the behavioral target—dietary intake—can be measured reliably over time.
      • Willett W.C.
      • Sampson L.
      • Stampfer M.J.
      • et al.
      Reproducibility and validity of a semiquantitative food frequency questionnaire.
      Further, these behaviors can also be observed independently, and those recalled accounts can be falsified (ie, experimentally tested and proven false [or confirmed]).
      • Moshfegh A.J.
      • Rhodes D.G.
      • Baer D.J.
      • et al.
      The US Department of Agriculture Automated Multiple-Pass Method reduces bias in the collection of energy intakes.
      • Rhodes D.G.
      • Murayi T.
      • Clemens J.C.
      • Baer D.J.
      • Sebastian R.S.
      • Moshfegh A.J.
      The USDA Automated Multiple-Pass Method accurately assesses population sodium intakes.
      Thus, we disagree that the What We Eat In America (WWEIA)/National Health and Nutrition Examination Survey (NHANES) dietary recall data represent a “highly edited anecdote”
      • Archer E.
      • Pavela G.
      • Lavie C.
      The inadmissibility of What We Eat In America and NHANES dietary data in nutrition and obesity research and the scientific formulation of national dietary guidelines.
      of what was consumed; instead, we believe that they reflect a reasonable representation of usual dietary intake.
      Archer et al
      • Archer E.
      • Pavela G.
      • Lavie C.
      The inadmissibility of What We Eat In America and NHANES dietary data in nutrition and obesity research and the scientific formulation of national dietary guidelines.
      imply that there are superior alternatives to assess dietary intake to derive population prevalence estimates and more accurate accounts of what Americans eat every day. However, there is no error-free, practical, and affordable method to capture the broad dietary pattern information that is currently acquired using WWEIA/NHANES.
      • Hébert J.R.
      • Hurley T.G.
      • Stech S.E.
      • et al.
      Considering the value of dietary assessment data in informing nutrition-related health policy.
      Self-reported dietary assessment methods are a useful and appropriate measurement technique for many types of studies including assessments of population prevalence and epidemiological studies to identify potential health risks associated with a range of foods and eating patterns.
      • Satija A.
      • Yu E.
      • Willett W.C.
      • Hu F.B.
      Understanding nutritional epidemiology and its role in policy.
      • Hébert J.R.
      • Hurley T.G.
      • Stech S.E.
      • et al.
      Considering the value of dietary assessment data in informing nutrition-related health policy.
      In addition, dietary pattern data provide actionable information for intervention that is not available from methods that simply provide information on total energy or nutrient intake.
      In some circumstances, depending on factors such as the study’s question, design, setting,
      • Farquhar J.W.
      • Fortmann S.P.
      • Flora J.A.
      • et al.
      Effects of communitywide education on cardiovascular disease risk factors: the Stanford Five-City Project.
      resources, target population, and participant burden,
      • Rhodes D.G.
      • Murayi T.
      • Clemens J.C.
      • Baer D.J.
      • Sebastian R.S.
      • Moshfegh A.J.
      The USDA Automated Multiple-Pass Method accurately assesses population sodium intakes.
      • Hedrick V.E.
      • Dietrich A.M.
      • Estabrooks P.A.
      • Savla J.
      • Serrano E.
      • Davy B.M.
      Dietary biomarkers: advances, limitations and future directions.
      recall measures can be appropriate. For example, a small, tightly controlled efficacy trial investigating the influence of prebiotic supplementation on cardiometabolic health may warrant different types of assessments than would a large, multicity trial investigating comprehensive interventions to reduce coronary heart disease risk.
      • Farquhar J.W.
      • Fortmann S.P.
      • Flora J.A.
      • et al.
      Effects of communitywide education on cardiovascular disease risk factors: the Stanford Five-City Project.
      In the latter case, a validated dietary recall instrument could be considered, and so too could an extensive panel of dietary biomarkers (ie, objective indicators of dietary intake) requiring biological sample collection at multiple time points. Dietary biomarker assessment would be costly (and likely exceed the budget limitations of a federal grant), burdensome for study participants, and less feasible for research teams to implement in field/community settings. Furthermore, the biomarkers could provide information on specific dietary factors (eg, energy intake [doubly labeled water], protein intake [urinary nitrogen excretion], or fruit/vegetable intake [blood carotenoid concentrations]) but not specific foods consumed (eg, carrots, cantaloupe) or overall diet quality.
      • Pfeiffer C.M.
      • Sternberg M.R.
      • Schleicher R.L.
      • Haynes B.M.
      • Rybak M.E.
      • Pirkle J.L.
      The CDC’s Second National Report on Biochemical Indicators of Diet and Nutrition in the U.S. Population is a valuable tool for researchers and policy makers.
      The sources of nutrients (ie, foods vs dietary supplements) would not be known, unless individuals were asked to recall this information. Biomarker results may also vary depending on physiologic states (eg, fasted vs fed
      • Haynes B.M.
      • Pfeiffer C.M.
      • Sternberg M.R.
      • Schleicher R.L.
      Selected physiologic variables are weakly to moderately associated with 29 biomarkers of diet and nutrition, NHANES 2003-2006.
      ) and whether the biomarker reflected long-term vs short-term dietary intake.
      • Satija A.
      • Yu E.
      • Willett W.C.
      • Hu F.B.
      Understanding nutritional epidemiology and its role in policy.
      • Haynes B.M.
      • Pfeiffer C.M.
      • Sternberg M.R.
      • Schleicher R.L.
      Selected physiologic variables are weakly to moderately associated with 29 biomarkers of diet and nutrition, NHANES 2003-2006.
      Objective tools also suffer from measurement error, and the limitations of contemporary dietary biomarker techniques have been discussed in the scientific literature.
      • Satija A.
      • Yu E.
      • Willett W.C.
      • Hu F.B.
      Understanding nutritional epidemiology and its role in policy.
      • Hedrick V.E.
      • Dietrich A.M.
      • Estabrooks P.A.
      • Savla J.
      • Serrano E.
      • Davy B.M.
      Dietary biomarkers: advances, limitations and future directions.
      • Pfeiffer C.M.
      • Sternberg M.R.
      • Schleicher R.L.
      • Haynes B.M.
      • Rybak M.E.
      • Pirkle J.L.
      The CDC’s Second National Report on Biochemical Indicators of Diet and Nutrition in the U.S. Population is a valuable tool for researchers and policy makers.
      • Haynes B.M.
      • Pfeiffer C.M.
      • Sternberg M.R.
      • Schleicher R.L.
      Selected physiologic variables are weakly to moderately associated with 29 biomarkers of diet and nutrition, NHANES 2003-2006.
      Hébert et al
      • Hébert J.R.
      • Hurley T.G.
      • Stech S.E.
      • et al.
      Considering the value of dietary assessment data in informing nutrition-related health policy.
      provided an excellent overview of key advances in dietary assessment methods, suggestions for further improvement, and detailed counterpoints that address many of the issues raised by Archer et al.
      • Archer E.
      • Pavela G.
      • Lavie C.
      The inadmissibility of What We Eat In America and NHANES dietary data in nutrition and obesity research and the scientific formulation of national dietary guidelines.
      Adding to these suggestions for improvement, research investigating a greater variety of validated biomarkers for both foods and nutrients, the development of low-cost and minimally invasive dietary biomarkers, study designs that utilize dietary biomarkers to support self-reported dietary intake data, and novel technologies to assess and monitor dietary intake could greatly advance nutrition and obesity research.
      The commentary by Archer et al
      • Archer E.
      • Pavela G.
      • Lavie C.
      The inadmissibility of What We Eat In America and NHANES dietary data in nutrition and obesity research and the scientific formulation of national dietary guidelines.
      asserts that the use of memory-based assessment methods by NHANES and others “constitutes the single greatest impediment to actual scientific progress in the fields of obesity and nutrition research.”
      • Archer E.
      • Pavela G.
      • Lavie C.
      The inadmissibility of What We Eat In America and NHANES dietary data in nutrition and obesity research and the scientific formulation of national dietary guidelines.
      We suggest that progress in the fields of obesity and nutrition research will be made through interdisciplinary research utilizing a combination of research approaches. Consistent with this view, the DGAC used a variety of types of scientific evidence, including state-of-the-art systematic reviews, meta-analyses, individual reports, and NHANES data analyses.
      Dietary Guidelines Advisory Committee
      Scientific Report of the 2015 Dietary Guidelines Advisory Committee.
      Satija et al
      • Satija A.
      • Yu E.
      • Willett W.C.
      • Hu F.B.
      Understanding nutritional epidemiology and its role in policy.
      reported 3 excellent examples of how various types of nutrition research were combined to create an evidence base that can then be used by groups such as the DGAC to inform public policy.
      As stated by the 2015 DGAC, “repeated 24-hour recalls remain the backbone of dietary assessment and monitoring.”
      Dietary Guidelines Advisory Committee
      Scientific Report of the 2015 Dietary Guidelines Advisory Committee.
      The WWEIA/NHANES dietary data represent the only comprehensive source of information on the food and nutrient intake of the US population—the survey includes a nationally representative sample of US residents, and dietary intake data are collected using standardized, validated protocols.
      • Moshfegh A.J.
      • Rhodes D.G.
      • Baer D.J.
      • et al.
      The US Department of Agriculture Automated Multiple-Pass Method reduces bias in the collection of energy intakes.
      • Rhodes D.G.
      • Murayi T.
      • Clemens J.C.
      • Baer D.J.
      • Sebastian R.S.
      • Moshfegh A.J.
      The USDA Automated Multiple-Pass Method accurately assesses population sodium intakes.
      These and other sources of broad population assessments of dietary intake and physical activity are the foundation of many seminal articles and research findings that have linked dietary intake and physical activity as key determinants of health. To argue that these data represent a waste of resources, while concurrently citing scientific findings that those same data collection methods were used to document the importance of diet and activity in health, is scientific doublespeak—and an impediment to scientific progress in obesity and nutrition research.

      References

        • Archer E.
        • Pavela G.
        • Lavie C.
        The inadmissibility of What We Eat In America and NHANES dietary data in nutrition and obesity research and the scientific formulation of national dietary guidelines.
        Mayo Clinic Proc. 2015; 90: 911-926
        • Dietary Guidelines Advisory Committee
        Scientific Report of the 2015 Dietary Guidelines Advisory Committee.
        US Dept of Health and Human Services and US Dept of Agriculture, Washington, DC2015
        • Popper K.R.
        The Logic of Scientific Discovery.
        2nd ed. Routledge, London, UK, and New York, NY2002
        • Estabrooks P.A.
        • Boyle M.
        • Emmons K.M.
        • et al.
        Harmonized patient-reported data elements in the electronic health record: supporting meaningful use by primary care action on health behaviors and key psychosocial factors.
        J Am Med Inform Assoc. 2012; 19: 575-582
        • Fisher E.B.
        • Fitzgibbon M.L.
        • Glasgow R.E.
        • et al.
        Behavior matters.
        Am J Prev Med. 2011; 40: e15-e30
        • Mokdad A.H.
        • Marks J.S.
        • Stroup D.F.
        • Gerberding J.L.
        Actual causes of death in the United States, 2000.
        JAMA. 2004; 291 ([published correction appears in JAMA. 2005;293(3):298]): 1238-1245
        • Leitzmann M.F.
        • Park Y.
        • Blair A.
        • et al.
        Physical activity recommendations and decreased risk of mortality.
        Arch Intern Med. 2007; 167: 2453-2460
        • Lee D.C.
        • Pate R.R.
        • Lavie C.J.
        • Sui X.
        • Church T.S.
        • Blair S.N.
        Leisure-time running reduces all-cause and cardiovascular mortality risk.
        J Am Coll Cardiol. 2014; 64 ([published correction appears in J Am Coll Cardiol. 2014;64(14):1537]): 472-481
        • Hardee J.P.
        • Porter R.R.
        • Sui X.
        • et al.
        The effect of resistance exercise on all-cause mortality in cancer survivors.
        Mayo Clin Proc. 2014; 89: 1108-1115
      1. Drenowatz C, Jakicic JM, Blair SN, Hand GA. Differences in correlates of energy balance in normal weight, overweight and obese adults [published online ahead of print April 8, 2015]. Obes Res Clin Pract. http://dx.doi.org/10.1016/j.orcp.2015.03.007.

        • Hu F.B.
        • Stampfer M.J.
        • Manson J.E.
        • et al.
        Dietary fat intake and the risk of coronary heart disease in women.
        N Engl J Med. 1997; 337: 1491-1499
        • Satija A.
        • Yu E.
        • Willett W.C.
        • Hu F.B.
        Understanding nutritional epidemiology and its role in policy.
        Adv Nutr. 2015; 6: 5-18
        • Pickering T.G.
        • Hall J.E.
        • Appel L.J.
        • et al.
        Recommendations for blood pressure measurement in humans and experimental animals, Part 1: Blood pressure measurement in humans; a statement for professionals from the Subcommittee of Professional and Public Education of the American Heart Association Council on High Blood Pressure Research.
        Hypertension. 2005; 45: 142-161
        • Moshfegh A.J.
        • Rhodes D.G.
        • Baer D.J.
        • et al.
        The US Department of Agriculture Automated Multiple-Pass Method reduces bias in the collection of energy intakes.
        Am J Clin Nutr. 2008; 88: 324-332
        • Willett W.C.
        • Sampson L.
        • Stampfer M.J.
        • et al.
        Reproducibility and validity of a semiquantitative food frequency questionnaire.
        Am J Epidemiol. 1985; 122: 51-65
        • Rhodes D.G.
        • Murayi T.
        • Clemens J.C.
        • Baer D.J.
        • Sebastian R.S.
        • Moshfegh A.J.
        The USDA Automated Multiple-Pass Method accurately assesses population sodium intakes.
        Am J Clin Nutr. 2013; 97: 958-964
        • Hébert J.R.
        • Hurley T.G.
        • Stech S.E.
        • et al.
        Considering the value of dietary assessment data in informing nutrition-related health policy.
        Adv Nutr. 2014; 5: 447-455
        • Farquhar J.W.
        • Fortmann S.P.
        • Flora J.A.
        • et al.
        Effects of communitywide education on cardiovascular disease risk factors: the Stanford Five-City Project.
        JAMA. 1990; 264: 359-365
        • Hedrick V.E.
        • Dietrich A.M.
        • Estabrooks P.A.
        • Savla J.
        • Serrano E.
        • Davy B.M.
        Dietary biomarkers: advances, limitations and future directions.
        Nutr J. 2012; 11: 109
        • Pfeiffer C.M.
        • Sternberg M.R.
        • Schleicher R.L.
        • Haynes B.M.
        • Rybak M.E.
        • Pirkle J.L.
        The CDC’s Second National Report on Biochemical Indicators of Diet and Nutrition in the U.S. Population is a valuable tool for researchers and policy makers.
        J Nutr. 2013; 143: 938S-947S
        • Haynes B.M.
        • Pfeiffer C.M.
        • Sternberg M.R.
        • Schleicher R.L.
        Selected physiologic variables are weakly to moderately associated with 29 biomarkers of diet and nutrition, NHANES 2003-2006.
        J Nutr. 2013; 143: 1001S-1010S

      Linked Article