Advertisement
Mayo Clinic Proceedings Home

The Inadmissibility of What We Eat in America and NHANES Dietary Data in Nutrition and Obesity Research and the Scientific Formulation of National Dietary Guidelines

      Abstract

      The Scientific Report of the 2015 Dietary Guidelines Advisory Committee was primarily informed by memory-based dietary assessment methods (M-BMs) (eg, interviews and surveys). The reliance on M-BMs to inform dietary policy continues despite decades of unequivocal evidence that M-BM data bear little relation to actual energy and nutrient consumption. Data from M-BMs are defended as valid and valuable despite no empirical support and no examination of the foundational assumptions regarding the validity of human memory and retrospective recall in dietary assessment. We assert that uncritical faith in the validity and value of M-BMs has wasted substantial resources and constitutes the greatest impediment to scientific progress in obesity and nutrition research. Herein, we present evidence that M-BMs are fundamentally and fatally flawed owing to well-established scientific facts and analytic truths. First, the assumption that human memory can provide accurate or precise reproductions of past ingestive behavior is indisputably false. Second, M-BMs require participants to submit to protocols that mimic procedures known to induce false recall. Third, the subjective (ie, not publicly accessible) mental phenomena (ie, memories) from which M-BM data are derived cannot be independently observed, quantified, or falsified; as such, these data are pseudoscientific and inadmissible in scientific research. Fourth, the failure to objectively measure physical activity in analyses renders inferences regarding diet-health relationships equivocal. Given the overwhelming evidence in support of our position, we conclude that M-BM data cannot be used to inform national dietary guidelines and that the continued funding of M-BMs constitutes an unscientific and major misuse of research resources.

      Abbreviations and Acronyms:

      BMR (basal metabolic rate), CRF (cardiorespiratory fitness), DRM (Deese-Roediger and McDermott), DGAC (Dietary Guidelines Advisory Committee), EI (energy intake), FFQ (food frequency questionnaire), M-BM (memory-based dietary assessment method), NHANES (National Health and Nutrition Examination Survey), PA (physical activity), rEI (reported energy intake), TEE (total energy expenditure), USDA (US Department of Agriculture), WWEIA (What We Eat in America), 24HR (24-hour dietary recall)
      When the facts change, I change my mind. What do you do, sir?John Maynard Keynes
      • Samuelson P.
      The Keynes centenary.
      ,p19

      Success, Failure, and Confusion in Nutrition Research

      During the past century, our nation’s food supply and the nutritional status of Americans have improved to a level unparalleled in human history.
      • Pfeiffer C.M.
      • Sternberg M.R.
      • Schleicher R.L.
      • Haynes B.M.
      • Rybak M.E.
      • Pirkle J.L.
      The CDC's Second National Report on Biochemical Indicators of Diet and Nutrition in the U.S. Population is a valuable tool for researchers and policy makers.
      • Gerrior S.
      • Bente L.
      • Hiza H.
      Nutrient Content of the U.S. Food Supply, 1909-2000.
      Although this reality may be contrary to the popular belief that our modern diet is inherently inadequate, the data are clear. In the early 20th century, nutritional diseases such as pellagra, beriberi, rickets, and goiter were substantial public health challenges. In the United States alone, pellagra (a disease of niacin deficiency) claimed more than 100,000 lives and severely affected more than 3 million people.
      • Rajakumar K.
      Pellagra in the United States: a historical perspective.
      Yet in 2013, the Centers for Disease Control and Prevention’s Second National Report on Biochemical Indicators of Diet and Nutrition reported that nearly “80% of Americans (aged ≥6 y) were not at risk of deficiencies in any of the 7 vitamins”
      • Rajakumar K.
      Pellagra in the United States: a historical perspective.
      ,p938 examined via biomarkers (ie, vitamins A, B6, B12, C, D, E, and folate; emphasis added).
      • Pfeiffer C.M.
      • Sternberg M.R.
      • Schleicher R.L.
      • Haynes B.M.
      • Rybak M.E.
      • Pirkle J.L.
      The CDC's Second National Report on Biochemical Indicators of Diet and Nutrition in the U.S. Population is a valuable tool for researchers and policy makers.
      In addition, approximately 90% of women of childbearing age (12-49 years) were not at risk for iron deficiency, and folate levels have increased by approximately 50% since the previous national report.
      • Pfeiffer C.M.
      • Sternberg M.R.
      • Schleicher R.L.
      • Haynes B.M.
      • Rybak M.E.
      • Pirkle J.L.
      The CDC's Second National Report on Biochemical Indicators of Diet and Nutrition in the U.S. Population is a valuable tool for researchers and policy makers.
      CDC
      National Report on Biochemical Indicators of Diet and Nutrition in the US Population 1999–2002.
      As such, most of the US population is not at risk for nutritional deficiencies, and neither do they have nutritional deficiencies and associated diseases.
      Given these important improvements in diet-related health and recent work demonstrating that nongenetic evolution is the predominant driver of the diseases of excess (eg, obesity and type 2 diabetes mellitus),
      • Archer E.
      The childhood obesity epidemic as a result of nongenetic evolution: the maternal resources hypothesis.
      • Archer E.
      The mother of all problems.
      • Archer E.
      In reply—Maternal, paternal, and societal efforts are needed to “cure” childhood obesity.
      it can be posited that diet is no longer a major risk factor for disease for most Americans. If accurate, this hypothesis suggests that the billions of research dollars targeted for diet and nutrition–related health research are misdirected.

      American Association for the Advancement of Science. Historical trends in federal R&D. http://www.aaas.org/page/historical-trends-federal-rd. Accessed February 21, 2015.

      Nevertheless, despite the important dietary milestones of the past century and the substantial increases in federal funding during the past 2 decades,

      American Association for the Advancement of Science. Historical trends in federal R&D. http://www.aaas.org/page/historical-trends-federal-rd. Accessed February 21, 2015.

      research into human nutrition has been increasingly criticized.
      • Ioannidis J.P.A.
      Implausible results in human nutrition research.
      • Taubes G.
      Epidemiology faces its limits.
      • Schoeller D.A.
      • Thomas D.
      • Archer E.
      • et al.
      Self-report-based estimates of energy intake offer an inadequate basis for scientific conclusions.
      The genesis of these criticisms is the appalling track record of highly publicized nutrition claims derived from epidemiologic studies (eg, see the studies by Stampfer et al
      • Stampfer M.J.
      • Hennekens C.H.
      • Manson J.E.
      • Colditz G.A.
      • Rosner B.
      • Willett W.C.
      Vitamin E consumption and the risk of coronary disease in women.
      and Rimm et al
      • Rimm E.B.
      • Stampfer M.J.
      • Ascherio A.
      • Giovannucci E.
      • Colditz G.A.
      • Willett W.C.
      Vitamin E consumption and the risk of coronary heart disease in men.
      ) that consistently failed to be supported when tested using objective study designs.
      • Ioannidis J.P.A.
      Implausible results in human nutrition research.
      • Tatsioni A.
      • Bonitsis N.G.
      • Ioannidis J.P.
      Persistence of contradicted claims in the literature.
      Young and Karr examined
      • Young S.S.
      • Karr A.
      Deming, data and observational studies.
      more than 50 nutritional claims from observational studies for a variety of dietary patterns and nutrient supplementation and found that “100% of the observational claims failed to replicate”p117 and that 5 claims were statistically significant “in the opposite direction.”p117 These outcomes and others
      • Moorthy D.
      • Chung M.
      • Lee J.
      • Yu W.W.
      • Lau J.
      • Trikalinos T.A.
      Concordance Between the Findings of Epidemiological Studies and Randomized Trials in Nutrition: An Empirical Evaluation and Citation Analysis.
      • Prentice R.L.
      Clinical trials and observational studies to assess the chronic disease benefits and risks of multivitamin-multimineral supplements.
      • Druesne-Pecollo N.
      • Latino-Martel P.
      • Norat T.
      • et al.
      Beta-carotene supplementation and cancer risk: a systematic review and metaanalysis of randomized controlled trials.
      • Schatzkin A.
      • Mouw T.
      • Park Y.
      • et al.
      Dietary fiber and whole-grain consumption in relation to colorectal cancer in the NIH-AARP Diet and Health Study.
      suggest that as often as not, when epidemiologic nutrition claims are tested against objective research methods, the results are either inconclusive or indicative of a contrary outcome.

      A Failed Research Paradigm

      Epidemiologic studies suggest that almost any nutrient can be associated with a myriad of outcomes,
      • Ioannidis J.P.A.
      Implausible results in human nutrition research.
      • Schoenfeld J.D.
      • Ioannidis J.P.
      Is everything we eat associated with cancer? a systematic cookbook review.
      as observed in Schoenfeld and Ioannidis’ article, “Is Everything We Eat Associated With Cancer?”
      • Schoenfeld J.D.
      • Ioannidis J.P.
      Is everything we eat associated with cancer? a systematic cookbook review.
      ,p117 With persistent cycles of specious nutrition claims in the media, it is not surprising that the public is confused and incredulous.
      • Rowe S.
      • Alexander N.
      Are nutrtion scientists communicating worse or has the situation just gotten more complicated?.
      Insofar as the provision of clear and consistent dietary guidelines for the consuming public is a goal of nutrition epidemiology, it has failed in decisively answering the simple question, “What should we eat?”

      Archer E. A wolf in sheep’s clothing. TheScientist website. http://www.the-scientist.com/?articles.view/articleNo/37918/title/Opinion–A-Wolf-in-Sheep-s-Clothing. Published October 22, 2013. Accessed February 21, 2015.

      Nowhere is this fact more evident than the shifting sands of opinion on the relative risks of fat, salt, cholesterol, and sugar.
      • Oparil S.
      Low sodium intake: cardiovascular health benefit or risk?.
      • Keys A.
      • Aravanis C.
      • Blackburn H.W.
      • et al.
      Epidemiological studies related to coronary heart disease: characteristics of men aged 40-59 in seven countries.
      • Ravnskov U.
      The questionable role of saturated and polyunsaturated fatty acids in cardiovascular disease.
      • Sievenpiper J.L.
      • de Souza R.J.
      • Kendall C.W.
      • Jenkins D.J.
      Is fructose a story of mice but not men?.
      • David Wang D.
      • Sievenpiper J.L.
      • de Souza R.J.
      • et al.
      Effect of fructose on postprandial triglycerides: a systematic review and meta-analysis of controlled feeding trials.
      Dietary Guidelines Advisory Committee
      Scientific Report of the 2015 Dietary Guidelines Advisory Committee.
      Five decades of controversy surrounding basic dietary guidelines and nutrition recommendations is a public acknowledgement of a failed research paradigm. The striking incongruence between the improvements in the nutritional status of the US population
      • Pfeiffer C.M.
      • Sternberg M.R.
      • Schleicher R.L.
      • Haynes B.M.
      • Rybak M.E.
      • Pirkle J.L.
      The CDC's Second National Report on Biochemical Indicators of Diet and Nutrition in the U.S. Population is a valuable tool for researchers and policy makers.
      CDC
      National Report on Biochemical Indicators of Diet and Nutrition in the US Population 1999–2002.
      and the current state of confusion, controversy, and clinical failure of epidemiologic nutrition research could not be clearer and necessitates an examination of the validity and value of epidemiologic nutrition research.

      Purpose of this Review

      Memory-based dietary assessment methods (M-BMs) (eg, interviews, questionnaires, and surveys
      • Thompson F.E.
      • Subar A.F.
      Dietary assessment methodology.
      • Willett W.
      Nutritional Epidemiology.
      ) are the dominant data collection protocols in national nutrition surveillance
      • Archer E.
      • Hand G.A.
      • Blair S.N.
      Validity of U.S. nutritional surveillance: National Health and Nutrition Examination Survey caloric energy intake data, 1971-2010.
      and government-funded epidemiologic nutrition
      • Dwyer J.
      • Picciano M.F.
      • Raiten D.J.
      Estimation of usual intakes: What We Eat in America-NHANES.
      and obesity
      • Archer E.
      • Hand G.A.
      • Blair S.N.
      Validity of U.S. nutritional surveillance: National Health and Nutrition Examination Survey caloric energy intake data, 1971-2010.
      research. Importantly, M-BM data are used to inform national nutritional policy and dietary guidelines.
      Dietary Guidelines Advisory Committee
      Scientific Report of the 2015 Dietary Guidelines Advisory Committee.
      The recent Scientific Report of the 2015 Dietary Guidelines Advisory Committee (DGAC) stated explicitly that most of the DGAC data analyses used the M-BMs of the National Health and Nutrition Examination Survey (NHANES) dietary component, What We Eat in America (WWEIA).
      Dietary Guidelines Advisory Committee
      Scientific Report of the 2015 Dietary Guidelines Advisory Committee.
      Although decades of unequivocal evidence demonstrate that the indirect, proxy estimates derived from M-BMs bear little relation to actual energy or nutrient consumption,
      • Schoeller D.A.
      • Thomas D.
      • Archer E.
      • et al.
      Self-report-based estimates of energy intake offer an inadequate basis for scientific conclusions.
      • Archer E.
      • Hand G.A.
      • Blair S.N.
      Validity of U.S. nutritional surveillance: National Health and Nutrition Examination Survey caloric energy intake data, 1971-2010.
      • Lissner L.
      • Troiano R.P.
      • Midthune D.
      • et al.
      OPEN about obesity: recovery biomarkers, dietary reporting errors and BMI.
      • Freedman L.S.
      • Commins J.M.
      • Moler J.E.
      • et al.
      Pooled results from 5 validation studies of dietary self-report instruments using recovery biomarkers for energy and protein intake.
      • Schoeller D.A.
      Limitations in the assessment of dietary energy intake by self-report.
      • Subar A.F.
      • Kipnis V.
      • Troiano R.P.
      • et al.
      Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: the OPEN study.
      • Goris A.H.
      • Westerterp-Plantenga M.S.
      • Westerterp K.R.
      Undereating and underrecording of habitual food intake in obese men: selective underreporting of fat intake.
      • Scagliusi F.B.
      • Polacow V.O.
      • Artioli G.G.
      • Benatti F.B.
      • Lancha Jr., A.H.
      Selective underreporting of energy intake in women: magnitude, determinants, and effect of training.
      • Heitmann B.L.
      • Lissner L.
      Can adverse effects of dietary fat intake be overestimated as a consequence of dietary fat underreporting?.
      • Heitmann B.L.
      • Lissner L.
      • Osler M.
      Do we eat less fat, or just report so?.
      • Heitmann B.L.
      • Milman N.
      • Hansen G.L.
      Relationship between dietary iron intake, corrected for diet reporting error, and serum ferritin in Danish women aged 35-65 years.
      • Black A.E.
      • Goldberg G.R.
      • Jebb S.A.
      • Livingstone M.B.
      • Cole T.J.
      • Prentice A.M.
      Critical evaluation of energy intake data using fundamental principles of energy physiology, 2: evaluating the results of published surveys.
      • Goldberg G.R.
      • Black A.E.
      • Jebb S.A.
      • et al.
      Critical evaluation of energy intake data using fundamental principles of energy physiology, 1: derivation of cut-off limits to identify under-recording.
      the underlying assumptions regarding the validity of human memory and recall in dietary assessment have not been questioned. To the contrary, M-BM data are vigorously defended as valid and inherently valuable despite no empirical support for those assertions.
      • Hébert J.
      • Hurley T.G.
      • Steck S.E.
      • et al.
      Considering the value of dietary assessment data in informing nutrition-related health policy.
      Although the relationship between 2 different constructs may be expected to be weak, the trivial relationships between the proxy estimates (ie, self-reported energy intake [EI] and nutrient intake) and their referents (ie, actual EI and nutrient intake) are unacceptable. We assert that the explanatory and predictive failure of epidemiologic nutrition research is explained by its reliance on M-BMs, and, as such, the uncritical faith in the validity and value of M-BMs has wasted significant resources and constitutes the single greatest impediment to actual scientific progress in the fields of obesity and nutrition research.
      The purpose of this review is to survey the explanatory and predictive failure of nutrition epidemiology in general,
      • Ioannidis J.P.A.
      Implausible results in human nutrition research.
      • Young S.S.
      • Karr A.
      Deming, data and observational studies.
      with a focus on the WWEIA-NHANES data,
      • Archer E.
      • Hand G.A.
      • Blair S.N.
      Validity of U.S. nutritional surveillance: National Health and Nutrition Examination Survey caloric energy intake data, 1971-2010.
      and argue that these failures are due to the reliance on M-BMs. First, we present evidence that the anecdotally derived proxy data produced by M-BMs bear little relation to actual EI or nutrient consumption.
      • Schoeller D.A.
      • Thomas D.
      • Archer E.
      • et al.
      Self-report-based estimates of energy intake offer an inadequate basis for scientific conclusions.
      • Archer E.
      • Hand G.A.
      • Blair S.N.
      Validity of U.S. nutritional surveillance: National Health and Nutrition Examination Survey caloric energy intake data, 1971-2010.
      • Lissner L.
      • Troiano R.P.
      • Midthune D.
      • et al.
      OPEN about obesity: recovery biomarkers, dietary reporting errors and BMI.
      • Freedman L.S.
      • Commins J.M.
      • Moler J.E.
      • et al.
      Pooled results from 5 validation studies of dietary self-report instruments using recovery biomarkers for energy and protein intake.
      • Schoeller D.A.
      Limitations in the assessment of dietary energy intake by self-report.
      • Subar A.F.
      • Kipnis V.
      • Troiano R.P.
      • et al.
      Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: the OPEN study.
      • Goris A.H.
      • Westerterp-Plantenga M.S.
      • Westerterp K.R.
      Undereating and underrecording of habitual food intake in obese men: selective underreporting of fat intake.
      • Scagliusi F.B.
      • Polacow V.O.
      • Artioli G.G.
      • Benatti F.B.
      • Lancha Jr., A.H.
      Selective underreporting of energy intake in women: magnitude, determinants, and effect of training.
      • Heitmann B.L.
      • Lissner L.
      Can adverse effects of dietary fat intake be overestimated as a consequence of dietary fat underreporting?.
      • Heitmann B.L.
      • Lissner L.
      • Osler M.
      Do we eat less fat, or just report so?.
      • Heitmann B.L.
      • Milman N.
      • Hansen G.L.
      Relationship between dietary iron intake, corrected for diet reporting error, and serum ferritin in Danish women aged 35-65 years.
      • Black A.E.
      • Goldberg G.R.
      • Jebb S.A.
      • Livingstone M.B.
      • Cole T.J.
      • Prentice A.M.
      Critical evaluation of energy intake data using fundamental principles of energy physiology, 2: evaluating the results of published surveys.
      • Goldberg G.R.
      • Black A.E.
      • Jebb S.A.
      • et al.
      Critical evaluation of energy intake data using fundamental principles of energy physiology, 1: derivation of cut-off limits to identify under-recording.
      Second, we provide interdisciplinary evidence that human memory is an amalgam of constructive and reconstructive processes
      • Johnson M.K.
      • Raye C.L.
      • Mitchell K.J.
      • Ankudowich E.
      The cognitive neuroscience of true and false memories.
      • Schacter D.L.
      Constructive memory: past and future.
      • Schacter D.L.
      • Slotnick S.D.
      The cognitive neuroscience of memory distortion.
      • Patihis L.
      • Frenda S.J.
      • LePort A.K.R.
      • et al.
      False memories in highly superior autobiographical memory individuals.
      • Schooler J.W.
      The distinctions of false and fuzzy memories.
      • Loftus E.
      Our changeable memories: legal and practical implications.
      (eg, imagination
      • Goff L.
      • Roediger H.
      Imagination inflation for action events: repeated imaginings lead to illusory recollections.
      ) that render the archival model of human memory
      • Brockmeier J.
      After the archive: remapping memory.
      and the naive assumption that recall provides literal, accurate, or precise reproductions of past events indisputably false.
      • Patihis L.
      • Frenda S.J.
      • LePort A.K.R.
      • et al.
      False memories in highly superior autobiographical memory individuals.
      • Loftus E.
      Our changeable memories: legal and practical implications.
      • Bernstein D.M.
      • Loftus E.F.
      The consequences of false memories for food preferences and choices.
      • Loftus E.F.
      The malleability of human memory.
      • Schacter D.L.
      • Loftus E.F.
      Memory and law: what can cognitive neuroscience contribute?.
      • Bernard H.R.
      • Killworth P.
      • Kronenfeld D.
      • Sailer L.
      The problem of informant accuracy: the validity of retrospective data.
      Third, M-BMs require respondents to undergo protocols

      Centers for Disease Control and Prevention. NHANES MEC in-person dietary interviewers procedures manual. http://www.cdc.gov/nchs/data/nhanes/nhanes_09_10/DietaryInterviewers_Inperson.pdf. Published January 2009. Accessed August 9, 2014.

      and perform behaviors
      • Thompson F.E.
      • Subar A.F.
      Dietary assessment methodology.
      that mimic procedures known to induce false recall.
      • Patihis L.
      • Frenda S.J.
      • LePort A.K.R.
      • et al.
      False memories in highly superior autobiographical memory individuals.
      • Loftus E.
      Our changeable memories: legal and practical implications.
      • Goff L.
      • Roediger H.
      Imagination inflation for action events: repeated imaginings lead to illusory recollections.
      • Shaw J.
      • Porter S.
      Constructing rich false memories of committing crime.
      • Bernstein D.M.
      • Laney C.
      • Morris E.K.
      • Loftus E.F.
      False beliefs about fattening foods can have healthy consequences.
      Fourth, the subjective (ie, private, not publicly accessible) mental phenomena (ie, memories) from which M-BM data are derived are not subject to independent observation, quantification, falsification, or verification; as such, M-BM data are pseudoscientific and inadmissible in scientific research.
      • Platt J.R.
      Strong inference: certain systematic methods of scientific thinking may produce much more rapid progress than others.
      • Sober E.
      Philosophy of Biology.
      • Popper K.
      Conjectures and Refutations: The Growth of Scientific Knowledge.
      • Popper K.R.
      The Logic of Scientific Discovery.
      • Popper K.R.
      Objective Knowledge: An Evolutionary Approach.
      Fifth, the failure to accurately and objectively measure and control for physical activity (PA), cardiorespiratory fitness (CRF), and other obvious confounders annuls inferences regarding diet-health relationships.

      The M-BMs of Nutrition Epidemiology

      Self-reported Dietary Intake

      The primary methods of data collection for nutrition epidemiologic research (eg, the WWEIA-NHANES) are M-BMs (eg, 24-hour dietary recalls [24HRs] and food frequency questionnaires [FFQs]
      • Thompson F.E.
      • Subar A.F.
      Dietary assessment methodology.
      • Willett W.
      Nutritional Epidemiology.
      • Archer E.
      • Hand G.A.
      • Blair S.N.
      Validity of U.S. nutritional surveillance: National Health and Nutrition Examination Survey caloric energy intake data, 1971-2010.
      ). For clarity, these methods do not directly or objectively measure EI or nutrient intake, and neither do they directly or objectively measure food and beverage consumption. The actual data derived from M-BMs are the a priori numeric values from nutrient databases that are assigned by researchers to the participants’ reports of their memories of past eating and drinking behaviors. In other words, nutrition researchers designate numeric values to whatever the respondents are willing or able to recall about what they think (or would like the researcher to think
      • Heitmann B.L.
      Social desirability bias in dietary self-report may compromise the validity of dietary intake measures. Implications for diet disease relationships.
      ) he or she consumed during the study period. Given the indirect, pseudoquantitative (ie, number-generating
      • Michell J.
      Measurement: a beginner's guide.
      ) nature of M-BMs and the fact that the respondents’ reports of their memories are subject to intentional and unintentional distorting factors (eg, perceptual, encoding, and retrieval errors
      • Straube B.
      An overview of the neuro-cognitive processes involved in the encoding, consolidation, and retrieval of true and false memories.
      ; social desirability
      • Heitmann B.L.
      • Lissner L.
      • Osler M.
      Do we eat less fat, or just report so?.
      ; false memories
      • Bernstein D.M.
      • Loftus E.F.
      The consequences of false memories for food preferences and choices.
      ; and omissions
      • Schacter D.L.
      Constructive memory: past and future.
      • Schacter D.L.
      • Slotnick S.D.
      The cognitive neuroscience of memory distortion.
      • Schacter D.
      The Seven Sins of Memory.
      ), it is not surprising that most conclusions drawn from these number-generating protocols have not been supported when subjected to rigorous objective examination.
      • Ioannidis J.P.A.
      Implausible results in human nutrition research.
      • Young S.S.
      • Karr A.
      Deming, data and observational studies.

      The Implausibility of M-BMs in Dietary Assessment

      It is the natural tendency of the ignorant to believe what is not true. In order to overcome that tendency it is not sufficient to exhibit the true; it is also necessary to expose and denounce the false.H. L. Mencken

      Mencken HL. Counter-Offensive. The American Mercury Magazine. Vol May. Whitefish, MT: Kessinger Publishing, LLC; 1926:123-124.

      ,p124
      Research into M-BMs reports a wide range of EIs that are not physiologically plausible (ie, incompatible with survival) and that do not accurately quantify the foods and nutrients consumed.
      • Ioannidis J.P.A.
      Implausible results in human nutrition research.
      • Archer E.
      • Hand G.A.
      • Blair S.N.
      Validity of U.S. nutritional surveillance: National Health and Nutrition Examination Survey caloric energy intake data, 1971-2010.
      • Lissner L.
      • Troiano R.P.
      • Midthune D.
      • et al.
      OPEN about obesity: recovery biomarkers, dietary reporting errors and BMI.
      • Subar A.F.
      • Kipnis V.
      • Troiano R.P.
      • et al.
      Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: the OPEN study.
      • Goris A.H.
      • Westerterp-Plantenga M.S.
      • Westerterp K.R.
      Undereating and underrecording of habitual food intake in obese men: selective underreporting of fat intake.
      • Scagliusi F.B.
      • Polacow V.O.
      • Artioli G.G.
      • Benatti F.B.
      • Lancha Jr., A.H.
      Selective underreporting of energy intake in women: magnitude, determinants, and effect of training.
      • Heitmann B.L.
      • Lissner L.
      • Osler M.
      Do we eat less fat, or just report so?.
      Recently, we used multiple methods to ascertain the validity and plausibility of the NHANES and WWEIA-NHANES EI data from 1971 to 2010
      • Archer E.
      • Hand G.A.
      • Blair S.N.
      Validity of U.S. nutritional surveillance: National Health and Nutrition Examination Survey caloric energy intake data, 1971-2010.
      and found that they had such severe systematic biases as to render them fatally flawed. Given that “[a]cross the 39-year history of the NHANES, [self-reported energy intake] data on the majority of respondents (67.3% of women and 58.7% of men) were not physiologically plausible”
      • Archer E.
      • Hand G.A.
      • Blair S.N.
      Validity of U.S. nutritional surveillance: National Health and Nutrition Examination Survey caloric energy intake data, 1971-2010.
      (Figure), we concluded that these data are not valid for any inferences regarding EI and the etiology of the obesity epidemic. A recent editorial in the British Medical Journal concurred and stated that the NHANES dietary data are “incompatible with life.”
      • Ioannidis J.P.A.
      Implausible results in human nutrition research.
      ,p7
      Figure thumbnail gr1
      FigurePercentage of implausible reporters by body mass index (BMI) for US women aged 20 to 74 years in the National Health and Nutrition Examination Survey (NHANES) (1971-2010). Physiologically implausible values were determined via the following equation: (reported energy intake/basal metabolic rate) ≤1.35. Implausible values may be considered “incompatible with life.”
      • Ioannidis J.P.A.
      Implausible results in human nutrition research.
      ,p7
      In a previous report,
      • Archer E.
      • Hand G.A.
      • Blair S.N.
      Validity of U.S. nutritional surveillance: National Health and Nutrition Examination Survey caloric energy intake data, 1971-2010.
      we used 2 objective, physiologically based methods to determine misreporting: (1) Goldberg cutoff values
      • Black A.E.
      • Goldberg G.R.
      • Jebb S.A.
      • Livingstone M.B.
      • Cole T.J.
      • Prentice A.M.
      Critical evaluation of energy intake data using fundamental principles of energy physiology, 2: evaluating the results of published surveys.
      • Goldberg G.R.
      • Black A.E.
      • Jebb S.A.
      • et al.
      Critical evaluation of energy intake data using fundamental principles of energy physiology, 1: derivation of cut-off limits to identify under-recording.
      • Black A.E.
      Critical evaluation of energy intake using the Goldberg cut-off for energy intake:basal metabolic rate: a practical guide to its calculation, use and limitations.
      (ie, reported EI [rEI] divided by basal metabolic rate [BMR]) and (2) the disparity between the Institute of Medicine total energy expenditure (TEE) equations
      Institute of Medicine
      Report of the Panel on Macronutrients: Dietary Reference Intakes for Energy, Carbohydrate, Fiber, Fat, Fatty Acids, Cholesterol, Protein, and Amino Acids (Macronutrients).
      and rEI via NHANES M-BMs. The 2 methods were in close agreement, demonstrating significant misreporting. The cutoff values we used (ie, rEI/BMR <1.35 and >2.40) were more generous than the rEI/BMR cutoff value of 1.50 suggested by Goldberg et al
      • Goldberg G.R.
      • Black A.E.
      • Jebb S.A.
      • et al.
      Critical evaluation of energy intake data using fundamental principles of energy physiology, 1: derivation of cut-off limits to identify under-recording.
      when using a single 24HR, and the BMR is “predicted from the Schofield equations” with a sample size of 300 or greater.
      • Goldberg G.R.
      • Black A.E.
      • Jebb S.A.
      • et al.
      Critical evaluation of energy intake data using fundamental principles of energy physiology, 1: derivation of cut-off limits to identify under-recording.
      ,p577 Given the reduced sensitivity of the cutoff values, we captured far fewer underreporters. As reported, when using the proposed cutoff value of 1.50, underreporting increased to more than 70% for the entire NHANES sample and to approximately 77% and 85% for obese men and women, respectively. We also reported the large and significant disparity between rEI and the Institute of Medicine TEE: –467 and –554 kcal/d (>17% and 30%) for obese men and women, respectively. In addition to underreporting, there was significant overreporting in all of the subpopulations (eg, normal, overweight, and obese men and women). One important caveat with the use of cutoff values is that the term plausible reporter is not synonymous with accurate reporter. Participants with high levels of PA may substantially underreport yet still be considered plausible reporters.
      Given these results, we ask 4 questions, (1) What is the value of WWEIA-NHANES M-BM data if 70% to 80% of obese women’s self-reported EI is physiologically implausible and, therefore, incompatible with life (Figure)? (2) Given the extant objective data on the nutrition-related health status of Americans,
      • Pfeiffer C.M.
      • Sternberg M.R.
      • Schleicher R.L.
      • Haynes B.M.
      • Rybak M.E.
      • Pirkle J.L.
      The CDC's Second National Report on Biochemical Indicators of Diet and Nutrition in the U.S. Population is a valuable tool for researchers and policy makers.
      why does the DGAC rely on the subjective M-BM data?
      Dietary Guidelines Advisory Committee
      Scientific Report of the 2015 Dietary Guidelines Advisory Committee.
      (3) What is the “unrealized potential”
      • Hébert J.
      • Hurley T.G.
      • Steck S.E.
      • et al.
      Considering the value of dietary assessment data in informing nutrition-related health policy.
      ,p447 and “utility”
      • Satija A.
      • Yu E.
      • Willett W.C.
      • Hu F.B.
      Understanding nutritional epidemiology and its role in policy.
      ,p5 of these data when implausible overreporting and implausible underreporting are demonstrated in all of the subgroups? (4) Can statistical alchemy transform these implausible data into valid estimates of dietary consumption, or will it continue to spawn searches for machinations that generate numbers with improved correlations (ie, post hoc data manipulation) while ignoring the lack of validity?

      The Pervasiveness of Implausible Results

      The conclusions drawn by our study
      • Archer E.
      • Hand G.A.
      • Blair S.N.
      Validity of U.S. nutritional surveillance: National Health and Nutrition Examination Survey caloric energy intake data, 1971-2010.
      and the recent British Medical Journal editorial
      • Ioannidis J.P.A.
      Implausible results in human nutrition research.
      are, in fact, supported by many decades of evidence demonstrating that M-BMs have severe, intractable systematic biases that render the data implausible and, therefore, invalid.
      • Ioannidis J.P.A.
      Implausible results in human nutrition research.
      • Schoeller D.A.
      • Thomas D.
      • Archer E.
      • et al.
      Self-report-based estimates of energy intake offer an inadequate basis for scientific conclusions.
      • Schoeller D.A.
      Limitations in the assessment of dietary energy intake by self-report.
      • Black A.E.
      • Goldberg G.R.
      • Jebb S.A.
      • Livingstone M.B.
      • Cole T.J.
      • Prentice A.M.
      Critical evaluation of energy intake data using fundamental principles of energy physiology, 2: evaluating the results of published surveys.

      Dhurandhar NV, Schoeller D, Brown AW, et al. Energy balance measurement: when something is not better than nothing [published online November 13, 2014]. Int J Obes (Lond).

      • Archer E.
      • Blair S.N.
      Implausible data, false memories, and the status quo in dietary assessment.
      Research with “… motivated … well-educated, non-smoking Caucasians”
      • Lissner L.
      • Troiano R.P.
      • Midthune D.
      • et al.
      OPEN about obesity: recovery biomarkers, dietary reporting errors and BMI.
      ,p957 (ie, respondents less likely to misreport) demonstrated that compared with doubly labeled water, a biomarker for TEE, self-reported dietary intake was significantly misestimated.
      • Lissner L.
      • Troiano R.P.
      • Midthune D.
      • et al.
      OPEN about obesity: recovery biomarkers, dietary reporting errors and BMI.
      • Subar A.F.
      • Kipnis V.
      • Troiano R.P.
      • et al.
      Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: the OPEN study.
      Men underreported EI 12% to 14% using the average of two 24HRs and 31% to 36% using FFQs. Women underreported by 16% to 20% using the average of two 24HRs and by 34% to 38% using FFQs. Contrary to the oft-repeated statement that additional self-reports improve precision and accuracy, the second administration of the 24HR “showed greater underreporting.”
      • Subar A.F.
      • Kipnis V.
      • Troiano R.P.
      • et al.
      Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: the OPEN study.
      ,p12 These results are in agreement with our analyses of the NHANES in which the mean estimates for the second 24HR in every NHANES wave from 2001 to 2010 exhibited significantly greater levels of underreporting than the first. We agree with the authors of the Observing Protein and Energy Nutrition study when they wrote, “[w]e measure energy so poorly …”
      • Subar A.F.
      • Kipnis V.
      • Troiano R.P.
      • et al.
      Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: the OPEN study.
      ,p12 and “[t]he 24HR … may be particularly problematic in the obese.”
      • Lissner L.
      • Troiano R.P.
      • Midthune D.
      • et al.
      OPEN about obesity: recovery biomarkers, dietary reporting errors and BMI.
      ,p956 These words echo statements on underreporting from 60 years ago.
      • Beaudoin R.
      • Mayer J.
      Food intakes of obese and non-obese women.
      Recently, some of the strongest proponents of M-BMs have provided additional data that clearly demonstrate the futility of the continued use of these methods.
      • Freedman L.S.
      • Commins J.M.
      • Moler J.E.
      • et al.
      Pooled results from 5 validation studies of dietary self-report instruments using recovery biomarkers for energy and protein intake.
      In the paper by Freedman et al,
      • Freedman L.S.
      • Commins J.M.
      • Moler J.E.
      • et al.
      Pooled results from 5 validation studies of dietary self-report instruments using recovery biomarkers for energy and protein intake.
      the pooled, squared average correlation between true EI and self-reported EI were similar to our results using NHANES data, ranging from 0.04 to 0.10. This suggests that the measurement noise (ie, error) is more than 9 times greater than the signal (ie, valid information) derived from M-BMs. Nevertheless, an important finding from the Observing Protein and Energy Nutrition study that Freedman et al
      • Freedman L.S.
      • Commins J.M.
      • Moler J.E.
      • et al.
      Pooled results from 5 validation studies of dietary self-report instruments using recovery biomarkers for energy and protein intake.
      overlook in their analyses is that despite the fact that the second administration of the 24HR “showed greater underreporting,”
      • Subar A.F.
      • Kipnis V.
      • Troiano R.P.
      • et al.
      Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: the OPEN study.
      ,p12 the correlations between true and reported EI increased. This demonstrates an increase in precision with a concomitant reduction in the accuracy of the estimate. These results clearly support our position that M-BM data “offer an inadequate basis for scientific conclusions”
      • Schoeller D.A.
      • Thomas D.
      • Archer E.
      • et al.
      Self-report-based estimates of energy intake offer an inadequate basis for scientific conclusions.
      ,p1413 and, more importantly, that statistical machinations, however sophisticated, cannot overcome the systematic recall bias that renders all inferences suspect.
      • Heitmann B.L.
      • Lissner L.
      Can adverse effects of dietary fat intake be overestimated as a consequence of dietary fat underreporting?.
      • Bellach B.
      • Kohlmeier L.
      Energy adjustment does not control for differential recall bias in nutritional epidemiology.
      The phenomenon of misreporting is not limited to US epidemiologic studies or specific populations.
      • Goldberg G.R.
      • Black A.E.
      • Jebb S.A.
      • et al.
      Critical evaluation of energy intake data using fundamental principles of energy physiology, 1: derivation of cut-off limits to identify under-recording.
      The European Prospective Investigation Into Cancer and Nutrition study is one of the largest epidemiologic studies in the world and found strong evidence of systemic underreporting across all study sites, with approximately 10% to 14% of survey respondents being “extreme underreporters,”
      • Ferrari P.
      • Slimani N.
      • Ciampi A.
      • et al.
      Evaluation of under- and overreporting of energy intake in the 24-hour diet recalls in the European Prospective Investigation into Cancer and Nutrition (EPIC).
      ,p1329 and “… most centres were below the expected reference value.”
      • Ferrari P.
      • Slimani N.
      • Ciampi A.
      • et al.
      Evaluation of under- and overreporting of energy intake in the 24-hour diet recalls in the European Prospective Investigation into Cancer and Nutrition (EPIC).
      ,p1330 These results are consistent with research from the early 1990s that found that more than 65% of the mean rEI values were physiologically implausible in 37 studies across 10 countries.
      • Goldberg G.R.
      • Black A.E.
      • Jebb S.A.
      • et al.
      Critical evaluation of energy intake data using fundamental principles of energy physiology, 1: derivation of cut-off limits to identify under-recording.
      The misreporting value of more than 65% is strikingly similar to our NHANES results using similar methods.
      • Archer E.
      • Hand G.A.
      • Blair S.N.
      Validity of U.S. nutritional surveillance: National Health and Nutrition Examination Survey caloric energy intake data, 1971-2010.
      In 2015, a multinational report demonstrated that misreporting “in five populations of the African diaspora”
      • Orcholski L.
      • Luke A.
      • Plange-Rhule J.
      • et al.
      Under-reporting of dietary energy intake in five populations of the African diaspora.
      ,p464 was substantial, with the South African cohort exhibiting an astounding 52.1% underreporting of dietary EI.
      • Orcholski L.
      • Luke A.
      • Plange-Rhule J.
      • et al.
      Under-reporting of dietary energy intake in five populations of the African diaspora.
      With respect to age, Forrestal
      • Forrestal S.G.
      Energy intake misreporting among children and adolescents: a literature review.
      found in children and adolescents that misreporting “… appeared to be more common than it is among adults.”p112 The ubiquitous nature of misreporting and the consistency of research results over many decades and across multiple populations, cohorts, and countries provide strong support that M-BM measures of EI are fatally flawed, and, therefore, diet-health inferences from studies that use M-BMs are essentially meaningless.

      Examinations of Dietary Patterns via M-BMs

      It is well-established that specific macronutrients, foods, beverages, and food groups (eg, protein, fat, carbohydrate, alcohol, sugar, and vegetables) are subject to differential misreporting that significantly affects subsequent estimates of EI.
      • Subar A.F.
      • Kipnis V.
      • Troiano R.P.
      • et al.
      Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: the OPEN study.
      • Bellach B.
      • Kohlmeier L.
      Energy adjustment does not control for differential recall bias in nutritional epidemiology.
      • Pomerleau J.
      • Ostbye T.
      • Bright-See E.
      Potential underreporting of energy intake in the Ontario Health Survey and its relationship with nutrient and food intakes.
      • Poslusna K.
      • Ruprich J.
      • de Vries J.H.
      • Jakubikova M.
      • van't Veer P.
      Misreporting of energy and micronutrient intake estimated by food records and 24 hour recalls, control and adjustment methods in practice.

      Banna JC, Fialkowski MK, Townsend MS. Misreporting of dietary intake affects estimated nutrient intakes in low-income Spanish-speaking women [published online August 12, 2014]. J Acad Nutr Diet.

      • Hill R.J.
      • Davies P.S.
      The validity of self-reported energy intake as determined using the doubly labelled water technique.
      • Lafay L.
      • Mennen L.
      • Basdevant A.
      • et al.
      Does energy intake underreporting involve all kinds of food or only specific food items? results from the Fleurbaix Laventie Ville Sante (FLVS) study.
      • Westerterp K.R.
      • Goris A.H.
      Validity of the assessment of dietary intake: problems of misreporting.
      • Krebs-Smith S.M.
      • Graubard B.I.
      • Kahle L.L.
      • Subar A.F.
      • Cleveland L.E.
      • Ballard-Barbash R.
      Low energy reporters vs others: a comparison of reported food intakes.
      Because EI is the foundation of dietary consumption and all nutrients must be consumed within the quantity of food and beverages needed to meet minimum energy requirements,
      • Livingstone M.B.
      • Black A.E.
      Markers of the validity of reported energy intake.
      it is a logical and analytic truth that dietary patterns (ie, macronutrient and micronutrient consumption; eg, protein, carbohydrate, fat, vitamins, and minerals) are differentially and unpredictably misreported when total rEI is physiologically implausible. For example, both macronutrient and micronutrient composition are significantly altered in underreporters, with reported fat and carbohydrate consumption often lower and reported protein, fruit, and vegetable intakes higher.
      • Heitmann B.L.
      • Lissner L.
      • Osler M.
      Do we eat less fat, or just report so?.
      • Pomerleau J.
      • Ostbye T.
      • Bright-See E.
      Potential underreporting of energy intake in the Ontario Health Survey and its relationship with nutrient and food intakes.
      • Lafay L.
      • Mennen L.
      • Basdevant A.
      • et al.
      Does energy intake underreporting involve all kinds of food or only specific food items? results from the Fleurbaix Laventie Ville Sante (FLVS) study.
      In other words, participants qualitatively and quantitatively misreport owing to both unintentional (eg, forgetting and false memories) and intentional (eg, health-related perceptions) factors. This nonuniformity of misreporting leads to macronutrient- and micronutrient-specific errors,
      • Lafay L.
      • Mennen L.
      • Basdevant A.
      • et al.
      Does energy intake underreporting involve all kinds of food or only specific food items? results from the Fleurbaix Laventie Ville Sante (FLVS) study.
      • Westerterp K.R.
      • Goris A.H.
      Validity of the assessment of dietary intake: problems of misreporting.
      which alter nutrient to EI ratios in an unpredictable and nonquantifiable manner. This simple fact renders energy adjustments fallacious
      • Heitmann B.L.
      • Lissner L.
      Can adverse effects of dietary fat intake be overestimated as a consequence of dietary fat underreporting?.
      • Bellach B.
      • Kohlmeier L.
      Energy adjustment does not control for differential recall bias in nutritional epidemiology.
      and demonstrates that the assumption that M-BM data can be used to examine patterns of diet or dietary composition is not logically valid.

      The Validity of Human Memory and Recall as Instruments for the Generation of Scientific Data

      Overview

      The use of M-BMs requires faith in the belief that human perception, memory, and recall are accurate and reliable instruments for the generation of scientific data. Nevertheless, more than 80 years of research demonstrates that this belief is patently false.
      • Patihis L.
      • Frenda S.J.
      • LePort A.K.R.
      • et al.
      False memories in highly superior autobiographical memory individuals.
      • Bernard H.R.
      • Killworth P.
      • Kronenfeld D.
      • Sailer L.
      The problem of informant accuracy: the validity of retrospective data.
      • Schacter D.
      The Seven Sins of Memory.
      • Schacter D.L.
      • Norman K.A.
      • Koutstaal W.
      The cognitive neuroscience of constructive memory.
      The discrepancy between objective reality and human memory is well established,
      • Schacter D.L.
      Constructive memory: past and future.
      • Johnson M.K.
      Memory and reality.
      and the limitations of recall are widely acknowledged in disciplines outside of nutrition and obesity.
      • Johnson M.K.
      • Raye C.L.
      • Mitchell K.J.
      • Ankudowich E.
      The cognitive neuroscience of true and false memories.
      • Schacter D.L.
      Constructive memory: past and future.
      • Schacter D.L.
      • Slotnick S.D.
      The cognitive neuroscience of memory distortion.
      • Straube B.
      An overview of the neuro-cognitive processes involved in the encoding, consolidation, and retrieval of true and false memories.
      • Schacter D.
      The Seven Sins of Memory.
      • Gallo D.A.
      False memories and fantastic beliefs: 15 years of the DRM illusion.
      In fact, the scientific study and analysis of memory would be impossible if it were not for the inherent fallibility of memory.
      • Schacter D.L.
      • Slotnick S.D.
      The cognitive neuroscience of memory distortion.
      Bartlett
      • Bartlett F.C.
      Remembering: A Study in Experimental and Social Psychology.
      presented the first empirical evidence that the human memory is not a literal, accurate, or precise reproduction of past events. During the ensuing 80 years, research has clearly demonstrated that the encoding of memories
      • Straube B.
      An overview of the neuro-cognitive processes involved in the encoding, consolidation, and retrieval of true and false memories.
      • Johnson M.K.
      Memory and reality.
      and subsequent recall depend on constructive and reconstructive processes (eg, imagination)
      • Schacter D.L.
      Constructive memory: past and future.
      • Straube B.
      An overview of the neuro-cognitive processes involved in the encoding, consolidation, and retrieval of true and false memories.
      • Goff L.
      • Roediger H.
      Imagination inflation for action events: repeated imaginings lead to illusory recollections.
      that are susceptible to errors, distortions, omissions, complete fabrications, false reports, and illusions.
      • Patihis L.
      • Frenda S.J.
      • LePort A.K.R.
      • et al.
      False memories in highly superior autobiographical memory individuals.
      • Bernard H.R.
      • Killworth P.
      • Kronenfeld D.
      • Sailer L.
      The problem of informant accuracy: the validity of retrospective data.
      • Straube B.
      An overview of the neuro-cognitive processes involved in the encoding, consolidation, and retrieval of true and false memories.
      • Schacter D.
      The Seven Sins of Memory.
      • Schacter D.L.
      • Norman K.A.
      • Koutstaal W.
      The cognitive neuroscience of constructive memory.
      Given the breadth of this research, reported memories, such as those presented in 24HRs and FFQs, can be most accurately defined as mere attributions based on mental experiences that are strongly influenced by the respondents’ idiosyncratic qualities (ie, education), previous memories and information, knowledge and beliefs, motives, goals, habitual behavior, and the social context in which the memories are encoded or reported.
      • Johnson M.K.
      • Raye C.L.
      • Mitchell K.J.
      • Ankudowich E.
      The cognitive neuroscience of true and false memories.
      • Schacter D.L.
      • Slotnick S.D.
      The cognitive neuroscience of memory distortion.
      • Bernard H.R.
      • Killworth P.
      • Kronenfeld D.
      • Sailer L.
      The problem of informant accuracy: the validity of retrospective data.
      Perhaps the most salient example of the fallibility of memory and recall (and misplaced confidence) is that false reporting (ie, inaccurate eyewitness testimony) was a key factor in approximately 75% of the first 100 cases of individuals exonerated by DNA evidence after conviction for crimes that they did not commit.
      • Schacter D.L.
      • Loftus E.F.
      Memory and law: what can cognitive neuroscience contribute?.
      The following subsections provide a survey of the evidence to support the contention that data can be only as valid as the accuracy of the instrument used in its collection and that human memory and recall are not valid instruments for the generation of data to be used in the scientific formulation of nutrition guidelines.

      The Social Sciences

      Numerous studies, dating back more than 50 years, have reported that there is little or no correlation between self-reported behavior and actual behavior.
      • Storms M.D.
      • Nisbett R.E.
      Insomnia and the attribution process.
      • Zimbardo P.G.
      • Cohen A.
      • Weisenberg M.
      • Dworkin L.
      • Firestone I.
      The control of experimental pain.
      Bernard et al
      • Bernard H.R.
      • Killworth P.
      • Kronenfeld D.
      • Sailer L.
      The problem of informant accuracy: the validity of retrospective data.
      reviewed the validity of self-reported data in “The Problem of Informant Accuracy.” Surveying multiple research domains, including health care, child care, communications, nutrition, criminal justice, economics, anthropology, and psychology, Bernard et al
      • Bernard H.R.
      • Killworth P.
      • Kronenfeld D.
      • Sailer L.
      The problem of informant accuracy: the validity of retrospective data.
      concluded that “[t]he results of all of these studies leads to one overwhelming conclusion: on average, about half of what informants report is probably incorrect in some way.”p503 Bernard et al
      • Bernard H.R.
      • Killworth P.
      • Kronenfeld D.
      • Sailer L.
      The problem of informant accuracy: the validity of retrospective data.
      also provide a prescient commentary: “In sum, despite the evidence, the basic fact of informant inaccuracy seems not to have penetrated either graduate training or professional social science research. Informant inaccuracy remains both a fugitive problem and a well-kept open secret.”p504 Given the substantial funding of M-BMs each year,

      American Association for the Advancement of Science. Historical trends in federal R&D. http://www.aaas.org/page/historical-trends-federal-rd. Accessed February 21, 2015.

      it seems that this 30-year-old commentary also applies to nutrition and obesity research.
      Furthermore, when events or behaviors are commonplace (eg, food and beverage consumption), previous experiences (eg, previous memories and mental schema
      • Straube B.
      An overview of the neuro-cognitive processes involved in the encoding, consolidation, and retrieval of true and false memories.
      • Mandler J.M.
      Categorical and schematic organization in memory.
      of past meals) will determine what is encoded in memory and not the actual perception of behavior. For example, Freeman et al
      • Freeman L.C.
      • Romney A.K.
      • Freeman S.C.
      Cognitive structure and informant accuracy.
      demonstrated a 52% error rate in recalling social interactions, with reports of social interactions shaped by typical past experiences. They explain their results by suggesting that when events are repeatedly experienced, each specific event will be minimally processed and the “actual memory of such elements will be poor,” and “attempts at recall result in a constructive process that taps into the general structure rather than the specific memory.”
      • Freeman L.C.
      • Romney A.K.
      • Freeman S.C.
      Cognitive structure and informant accuracy.
      ,p315
      Importantly, Bernard et al
      • Bernard H.R.
      • Killworth P.
      • Kronenfeld D.
      • Sailer L.
      The problem of informant accuracy: the validity of retrospective data.
      lamented 2 common problems with social scientific data: (1) the lack of an explicit formal theory of human behavior and (2) objective evidence from which to test the plausibility of self-reported data. Nevertheless, nutrition epidemiologists have both a formal theory (ie, human metabolism and the basic energy requirements of human life) and voluminous objective data
      • Black A.E.
      • Goldberg G.R.
      • Jebb S.A.
      • Livingstone M.B.
      • Cole T.J.
      • Prentice A.M.
      Critical evaluation of energy intake data using fundamental principles of energy physiology, 2: evaluating the results of published surveys.
      • Goldberg G.R.
      • Black A.E.
      • Jebb S.A.
      • et al.
      Critical evaluation of energy intake data using fundamental principles of energy physiology, 1: derivation of cut-off limits to identify under-recording.
      by which to test the validity of M-BMs.
      • Archer E.
      • Hand G.A.
      • Blair S.N.
      Validity of U.S. nutritional surveillance: National Health and Nutrition Examination Survey caloric energy intake data, 1971-2010.
      Despite the availability of formal theory and overwhelming evidence that self-reported EI data are not accurate, “plausible,”
      • Archer E.
      • Hand G.A.
      • Blair S.N.
      Validity of U.S. nutritional surveillance: National Health and Nutrition Examination Survey caloric energy intake data, 1971-2010.
      or even “compatible with life,”
      • Ioannidis J.P.A.
      Implausible results in human nutrition research.
      ,p7 self-reported EI continues to be assumed a valid measure of actual energy and nutrient consumption that can be used to inform public nutrition and dietary policy.
      Dietary Guidelines Advisory Committee
      Scientific Report of the 2015 Dietary Guidelines Advisory Committee.
      A detailed review of the social research literature is beyond the scope of this paper, and we direct our readers to Bernard et al’s review.
      • Bernard H.R.
      • Killworth P.
      • Kronenfeld D.
      • Sailer L.
      The problem of informant accuracy: the validity of retrospective data.
      Nevertheless, one more notable example is warranted. Immediately on leaving a restaurant, Kronenfeld et al
      • Kronenfeld D.B.
      • Kronenfeld J.
      • Kronenfeld J.E.
      Toward a science of design for successful food service.
      had participants report on the attire of the waitstaff and the restaurants’ choice of music.
      • Bernard H.R.
      • Killworth P.
      • Kronenfeld D.
      • Sailer L.
      The problem of informant accuracy: the validity of retrospective data.
      Participants demonstrated much greater agreement on what the waiters were wearing compared with the waitresses’ attire. The interesting finding was that these restaurants had an all-female waitstaff (ie, there were no waiters in the restaurants). Participants also provided much greater detail on the music from restaurants that were not playing music than from restaurants that were.
      • Bernard H.R.
      • Killworth P.
      • Kronenfeld D.
      • Sailer L.
      The problem of informant accuracy: the validity of retrospective data.
      • Kronenfeld D.B.
      • Kronenfeld J.
      • Kronenfeld J.E.
      Toward a science of design for successful food service.
      These results raise the question: What is the possibility that self-reported food and beverage consumption in a restaurant setting will be a literal, accurate, or reliable representation of actual ingestive behavior?

      Cognitive Neuroscience

      The domain of cognitive neuroscience supports the hypothesis that human memory is an amalgam of dynamic constructive and reconstructive processes.
      • Johnson M.K.
      • Raye C.L.
      • Mitchell K.J.
      • Ankudowich E.
      The cognitive neuroscience of true and false memories.
      • Schacter D.L.
      Constructive memory: past and future.
      • Schacter D.L.
      • Slotnick S.D.
      The cognitive neuroscience of memory distortion.
      • Patihis L.
      • Frenda S.J.
      • LePort A.K.R.
      • et al.
      False memories in highly superior autobiographical memory individuals.
      • Schooler J.W.
      The distinctions of false and fuzzy memories.
      • Loftus E.
      Our changeable memories: legal and practical implications.
      • Goff L.
      • Roediger H.
      Imagination inflation for action events: repeated imaginings lead to illusory recollections.
      • Bernstein D.M.
      • Loftus E.F.
      The consequences of false memories for food preferences and choices.
      • Loftus E.F.
      The malleability of human memory.
      • Schacter D.L.
      • Loftus E.F.
      Memory and law: what can cognitive neuroscience contribute?.
      • Straube B.
      An overview of the neuro-cognitive processes involved in the encoding, consolidation, and retrieval of true and false memories.
      • Schacter D.
      The Seven Sins of Memory.
      For example, encoding is not a process that begins de novo with each perception. Encoding is the result of the limited amount of information available to perception at any given moment being “patched together to form memories with varying degrees of accuracy”
      • Schacter D.L.
      • Slotnick S.D.
      The cognitive neuroscience of memory distortion.
      ,p149 (eg, the process of associative grouping via semantic relatedness
      • Patihis L.
      • Frenda S.J.
      • LePort A.K.R.
      • et al.
      False memories in highly superior autobiographical memory individuals.
      • Gallo D.A.
      False memories and fantastic beliefs: 15 years of the DRM illusion.
      • Gallo D.A.
      • Roberts M.J.
      • Seamon J.G.
      Remembering words not presented in lists: can we avoid creating false memories?.
      ) and subject to “the distorting influences of present knowledge, beliefs, and … previous experience.”
      • Schacter D.L.
      • Slotnick S.D.
      The cognitive neuroscience of memory distortion.
      ,p149 As such, the general knowledge and availability of mental schemas from previous eating occasions intrude on the encoding of current consumption to produce false and fuzzy (ie, gist) memories.
      • Schooler J.W.
      The distinctions of false and fuzzy memories.
      • Cann D.R.
      • McRae K.
      • Katz A.N.
      False recall in the Deese-Roediger-McDermott paradigm: the roles of gist and associative strength.
      Memory and recall are subject to a myriad of unintentional “sins,”
      • Schacter D.
      The Seven Sins of Memory.
      including but not limited to distortions, misattribution, suggestibility, simple forgetting, falsehoods, and omissions.
      • Schacter D.L.
      • Slotnick S.D.
      The cognitive neuroscience of memory distortion.
      • Schacter D.L.
      • Norman K.A.
      • Koutstaal W.
      The cognitive neuroscience of constructive memory.
      • Johnson M.K.
      Memory and reality.
      Because selective and elaborative processes operate on the perceptions that are encoded and recalled, “memory does not [and cannot] operate like a video recording.”
      • Schacter D.L.
      • Loftus E.F.
      Memory and law: what can cognitive neuroscience contribute?.
      ,p119
      Recently, the process of reconsolidation (ie, the reconstruction and re-encoding of memories after recall) has been demonstrated in rodents, and the evidence in humans is supportive.
      • Besnard A.
      • Caboche J.
      • Laroche S.
      Reconsolidation of memory: a decade of debate.
      • Schwabe L.
      • Nader K.
      • Pruessner J.C.
      Reconsolidation of human memory: brain mechanisms and clinical relevance.
      Reconsolidation involves the same neural processes as the encoding of the original memory.
      • Johnson M.K.
      Memory and reality.
      Therefore, each time a memory is recalled, it is irretrievably changed such that the original memory no longer exists and a new memory of unquantifiable error replaces it.
      • Besnard A.
      • Caboche J.
      • Laroche S.
      Reconsolidation of memory: a decade of debate.
      • Schwabe L.
      • Nader K.
      • Pruessner J.C.
      Reconsolidation of human memory: brain mechanisms and clinical relevance.
      This fact has implications for the current state-of-the-art 24HR instrument the US Department of Agriculture (USDA) Automated Multiple-Pass Method.
      • Thompson F.E.
      • Subar A.F.
      Dietary assessment methodology.
      With each pass of the multipass procedure, the process of reconsolidation alters the original memory so that by the end of the data collection period, the result will be an amalgam of multiple new memories and reports with unquantifiable error. As such, neither the researchers nor the participants know the validity or reliability of the reported food and beverage consumption.

      False Reporting: An Inherent Design Feature of M-BMs

      False Reporting and FFQs

      False reports are the recollection of an event, or details of an event, that did not actually occur.
      • Straube B.
      An overview of the neuro-cognitive processes involved in the encoding, consolidation, and retrieval of true and false memories.
      False memories and recalls may be produced in multiple contexts (eg, during research,
      • Bernstein D.M.
      • Loftus E.F.
      The consequences of false memories for food preferences and choices.
      • Mojet J.
      • Köster E.P.
      Sensory memory and food texture.
      psychotherapy, and criminal investigatory interviews
      • Shaw J.
      • Porter S.
      Constructing rich false memories of committing crime.
      ). Although research has demonstrated that false memories of ingestive behavior and subsequent false reporting of foods occur in laboratory settings,
      • Bernstein D.M.
      • Loftus E.F.
      The consequences of false memories for food preferences and choices.
      • Bernstein D.M.
      • Laney C.
      • Morris E.K.
      • Loftus E.F.
      False beliefs about fattening foods can have healthy consequences.
      • Mojet J.
      • Köster E.P.
      Sensory memory and food texture.
      there is a larger literature base outside of nutrition. The Deese-Roediger and McDermott (DRM) paradigm is commonly used in research settings to elicit false reports.
      • Deese J.
      On the prediction of occurrence of particular verbal intrusions in immediate recall.
      • Roediger H.L.
      • McDermott K.B.
      Creating false memories: remembering words not presented in lists.
      In this protocol, a list of semantically related words (eg, breakfast, bacon, sausage, orange juice, and cereal) are presented or read to participants. After a delay (minutes to days), participants are asked to report the words they remember. The mere presentation of lists of semantically related words induces extremely high levels (ie, >75%) of the false reporting of related but nonpresented words (ie, critical lures
      • Schacter D.L.
      • Slotnick S.D.
      The cognitive neuroscience of memory distortion.
      • Gallo D.A.
      • Roberts M.J.
      • Seamon J.G.
      Remembering words not presented in lists: can we avoid creating false memories?.
      • Roediger H.L.
      • McDermott K.B.
      Creating false memories: remembering words not presented in lists.
      ; eg, the word egg in the previous example). The DRM paradigm is so effective at inducing false reports that memory distortions occur even in the small percentage of individuals with highly superior memories.
      • Patihis L.
      • Frenda S.J.
      • LePort A.K.R.
      • et al.
      False memories in highly superior autobiographical memory individuals.
      With the DRM paradigm, respondents are often more confident in their false reports than in the presented words.
      • Gallo D.A.
      False memories and fantastic beliefs: 15 years of the DRM illusion.
      Researchers familiar with FFQs will recognize that, by design, FFQs mimic the DRM protocol in that lists of semantically related words (ie, foods and beverages) are presented and respondents are expected to provide a response. Given that FFQs mimic the procedures designed to produce false recall, it is not surprising that FFQs with longer lists of semantically related words elicit more responses.
      • Molag M.L.
      • de Vries J.H.M.
      • Ocké M.C.
      • et al.
      Design characteristics of food frequency questionnaires in relation to their validity.
      Given the vast literature demonstrating misreporting with FFQs
      • Lissner L.
      • Troiano R.P.
      • Midthune D.
      • et al.
      OPEN about obesity: recovery biomarkers, dietary reporting errors and BMI.
      • Subar A.F.
      • Kipnis V.
      • Troiano R.P.
      • et al.
      Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: the OPEN study.
      • Heitmann B.L.
      • Lissner L.
      • Osler M.
      Do we eat less fat, or just report so?.
      • Neuhouser M.L.
      • Tinker L.
      • Shaw P.A.
      • et al.
      Use of recovery biomarkers to calibrate nutrient consumption self-reports in the Women's Health Initiative.
      and the parallel literature on the extremely high level of false reports using the DRM paradigm,
      • Gallo D.A.
      False memories and fantastic beliefs: 15 years of the DRM illusion.
      • Cann D.R.
      • McRae K.
      • Katz A.N.
      False recall in the Deese-Roediger-McDermott paradigm: the roles of gist and associative strength.
      • Deese J.
      On the prediction of occurrence of particular verbal intrusions in immediate recall.
      • Roediger H.L.
      • McDermott K.B.
      Creating false memories: remembering words not presented in lists.
      it is not a question of whether FFQs induce false reporting but to what extent. As stated previously, neither the researchers nor the participants know the validity or reliability of the reported food and beverage consumption, and neither can they quantify the error induced via false reporting. As we discuss in a later section, the inability of current nutrition epidemiologic research designs to independently falsify or confirm M-BM data renders the error due to false reports unquantifiable and, therefore, inadmissible as scientific data.

      False Reporting and the WWEIA-NHANES 24HR

      Recent research has examined the effects of creating “false memories for food preferences and choices.”
      • Bernstein D.M.
      • Loftus E.F.
      The consequences of false memories for food preferences and choices.
      • Bernstein D.M.
      • Laney C.
      • Morris E.K.
      • Loftus E.F.
      False beliefs about fattening foods can have healthy consequences.
      ,p134 We refer our readers to a review by Bernstein and Loftus.
      • Bernstein D.M.
      • Loftus E.F.
      The consequences of false memories for food preferences and choices.
      Their work has established that it is relatively simple to “implant false beliefs and memories regarding a variety of early childhood food-related experiences.”
      • Bernstein D.M.
      • Loftus E.F.
      The consequences of false memories for food preferences and choices.
      ,p138 We assert that false memories and reports are induced via the NHANES interview protocol itself, as has been demonstrated in other interviewing contexts.
      • Shaw J.
      • Porter S.
      Constructing rich false memories of committing crime.
      The factors that potentially induce false memories and reporting are well established. For example, the development of a rapport between an authority figure and respondents followed by the use of guided imagery, silence in responding, repetition, props, suggestive or repeated questioning, and encouragement to reminisce, imagine, or elaborate on past behaviors have all been shown to increase false recall.
      • Bernstein D.M.
      • Loftus E.F.
      The consequences of false memories for food preferences and choices.
      • Straube B.
      An overview of the neuro-cognitive processes involved in the encoding, consolidation, and retrieval of true and false memories.
      • Johnson M.K.
      Memory and reality.
      • Gallo D.A.
      False memories and fantastic beliefs: 15 years of the DRM illusion.
      • Cann D.R.
      • McRae K.
      • Katz A.N.
      False recall in the Deese-Roediger-McDermott paradigm: the roles of gist and associative strength.
      • Roediger H.L.
      • McDermott K.B.
      Creating false memories: remembering words not presented in lists.
      All of these factors are explicitly described in the training manual for the research personnel who conduct the NHANES 24HR.

      Centers for Disease Control and Prevention. NHANES MEC in-person dietary interviewers procedures manual. http://www.cdc.gov/nchs/data/nhanes/nhanes_09_10/DietaryInterviewers_Inperson.pdf. Published January 2009. Accessed August 9, 2014.

      The use of rapport, silence, imagery, props, repeated questioning, eye contact, and “expectant looks,”p4-12 to “motivate the respondent to answer more fully”,p4-4 are explicit and noteworthy in the training manual.

      Centers for Disease Control and Prevention. NHANES MEC in-person dietary interviewers procedures manual. http://www.cdc.gov/nchs/data/nhanes/nhanes_09_10/DietaryInterviewers_Inperson.pdf. Published January 2009. Accessed August 9, 2014.

      For example, the following directive is an exemplar of the potentially false memory–inducing protocol: “If you sit quietly—but expectantly—your respondent will usually think of something. Silence and waiting are frequently your best probes for a ‘don’t know' reply. Always try at least once to obtain a reply to a ‘don’t know' response, before accepting it as the final answer.”

      Centers for Disease Control and Prevention. NHANES MEC in-person dietary interviewers procedures manual. http://www.cdc.gov/nchs/data/nhanes/nhanes_09_10/DietaryInterviewers_Inperson.pdf. Published January 2009. Accessed August 9, 2014.

      ,p4-13 The use of rapport combined with repeated questioning, silence, eye contact, and expectant looks is especially coercive when applied by an authority figure in a research context. In addition, NHANES personnel are directed to ask respondents to “imagine,” and “think” about their food intake and to “encourage” and ensure that the respondents are “convinced of the importance of the survey.”

      Centers for Disease Control and Prevention. NHANES MEC in-person dietary interviewers procedures manual. http://www.cdc.gov/nchs/data/nhanes/nhanes_09_10/DietaryInterviewers_Inperson.pdf. Published January 2009. Accessed August 9, 2014.

      ,p4-3 Throughout the manual there are examples of guided imagery and suggestive questioning, such as directing participants to begin “thinking about where you were, who you were with, or what you were doing, like working, eating out, or watching television,”

      Centers for Disease Control and Prevention. NHANES MEC in-person dietary interviewers procedures manual. http://www.cdc.gov/nchs/data/nhanes/nhanes_09_10/DietaryInterviewers_Inperson.pdf. Published January 2009. Accessed August 9, 2014.

      ,p6-2 and directives such as, “Your own state of mind—your conviction that the interview is important—will strongly influence the respondent’s cooperation. Your belief that the information you obtain will be significant and useful will help motivate the respondent to answer fully ….”p4-4 Although the NHANES training manual states that “[t]his methodology is designed to maximize respondents’ opportunities for remembering and reporting foods they have eaten,”p6-2 the scientific literature on false memories and recall strongly supports the contention that the NHANES M-BM generates significant false reporting. Given that imagination and coercive techniques (eg, the use of silence

      Centers for Disease Control and Prevention. NHANES MEC in-person dietary interviewers procedures manual. http://www.cdc.gov/nchs/data/nhanes/nhanes_09_10/DietaryInterviewers_Inperson.pdf. Published January 2009. Accessed August 9, 2014.

      ) are known to increase the probability of illusory (ie, false) recollections,
      • Goff L.
      • Roediger H.
      Imagination inflation for action events: repeated imaginings lead to illusory recollections.
      • Shaw J.
      • Porter S.
      Constructing rich false memories of committing crime.
      it may be that most 24HR data are false reports. If true, the NHANES 24HR is a mere exercise in number generation, and, therefore, by design, it does not provide proxy estimates of energy or nutrient consumption. This premise provides an empirically supported explanation why most M-BM data are implausible and have trivial relationships with reality (ie, actual EI and nutrient intake.) Nevertheless, without objective corroboration it is impossible to quantify what percentage of the recalled foods and beverages are completely false, grossly inaccurate, or somewhat congruent with actual consumption. Regardless, it is clear that people consistently “remember [and report] events that never happened.”
      • Roediger H.L.
      • McDermott K.B.
      Creating false memories: remembering words not presented in lists.
      ,p803

      The Inadmissibility of M-BM Data

      Criteria for Scientific Research: Observable, Measurable, and Falsifiable

      Although the terms science and research are used interchangeably, they are not synonymous. Science is more than mere data collection; it is an attempt to discover order, a potentially self-correcting, explanatory, and predictive process that demonstrates lawful relations (eg, diets high in vitamin C prevent scurvy). In contrast, research is simply the process of collecting information, and many forms of research do not meet the rigor necessary for the results to be scientific. There is a long history of efforts to formally demarcate scientific from nonscientific and pseudoscientific data, the most famous of which may be Popper’s falsifiability criterion.
      • Popper K.
      Conjectures and Refutations: The Growth of Scientific Knowledge.
      • Popper K.R.
      The Logic of Scientific Discovery.
      • Popper K.R.
      Objective Knowledge: An Evolutionary Approach.
      For example, in US jurisprudence, the Daubert standard
      • Berger M.A.
      What has a decade of Daubert wrought?.
      • Krimsky S.
      The weight of scientific evidence in policy and law.
      provides the rules of evidence for the admissibility of expert testimony. The criterion of falsifiability is central to expert scientific testimony and was used by Judge William Overton in ruling in McLean v. Arkansas Board of Education. This case determined that creation science was not a science because it was not falsifiable and, therefore, could not be taught as science in Arkansas public schools.

      McLean v. Arkansas Board of Education, 529 F. Sup. 1255, 1258-1264; E.D. Ark. 1982.

      As we detail in later sections, we assert that M-BM data are akin to creation science in that they fail to meet the basic requirements of scientific research.
      Although philosophers continue to debate demarcation criteria, practicing scientists must set forth principles from which to judge the admissibility of data in scientific research. We extend Popper’s criterion and proffer the following widely accepted principles of scientific inquiry. First, for results to be scientific, the study’s protocols must produce outcomes that are subject to replication. To accomplish this goal, the data must be (1) independently observable (ie, accessible by others), (2) measureable, (3) falsifiable, (4) valid, and (5) reliable. These nonmetaphysical criteria were first suggested by Roger Bacon in the 13th century and later were elaborated on by the “father of empiricism,” Sir Francis Bacon, in the late 16th century.
      • Bacon F.
      Of the Proficience and Advancement of Learning, Divine and Human.
      They were again reiterated by Sir Isaac Newton in the 17th century
      • Newton I.
      The Principia: Mathematical Principles of Natural Philosophy.
      and have been subsequently clarified and defined.
      • Platt J.R.
      Strong inference: certain systematic methods of scientific thinking may produce much more rapid progress than others.
      • Sober E.
      Philosophy of Biology.
      • Popper K.
      Conjectures and Refutations: The Growth of Scientific Knowledge.
      • Popper K.R.
      The Logic of Scientific Discovery.
      • Popper K.R.
      Objective Knowledge: An Evolutionary Approach.
      • Michell J.
      Measurement: a beginner's guide.
      The skepticism and empirical rigor inherent in these criteria are of such importance to science that The Royal Society of London, the oldest scientific society in the modern world, succinctly summarized them in its motto, Nullius in Verba. This phrase, derived from Horace’s Epistles,

      Horace. (Quintus Horatius Flaccus), Epistulae, Book I. http://www.thelatinlibrary.com/horace/epist1.shtml. Accessed April 2015.

      is translated as “on the word of no one” or “take no one’s word for it” and suggests that scientific knowledge should be based not on authority, rhetoric, or mere words but on objective evidence.
      The first 3 criteria (ie, independently observable, measureable, and falsifiable) define the phenomena that are in the domain of science (ie, able to be examined via the scientific method), and the final 2 criteria (ie, validity and reliability) refer to the concordance between a measurement and its referent as well as the error associated with the measurement protocols used to collect the data. Together, the 5 basic tenets distinguish scientific research from mere data collection and pseudoscience. For example, if someone is eating an apple, his or her behavior can be independently observed, measured, and verified or refuted. Yet, if he or she reports eating an apple at some point in the past (eg, as with an FFQ or 24HR), neither the past behavior nor the neural correlates of the memory of that behavior are independently observable or quantifiable, and without additional information, his or her statement cannot be falsified or confirmed. It is a rather obvious fact that the respondent is the only person who has access to the raw data of M-BMs (ie, his or her memories of consumption). As such, researchers cannot examine the validity of the memory and base M-BM research results on their faith in the verbal report (ie, the belief that the participant is telling the truth). Nevertheless, faith and belief are basic tenets of religion, not science. The unwavering credulity of nutrition epidemiologists with respect to verbal reports is literally in direct opposition to Nullius in Verba (ie, take no one’s word for it) and skeptical, rigorous science. The confluence of these simple facts and the well-documented failure of self-reported EI to accurately correspond to reality,
      • Archer E.
      • Hand G.A.
      • Blair S.N.
      Validity of U.S. nutritional surveillance: National Health and Nutrition Examination Survey caloric energy intake data, 1971-2010.
      • Lissner L.
      • Troiano R.P.
      • Midthune D.
      • et al.
      OPEN about obesity: recovery biomarkers, dietary reporting errors and BMI.
      demonstrate that the memory and subsequent recall of ingestive behavior are not within the realm of the scientific investigation of nutrition and obesity. As the philosopher Karl Popper stated, “all the statements of empirical science must be capable of being finally decided, with respect to their truth and falsity,”
      • Popper K.R.
      The Logic of Scientific Discovery.
      ,p17 and it is wholly impossible to verify or refute something that cannot be directly or indirectly independently observed and measured (eg, memories).

      The Pseudoscience of Nutrition Epidemiology

      The term pseudoscience describes data or results that are presented as scientific but lack plausibility because they cannot be reliably, accurately, and independently observed, quantified, and confirmed or refuted.
      • Platt J.R.
      Strong inference: certain systematic methods of scientific thinking may produce much more rapid progress than others.
      • Sober E.
      Philosophy of Biology.
      • Popper K.
      Conjectures and Refutations: The Growth of Scientific Knowledge.
      • Popper K.R.
      The Logic of Scientific Discovery.
      • Popper K.R.
      Objective Knowledge: An Evolutionary Approach.
      When M-BMs are examined from the perspective of the basic tenets of science, the reason for the explanatory and predictive failure of epidemiologic nutrition research becomes obvious. First and foremost, scientific conclusions cannot result from nonempirical (ie, unobserved) or subjective (ie, private, not publically accessible) data that are not subject to independent observation, quantification, and falsification. When a person provides a dietary report, the data collected are not actual food or beverage consumption but rather an error-prone and highly edited anecdote regarding memories of food and beverage consumption. As such, M-BMs do not meet the basic requirements of the scientific method and, by definition, are pseudoscientific when presented as actual estimates of energy or nutrient consumption. Two famous physicists of the 20th century, Wolfgang Pauli and Arthur Schuster, summed up the problem with pseudoscientific data eloquently when they stated, respectively, that a pseudoscientific conclusion “is not only not right, it is not even wrong …”
      • Peierls R.E.
      Wolfgang Ernst Pauli: 1900-1958.
      ,p186 and “[w]e all prefer being right to being wrong, but it is better to be wrong than to be neither right nor wrong.”
      • Schuster A.
      The Progress of Physics.
      ,p117
      It is difficult to determine the empirical consequences of M-BMs because the primary data (ie, memories: private information to which the respondents have privileged access) do not meet the basic tenets of scientific methods (eg, independent observation of data, falsifiability, and accuracy). If neither the researchers nor the participants are able to quantify what percentage of the recalled foods and beverages are completely false reports, grossly inaccurate, or reports that are somewhat congruent with actual consumption, it is impossible to know the validity and the error associated with each report. As Dhurandhar et al

      Dhurandhar NV, Schoeller D, Brown AW, et al. Energy balance measurement: when something is not better than nothing [published online November 13, 2014]. Int J Obes (Lond).

      recently suggested, the use of M-BM–based data is a context in which “… something is not better than nothing.”p1 Given the forgoing, M-BM–derived data are inadmissible and constitute a substantial ongoing threat to nutrition and obesity research and national dietary guidelines because the greatest obstacle to scientific progress is not ignorance but the illusion of knowledge created by pseudoscientific data that are neither right nor wrong.
      Nevertheless, performing rigorous science is a skill that can be learned, but only if mentors understand and practice rigorous science. Given the ubiquitous use of M-BMs over many decades, it seems that nutritional epidemiologists have eschewed the inherent rigor and skepticism of Nullius in Verba (ie, take no one’s word for it) and literally replaced it with Totius in Verba (ie, take everyone’s word for it). As a result, skeptical rigorous science is not practiced or taught in nutrition and obesity epidemiologic research.

      Archer E. A wolf in sheep’s clothing. TheScientist website. http://www.the-scientist.com/?articles.view/articleNo/37918/title/Opinion–A-Wolf-in-Sheep-s-Clothing. Published October 22, 2013. Accessed February 21, 2015.

      National Nutrition Surveillance: M-BM Data and USDA Food Availability Economic Data

      If the 2 major components of US national nutritional surveillance are valid (ie, NHANES M-BM data and USDA Food Availability economic data), estimates from these surveillance tools should track together and independently provide population-level approximations of trends in food consumption or use. Nevertheless, history demonstrates that this is not the case. Trends in estimates of macronutrient consumption from population-level epidemiologic surveys (ie, M-BMs) exhibited statistically significant trends that were in opposition to those of USDA economic data for fat, carbohydrates, protein, and energy (ie, kilocalories per day) from the 1960s to the late 1980s.
      • Crane N.T.
      • Lewis C.J.
      • Yetley E.A.
      Do time trends in food supply levels of macronutrients reflect survey estimates of macronutrient intake?.
      It should be apparent that US residents could not be simultaneously consuming more and less fat, protein, carbohydrates, and energy over time. The contradictory patterns and striking lack of correspondence between the 2 primary US nutrition surveillance tools suggest that 1 or more likely both protocols are invalid. As with the severe misreporting demonstrated across the globe,
      • Goldberg G.R.
      • Black A.E.
      • Jebb S.A.
      • et al.
      Critical evaluation of energy intake data using fundamental principles of energy physiology, 1: derivation of cut-off limits to identify under-recording.
      • Orcholski L.
      • Luke A.
      • Plange-Rhule J.
      • et al.
      Under-reporting of dietary energy intake in five populations of the African diaspora.
      these contradictory patterns are not limited to the United States; many countries exhibit considerable disparity between national surveillance via M-BMs and economic/food supply data.
      • Serra-Majem L.
      • MacLean D.
      • Ribas L.
      • et al.
      Comparative analysis of nutrition data from national, household, and individual levels: results from a WHO-CINDI collaborative project in Canada, Finland, Poland, and Spain.
      • Serra-Majem L.
      Food availability and consumption at national, household and individual levels: implications for food-based dietary guidelines development.
      • Rodriguez-Artalejo F.
      • Banegas J.R.
      • Graciani A.
      • Hernandez-Vecino R.
      • del Rey-Calero J.
      Food supply versus household survey data: nutrient consumption trends for Spain, 1958-1988.
      • Matsumura Y.
      Nutrition trends in Japan.
      This fact is further evidence that M-BMs are fatally flawed and diet-health inferences from M-BM–derived data are meaningless.

      PA and CRF: Essential Elements in Nutrition, Obesity, and Health Research

      The lack of explanatory and predictive power of epidemiologic nutrition research may also be explained by the limited acknowledgement of nonnutritional determinants of health and disease, such as nongenetic evolution,
      • Archer E.
      The childhood obesity epidemic as a result of nongenetic evolution: the maternal resources hypothesis.
      • Archer E.
      The mother of all problems.
      • Archer E.
      In reply—Maternal, paternal, and societal efforts are needed to “cure” childhood obesity.
      PA,
      • Blair S.N.
      Physical inactivity: the biggest public health problem of the 21st century.
      • Archer E.
      • Blair S.N.
      Physical activity and the prevention of cardiovascular disease: from evolution to epidemiology.
      CRF,
      • Blair S.N.
      • Kohl III, H.W.
      • Paffenbarger Jr., R.S.
      • Clark D.G.
      • Cooper K.H.
      • Gibbons L.W.
      Physical fitness and all-cause mortality: a prospective study of healthy men and women.
      and other components of nutrient partitioning and energy balance.
      • Krogh-Madsen R.
      • Thyfault J.P.
      • Broholm C.
      • et al.
      A 2-wk reduction of ambulatory activity attenuates peripheral insulin sensitivity.
      • Thyfault J.P.
      • Krogh-Madsen R.
      Metabolic disruptions induced by reduced ambulatory activity in free-living humans.
      • Bergouignan A.
      • Kealey E.H.
      • Schmidt S.L.
      • Jackman M.R.
      • Bessesen D.H.
      Twenty-four hour total and dietary fat oxidation in lean, obese and reduced-obese adults with and without a bout of exercise.
      • Bergouignan A.
      • Momken I.
      • Schoeller D.A.
      • et al.
      Regulation of energy balance during long-term physical inactivity induced by bed rest with and without exercise training.
      • Bergouignan A.
      • Rudwill F.
      • Simon C.
      • Blanc S.
      Physical inactivity as the culprit of metabolic inflexibility: evidence from bed-rest studies.
      • Bergouignan A.
      • Schoeller D.A.
      • Normand S.
      • et al.
      Effect of physical inactivity on the oxidation of saturated and monounsaturated dietary fatty acids: results of a randomized trial.
      • Bergouignan A.
      • Trudel G.
      • Simon C.
      • et al.
      Physical inactivity differentially alters dietary oleate and palmitate trafficking.
      For example, more than 50 years ago the Food and Agriculture Organization of the United Nations and the World Health Organization determined that human food energy requirements should be estimated using TEE and that PA and basal energy expenditure were the primary determinants.
      Food and Agriculture Organization
      Calorie Requirements: Report on the Second Committee on Calorie Requirements.
      FAO/WHO/UNU
      Energy and Protein Requirements.
      Yet, most nutrition research does not measure any form of energy expenditure or objectively quantify PA. Currently, there is only 1 manuscript of which we are aware that uses the NHANES objectively measured PA data to directly assess nutrition-related outcomes
      • Archer E.
      • Hand G.A.
      • Hébert J.R.
      • et al.
      Validation of a novel protocol for calculating estimated energy requirements and average daily physical activity ratio for the U.S. population: 2005-2006.
      and no nutrition-related publications that include the NHANES treadmill CRF data in analyses. The lack of publications may be due to the fact that only 2 waves in the more than 40-year history of the NHANES include objective measures of PA, and despite the widespread acknowledgment of the necessity of daily PA for health and well-being, it is routinely discounted by governmental public health funding agencies. For example, PA, CRF, and exercise are not even listed on the National Institutes of Health’s spreadsheet of categorical spending of nearly 250 classifications through 2016.
      This is unfortunate given that 80% of Americans are not at risk for most nutritional deficiencies,
      • Pfeiffer C.M.
      • Sternberg M.R.
      • Schleicher R.L.
      • Haynes B.M.
      • Rybak M.E.
      • Pirkle J.L.
      The CDC's Second National Report on Biochemical Indicators of Diet and Nutrition in the U.S. Population is a valuable tool for researchers and policy makers.
      but 95% of Americans are at risk for PA deficiency (ie, inactivity or high sedentary behavior) and do not meet the federal recommendations of 30 minutes per day of moderate to vigorous PA.
      • Troiano R.P.
      • Berrigan D.
      • Dodd K.W.
      • Masse L.C.
      • Tilert T.
      • McDowell M.
      Physical activity in the United States measured by accelerometer.
      Given that PA and CRF are major determinants of health
      • Archer E.
      • Blair S.N.
      Physical activity and the prevention of cardiovascular disease: from evolution to epidemiology.
      • Blair S.N.
      • Kohl III, H.W.
      • Paffenbarger Jr., R.S.
      • Clark D.G.
      • Cooper K.H.
      • Gibbons L.W.
      Physical fitness and all-cause mortality: a prospective study of healthy men and women.
      • Archer E.
      • Hand G.A.
      • Hébert J.R.
      • et al.
      Validation of a novel protocol for calculating estimated energy requirements and average daily physical activity ratio for the U.S. population: 2005-2006.
      • Lee I.M.
      • Shiroma E.J.
      • Lobelo F.
      • Puska P.
      • Blair S.N.
      • Katzmarzyk P.T.
      Effect of physical inactivity on major non-communicable diseases worldwide: an analysis of burden of disease and life expectancy.
      • LaMonte M.J.
      • Blair S.N.
      • Church T.S.
      Physical activity and diabetes prevention.
      • Sui X.
      • Lee D.C.
      • Matthews C.E.
      • et al.
      Influence of cardiorespiratory fitness on lung cancer mortality.
      and that PA is the only major modifiable determinant of TEE and nutrient-energy partitioning (ie, the metabolic fate of the foods we consume),
      • Archer E.
      The childhood obesity epidemic as a result of nongenetic evolution: the maternal resources hypothesis.
      • Krogh-Madsen R.
      • Thyfault J.P.
      • Broholm C.
      • et al.
      A 2-wk reduction of ambulatory activity attenuates peripheral insulin sensitivity.
      • Thyfault J.P.
      • Krogh-Madsen R.
      Metabolic disruptions induced by reduced ambulatory activity in free-living humans.
      • Bergouignan A.
      • Kealey E.H.
      • Schmidt S.L.
      • Jackman M.R.
      • Bessesen D.H.
      Twenty-four hour total and dietary fat oxidation in lean, obese and reduced-obese adults with and without a bout of exercise.
      • Bergouignan A.
      • Momken I.
      • Schoeller D.A.
      • et al.
      Regulation of energy balance during long-term physical inactivity induced by bed rest with and without exercise training.
      • Bergouignan A.
      • Rudwill F.
      • Simon C.
      • Blanc S.
      Physical inactivity as the culprit of metabolic inflexibility: evidence from bed-rest studies.
      • Bergouignan A.
      • Schoeller D.A.
      • Normand S.
      • et al.
      Effect of physical inactivity on the oxidation of saturated and monounsaturated dietary fatty acids: results of a randomized trial.
      • Bergouignan A.
      • Trudel G.
      • Simon C.
      • et al.
      Physical inactivity differentially alters dietary oleate and palmitate trafficking.
      • Archer E.
      • Hand G.A.
      • Hébert J.R.
      • et al.
      Validation of a novel protocol for calculating estimated energy requirements and average daily physical activity ratio for the U.S. population: 2005-2006.
      it is clear that PA and CRF must be objectively measured and controlled for in analyses if the health effects of any dietary intervention are to be examined accurately. Yet, because PA questionnaires are susceptible to many of the same systematic biases

      Dhurandhar NV, Schoeller D, Brown AW, et al. Energy balance measurement: when something is not better than nothing [published online November 13, 2014]. Int J Obes (Lond).

      • Prince S.A.
      • Adamo K.B.
      • Hamel M.E.
      • Hardt J.
      • Connor Gorber S.
      • Tremblay M.
      A comparison of direct versus self-report measures for assessing physical activity in adults: a systematic review.
      • Shephard R.J.
      Limits to the measurement of habitual physical activity by questionnaires.
      and inadmissibility issues as M-BMs, the failure to objectively measure PA and control for it in analyses renders health inferences from previous nutrition epidemiologic studies moot. Fortunately, for the science of health and disease, there are objective tools for the measurement of PA (eg, pedometers and accelerometry-based PA monitors),
      • Trost S.G.
      • McIver K.L.
      • Pate R.R.
      Conducting accelerometer-based activity assessments in field-based research.
      and despite limitations,
      • Pedisic Z.
      • Bauman A.
      Accelerometer-based measures in physical activity surveillance: current practices and issues.
      these should be used in place of surveys and questionnaires to quantify PA in future examinations of health and disease.

      Summary and Future Directions

      A wise man proportions his belief to the evidence.David Hume

      Hume D. An enquiry concerning human understanding, and concerning the principles of morals. In: Ingram J, ed. The Project Gutenberg. http://www.gutenberg.org/files/9662/9662-h/9662-h.htm. Accessed February 22, 2015.

      ,p87
      This critical review provides empirical and analytic evidence to support the position that (1) M-BM estimates of EI and nutrient intake have trivial relationships with actual EI and nutrient intake; (2) the assumption that human memory and recall provide literal, accurate, or precise reproductions of past ingestive behavior is indisputably false; (3) M-BMs require participants to submit to protocols that mimic procedures known to induce false recall; (4) the subjective (ie, private, not publically accessible) mental phenomena (ie, memories) from which M-BM data are derived are not subject to independent observation, quantification, or falsification; therefore, these data are pseudoscientific and inadmissible in scientific research; and (5) the failure to objectively measure and control for PA and CRF in analyses renders inferences regarding most diet-health relationships moot.
      Given the overwhelming evidence in support of our hypotheses, we conclude that M-BM data cannot be used to inform national dietary guidelines and that continued funding of M-BMs constitutes an unscientific and major misuse of research resources. In addition, given that there are objective data on the nutrition-related health status of Americans,
      • Pfeiffer C.M.
      • Sternberg M.R.
      • Schleicher R.L.
      • Haynes B.M.
      • Rybak M.E.
      • Pirkle J.L.
      The CDC's Second National Report on Biochemical Indicators of Diet and Nutrition in the U.S. Population is a valuable tool for researchers and policy makers.
      we find the DGAC’s reliance on M-BMs to be without scientific support or merit. We think that skepticism and rigor are essential requirements in scientific investigations, and we fault the overly credulous nature of nutrition epidemiology for the obvious and well-demonstrated failures of the scientific community to properly inform previous federal dietary guidelines (eg, cholesterol consumption).
      Dietary Guidelines Advisory Committee
      Scientific Report of the 2015 Dietary Guidelines Advisory Committee.
      Dietary Guidelines Advisory Committee
      Report of the Dietary Guidelines Advisory Committee on the Dietary Guidelines for Americans, 2010.
      We think that our nation’s dietary guidelines should not be based on the pseudoscientific and highly edited anecdotes of M-BMs, and although others may disagree, we ask that they do as we have done and provide empirical evidence rather than rhetoric to support their positions. Without valid evidence, the dogmatic defense of illusory knowledge and the status quo in nutrition and obesity research (eg, see previous commentaries and guidelines
      Dietary Guidelines Advisory Committee
      Scientific Report of the 2015 Dietary Guidelines Advisory Committee.
      • Hébert J.
      • Hurley T.G.
      • Steck S.E.
      • et al.
      Considering the value of dietary assessment data in informing nutrition-related health policy.
      • Satija A.
      • Yu E.
      • Willett W.C.
      • Hu F.B.
      Understanding nutritional epidemiology and its role in policy.
      ) is an impediment to scientific progress and empirically supported public nutrition and obesity policy.
      We began this critical review with evidence that our nation’s food supply and the nutritional status of Americans have improved to a level unparalleled in human history.
      • Pfeiffer C.M.
      • Sternberg M.R.
      • Schleicher R.L.
      • Haynes B.M.
      • Rybak M.E.
      • Pirkle J.L.
      The CDC's Second National Report on Biochemical Indicators of Diet and Nutrition in the U.S. Population is a valuable tool for researchers and policy makers.
      • Gerrior S.
      • Bente L.
      • Hiza H.
      Nutrient Content of the U.S. Food Supply, 1909-2000.
      CDC
      National Report on Biochemical Indicators of Diet and Nutrition in the US Population 1999–2002.
      Given this reality and recent work on the intergenerational transmission of obesity and type 2 diabetes mellitus,
      • Archer E.
      The childhood obesity epidemic as a result of nongenetic evolution: the maternal resources hypothesis.
      • Archer E.
      The mother of all problems.
      • Archer E.
      In reply—Maternal, paternal, and societal efforts are needed to “cure” childhood obesity.
      we posit that the American diet is no longer a significant risk factor for disease for most individuals. This hypothesis is supported by multiple lines of evidence, such as a 40% decline in the age-adjusted mortality rate from 1969 to 2010,
      • Hoyert D.L.
      75 Years of Mortality in the United States, 1935–2010.
      a progressive decades long reduction in age-adjusted cardiovascular disease incidence and mortality,
      • Members W.G.
      • Roger V.L.
      • Go A.S.
      • et al.
      Heart disease and stroke statistics—2012 update: a report from the American Heart Association.
      • Cooper R.
      • Cutler J.
      • Desvigne-Nickens P.
      • et al.
      Trends and disparities in coronary heart disease, stroke, and other cardiovascular diseases in the United States: findings of the National Conference on Cardiovascular Disease Prevention.
      and a 1.5% per annum reduction in age-adjusted mortality rates from all major cancers as well as significant reductions in lung cancer incidence in men and women between 2001 and 2010.
      • Edwards B.K.
      • Noone A.-M.
      • Mariotto A.B.
      • et al.
      Annual report to the nation on the status of cancer, 1975-2010, featuring prevalence of comorbidity and impact on survival among persons with lung, colorectal, breast, or prostate cancer.
      Given the forgoing and the evidence presented herein demonstrating the pseudoscientific nature of M-BMs, we assert that research efforts and funding of M-BMs and diet-health research are misdirected and argue that those resources would be better targeted to the most prevalent disease of deficiency of the 21st century: inactivity (ie, a lack of PA and exercise and high levels of sedentary behavior).
      • Blair S.N.
      Physical inactivity: the biggest public health problem of the 21st century.
      • Troiano R.P.
      • Berrigan D.
      • Dodd K.W.
      • Masse L.C.
      • Tilert T.
      • McDowell M.
      Physical activity in the United States measured by accelerometer.

      Conclusion

      In this critical review, we argued that the essence of science is the ability to discern fact from fiction, and we presented evidence from multiple fields to support the position that the data generated by nutrition epidemiologic surveys and questionnaires are not falsifiable. As such, these data are pseudoscientific and inadmissible in scientific research. Therefore, these protocols and the resultant data should not be used to inform national dietary guidelines or public health policy, and the continued funding of these methods constitutes an unscientific and major misuse of research resources.

      Acknowledgments

      The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

      Supplemental Online Material

      References

        • Samuelson P.
        The Keynes centenary.
        The Economist. 1983; 287: 19-21
        • Pfeiffer C.M.
        • Sternberg M.R.
        • Schleicher R.L.
        • Haynes B.M.
        • Rybak M.E.
        • Pirkle J.L.
        The CDC's Second National Report on Biochemical Indicators of Diet and Nutrition in the U.S. Population is a valuable tool for researchers and policy makers.
        J Nutr. 2013; 143: 938S-947S
        • Gerrior S.
        • Bente L.
        • Hiza H.
        Nutrient Content of the U.S. Food Supply, 1909-2000.
        Center for Nutrition Policy and Promotion, US Department of Agriculture, Washington, DC2004 (Home Economics Research Report No. 56)
        • Rajakumar K.
        Pellagra in the United States: a historical perspective.
        South Med J. 2000; 93: 272-277
        • CDC
        National Report on Biochemical Indicators of Diet and Nutrition in the US Population 1999–2002.
        National Center for Environmental Health, Atlanta, GA2008
        • Archer E.
        The childhood obesity epidemic as a result of nongenetic evolution: the maternal resources hypothesis.
        Mayo Clin Proc. 2015; 90: 77-92
        • Archer E.
        The mother of all problems.
        New Sci. 2015; 225: 32-33
        • Archer E.
        In reply—Maternal, paternal, and societal efforts are needed to “cure” childhood obesity.
        Mayo Clin Proc. 2015; 90: 555-557
      1. Estimates of Funding for Various Research, Condition, and Disease Categories. US Department of Health and Human Services, Washington, DC2013
      2. American Association for the Advancement of Science. Historical trends in federal R&D. http://www.aaas.org/page/historical-trends-federal-rd. Accessed February 21, 2015.

        • Ioannidis J.P.A.
        Implausible results in human nutrition research.
        BMJ. 2013; 347 ([editorial]): f6698
        • Taubes G.
        Epidemiology faces its limits.
        Science. 1995; 269: 164-169
        • Schoeller D.A.
        • Thomas D.
        • Archer E.
        • et al.
        Self-report-based estimates of energy intake offer an inadequate basis for scientific conclusions.
        Am J Clin Nutr. 2013; 97: 1413-1415
        • Stampfer M.J.
        • Hennekens C.H.
        • Manson J.E.
        • Colditz G.A.
        • Rosner B.
        • Willett W.C.
        Vitamin E consumption and the risk of coronary disease in women.
        N Engl J Med. 1993; 328: 1444-1449
        • Rimm E.B.
        • Stampfer M.J.
        • Ascherio A.
        • Giovannucci E.
        • Colditz G.A.
        • Willett W.C.
        Vitamin E consumption and the risk of coronary heart disease in men.
        N Engl J Med. 1993; 328: 1450-1456
        • Tatsioni A.
        • Bonitsis N.G.
        • Ioannidis J.P.
        Persistence of contradicted claims in the literature.
        JAMA. 2007; 298: 2517-2526
        • Young S.S.
        • Karr A.
        Deming, data and observational studies.
        Significance. 2011; 8: 116-120
        • Moorthy D.
        • Chung M.
        • Lee J.
        • Yu W.W.
        • Lau J.
        • Trikalinos T.A.
        Concordance Between the Findings of Epidemiological Studies and Randomized Trials in Nutrition: An Empirical Evaluation and Citation Analysis.
        Agency for Healthcare Research and Quality, Rockville, MD2013 (Nutritional Research Series; vol 6)
        • Prentice R.L.
        Clinical trials and observational studies to assess the chronic disease benefits and risks of multivitamin-multimineral supplements.
        Am J Clin Nutr. 2007; 85: 308S-313S
        • Druesne-Pecollo N.
        • Latino-Martel P.
        • Norat T.
        • et al.
        Beta-carotene supplementation and cancer risk: a systematic review and metaanalysis of randomized controlled trials.
        Int J Cancer. 2010; 127: 172-184
        • Schatzkin A.
        • Mouw T.
        • Park Y.
        • et al.
        Dietary fiber and whole-grain consumption in relation to colorectal cancer in the NIH-AARP Diet and Health Study.
        Am J Clin Nutr. 2007; 85: 1353-1360
        • Schoenfeld J.D.
        • Ioannidis J.P.
        Is everything we eat associated with cancer? a systematic cookbook review.
        Am J Clin Nutr. 2013; 97: 127-134