Mayo Clinic Proceedings Home

Participation Bias in a Survey of Community Patients With Heart Failure



      To identify differences between participants and nonparticipants in a survey of physical and psychosocial aspects of health among a population-based sample of patients with heart failure (HF).

      Patients and Methods

      Residents from 11 Minnesota counties with a first-ever code for HF (International Classification of Diseases, Ninth Revision 428 and Tenth Revision I50) between January 1, 2013, and December 31, 2016, were identified. Participants completed a questionnaire by mail or telephone. Characteristics and outcomes were extracted from medical records and compared between participants and nonparticipants. Response rate was calculated using guidelines of the American Association for Public Opinion Research. The association between nonparticipation and outcomes was examined using Cox proportional hazards regression for death and Andersen-Gill modeling for hospitalizations.


      Among 7911 patients, 3438 responded to the survey (American Association for Public Opinion Research response rate calculated using formula 2 = 43%). Clinical and demographic differences between participants and nonparticipants were noted, particularly for education, marital status, and neuropsychiatric conditions. After a mean ± SD of 1.5±1.0 years after survey administration, 1575 deaths and 5857 hospitalizations occurred. Nonparticipation was associated with a 2-fold increased risk for death (hazard ratio, 2.29; 95% CI, 2.05-2.56) and 11% increased risk for hospitalization (hazard ratio, 1.11; 95% CI, 1.02-1.22) after adjusting for age, sex, time from HF diagnosis to index date, marital status, coronary disease, arrhythmia, hyperlipidemia, diabetes, cancer, chronic kidney disease, arthritis, osteoporosis, depression, and anxiety.


      In a large survey of patients with HF, participation was associated with notable differences in clinical and demographic characteristics and outcomes. Examining the impact of participation is critical to draw inference from studies of patient-reported measures.

      Abbreviations and Acronyms:

      HF (heart failure), HR (hazard ratio), PROM (patient-reported outcome measures), REP (Rochester Epidemiology Project), RR (response rate)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Mayo Clinic Proceedings
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Gordis L.
        5th ed. Elsevier Saunders, Philadelphia, PA2014
        • Groves R.M.
        • Peytcheva E.
        The impact of nonresponse rates on nonresponse bias: a meta-analysis.
        Public Opinion Quarterly. 2008; 72: 167-189
        • Halbesleben J.R.
        • Whitman M.V.
        Evaluating survey quality in health services research: a decision framework for assessing nonresponse bias.
        Health Serv Res. 2013; 48: 913-930
        • Beebe T.J.
        • McAlpine D.D.
        • Ziegenfuss J.Y.
        • Jenkins S.
        • Haas L.
        • Davern M.E.
        Deployment of a mixed-mode data collection strategy does not reduce nonresponse bias in a general population health survey.
        Health Serv Res. 2012; 47: 1739-1754
        • Czajka J.L.
        • Beyler A.
        Background paper. Declining response rates in federal surveys: trends and implications. Mathematica. 2016.
        • Olson S.H.
        • Voigt L.F.
        • Begg C.B.
        • Weiss N.S.
        Reporting participation in case-control studies.
        Epidemiology. 2002; 13: 123-126
        • Galea S.
        • Tracy M.
        Participation rates in epidemiologic studies.
        Ann Epidemiol. 2007; 17: 643-653
        • Pocock S.J.
        • Collier T.J.
        • Dandreo K.J.
        • et al.
        Issues in the reporting of epidemiological studies: a survey of recent practice.
        BMJ. 2004; 329: 883
        • von Elm E.
        • Altman D.G.
        • Egger M.
        • Pocock S.J.
        • Gotzsche P.C.
        • Vandenbroucke J.P.
        • STROBE Initiative
        The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies.
        J Clin Epidemiol. 2008; 61: 344-349
        • The American Association for Public Opinion Research
        Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 9th ed.
        • Krueger B.S.
        • West B.T.
        Assessing the potential of paradata and other auxiliary data for nonresponse adjustments.
        Public Opinion Quarterly. 2014; 78: 795-831
        • Sandler D.P.
        On revealing what we'd rather hide: the problem of describing study participation.
        Epidemiology. 2002; 13: 117
        • Davern M.
        Nonresponse rates are a problematic indicator of nonresponse bias in survey research.
        Health Serv Res. 2013; 48: 905-912
        • Rocca W.A.
        • Grossardt B.R.
        • Brue S.M.
        • et al.
        Data resource profile: expansion of the Rochester Epidemiology Project medical records-linkage system (E-REP).
        Int J Epidemiol. 2018; 47 (368-368j)
        • Rocca W.A.
        • Yawn B.P.
        • St Sauver J.L.
        • Grossardt B.R.
        • Melton 3rd, L.J.
        History of the Rochester Epidemiology Project: half a century of medical records linkage in a US population.
        Mayo Clin Proc. 2012; 87: 1202-1213
        • Hall S.A.
        • Kaufman J.S.
        • Ricketts T.C.
        Defining urban and rural areas in U.S. epidemiologic studies.
        J Urban Health. 2006; 83: 162-175
      1. 2010 Rural-Urban Commuting Area (RUCA) Codes. United States Department of Agriculture Economic Research Service.
        • US Department of Health and Human Services
        Multiple Chronic Conditions: A strategic Framework. Optimum Health and Quality of Life for Individuals with Multiple Chronic Conditions. December 2010.
        • Groves R.
        • Fowler F.J.
        • Couper M.
        • Lepkowski J.
        • Singer E.
        • Tourangeau R.
        Survey Methodology.
        2nd ed. John Wiley & Sons, Inc, Hoboken, NJ2009
        • Saunders C.L.
        • Elliott M.N.
        • Lyratzopoulos G.
        • Abel G.A.
        Do differential response rates to patient surveys between organizations lead to unfair performance comparisons? Evidence from the English Cancer Patient Experience Survey.
        Med Care. 2016; 54: 45-54
      2. Office of Management and Budget Standards and Guidelines for Statistical Surveys. September 2006.
        • Gerber Y.
        • Jacobsen S.J.
        • Killian J.M.
        • Weston S.A.
        • Roger V.L.
        Participation bias assessment in a community-based study of myocardial infarction, 2002-2005.
        Mayo Clin Proc. 2007; 82: 933-938
        • Bopp M.
        • Braun J.
        • Faeh D.
        • Swiss National Cohort Study Group
        Variation in mortality patterns among the general population, study participants, and different types of nonparticipants: evidence from 25 years of follow-up.
        Am J Epidemiol. 2014; 180: 1028-1035
        • Fry A.
        • Littlejohns T.J.
        • Sudlow C.
        • et al.
        Comparison of sociodemographic and health-related characteristics of UK Biobank participants with those of the general population.
        Am J Epidemiol. 2017; 186: 1026-1034
        • Berlin N.L.
        • Hamill J.B.
        • Qi J.
        • Kim H.M.
        • Pusic A.L.
        • Wilkins E.G.
        Nonresponse bias in survey research: lessons from a prospective study of breast reconstruction.
        J Surg Res. 2018; 224: 112-120
        • Knudsen A.K.
        • Hotopf M.
        • Skogen J.C.
        • Overland S.
        • Mykletun A.
        The health status of nonparticipants in a population-based health study: the Hordaland Health Study.
        Am J Epidemiol. 2010; 172: 1306-1314
        • Goldberg M.
        • Chastang J.F.
        • Leclerc A.
        • et al.
        Socioeconomic, demographic, occupational, and health factors associated with participation in a long-term epidemiologic survey: a prospective study of the French GAZEL cohort and its target population.
        Am J Epidemiol. 2001; 154: 373-384
        • St Sauver J.L.
        • Grossardt B.R.
        • Leibson C.L.
        • Yawn B.P.
        • Melton 3rd, L.J.
        • Rocca W.A.
        Generalizability of epidemiological findings and public health decisions: an illustration from the Rochester Epidemiology Project.
        Mayo Clin Proc. 2012; 87: 151-160
        • St Sauver J.L.
        • Jacobsen S.J.
        • Jacobson R.M.
        • et al.
        Assessing participation bias in a population-based study of measles-mumps-rubella vaccine immunity in children and adolescents aged 12-18.
        Paediatr Perinat Epidemiol. 2007; 21: 376-384
        • May C.R.
        • Cummings A.
        • Myall M.
        • et al.
        Experiences of long-term life-limiting conditions among patients and carers: what can we learn from a meta-review of systematic reviews of qualitative studies of chronic heart failure, chronic obstructive pulmonary disease and chronic kidney disease?.
        BMJ Open. 2016; 6: e011694
        • Watanabe M.
        • Olson K.
        • Falci C.
        Social isolation, survey nonresponse, and nonresponse bias: An empirical evaluation using social network data within an organization.
        Soc Sci Res. 2017; 63: 324-338
        • Lavallee D.C.
        • Chenok K.E.
        • Love R.M.
        • et al.
        Incorporating patient-reported outcomes into health care to engage patients and enhance care.
        Health Aff (Millwood). 2016; 35: 575-582
        • Reeve B.B.
        • Wyrwich K.W.
        • Wu A.W.
        • et al.
        ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research.
        Qual Life Res. 2013; 22: 1889-1905