Mayo Clinic Proceedings Home
MCP Digital Health Home
Original article| Volume 88, ISSUE 11, P1266-1271, November 2013

Comparison of the Quality of Patient Referrals From Physicians, Physician Assistants, and Nurse Practitioners

Published:October 11, 2013DOI:



      To compare the quality of referrals of patients with complex medical problems from nurse practitioners (NPs), physician assistants (PAs), and physicians to general internists.

      Patients and Methods

      We conducted a retrospective comparison study involving regional referrals to an academic medical center from January 1, 2009, through December 31, 2010. All 160 patients referred by NPs and PAs combined and a random sample of 160 patients referred by physicians were studied. Five experienced physicians blinded to the source of referral used a 7-item instrument to assess the quality of referrals. Internal consistency, interrater reliability, and dimensionality of item scores were determined. Differences between item scores for patients referred by physicians and those for patients referred by NPs and PAs combined were analyzed by using multivariate ordinal logistical regression adjusted for patient age, sex, distance of the referral source from Mayo Clinic, and Charlson Index.


      Factor analysis revealed a 1-dimensional measure of the quality of patient referrals. Interrater reliability (intraclass correlation coefficient for individual items: range, 0.77-0.93; overall, 0.92) and internal consistency for items combined (Cronbach α=0.75) were excellent. Referrals from physicians were scored higher (percentage of agree/strongly agree responses) than were referrals from NPs and PAs for each of the following items: referral question clearly articulated (86.3% vs 76.0%; P=.0007), clinical information provided (72.6% vs 54.1%; P=.003), documented understanding of the patient's pathophysiology (51.0% vs 30.3%; P<.0001), appropriate evaluation performed locally (60.3% vs 39.0%; P<.0001), appropriate management performed locally (53.5% vs 24.1%; P<.0001), and confidence returning patient to referring health care professional (67.8% vs 41.4%; P<.0001). Referrals from physicians were also less likely to be evaluated as having been unnecessary (30.1% vs 56.2%; P<.0001).


      The quality of referrals to an academic medical center was higher for physicians than for NPs and PAs regarding the clarity of the referral question, understanding of pathophysiology, and adequate prereferral evaluation and documentation.

      Abbreviations and Acronyms:

      NP (nurse practitioner), PA (physician assistant)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Mayo Clinic Proceedings
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • MacDowell M.
        • Glasser M.
        • Fitts M.
        • Nielsen K.
        • Hunsaker M.
        A national view of rural health workforce issues in the USA.
        Rural Remote Health. 2010; 10: 1531
        • Dill M.J.
        • Salsberg E.S.
        • Center for Workforce Studies
        The Complexities of Physician Supply and Demand: Projections Through 2025.
        American Association of Medical Colleges, Washington, DCNovember 2008
      1. American Academy of Family Practice, American Academy of Pediatrics, American College of Physicians, American Osteopathic Association. Joint Principles of the Patient Centered Medical Home. 2007; Accessed September 19, 2013.

      2. American Academy of Physician Assistants website. Accessed June 13, 2013.

      3. American College of Nurse Practitioners website. Accessed June 13, 2013.

      4. Making the numbers count. Center for Health Professions, University of California at San Francisco website. Published September 1, 2012. Accessed June 13, 2013.

        • Emanuel E.
        • Tanden N.
        • Altman S.
        • et al.
        A systemic approach to containing health care spending.
        N Engl J Med. 2012; 367: 949-954
        • Fairman J.A.
        • Rowe J.W.
        • Hassmiller S.
        • Shalala D.E.
        Broadening the scope of nursing practice.
        N Engl J Med. 2011; 364: 193-196
        • Blumenthal D.
        • Abrams M.K.
        Putting aside preconceptions—time for dialogue among primary care clinicians.
        N Engl J Med. 2013; 368: 1933-1934
        • Atwater A.
        • Bednar S.
        • Hassman D.
        • Khouri J.
        Nurse practitioners and physician assistants in primary care.
        Dis Mon. 2008; 54: 728-744
        • Jacobson P.D.
        • Parker L.E.
        • Coulter I.D.
        Nurse practitioners and physician assistants as primary care providers in institutional settings.
        Inquiry. 1998-1999; 35: 432-446
        • Laurant M.
        • Reeves D.
        • Hermens R.
        • Braspenning J.
        • Grol R.
        • Sibbald B.
        Substitution of doctors by nurses in primary care.
        Cochrane Database Syst Rev. 2005; : CD001271
        • Gershengorn H.B.
        • Johnson M.P.
        • Factor P.
        The use of nonphysician providers in adult intensive care units.
        Am J Respir Crit Care Med. 2012; 185: 600-605
        • Gershengorn H.B.
        • Wunsch H.
        • Wahab R.
        • et al.
        Impact of nonphysician staffing on outcomes in a medical ICU.
        Chest. 2011; 139: 1347-1353
        • Kleinpell R.M.
        • Ely E.W.
        • Grabenkort R.
        Nurse practitioners and physician assistants in the intensive care unit: an evidence-based review.
        Crit Care Med. 2008; 36: 2888-2897
        • Fried T.R.
        • Tinetti M.E.
        • Iannone L.
        Primary care clinicians' experiences with treatment decision making for older persons with multiple conditions.
        Arch Intern Med. 2011; 171: 75-80
        • Subramanian U.
        • Kerr E.A.
        • Klamerus M.L.
        • Zikmund-Fisher B.J.
        • Holleman R.G.
        • Hofer T.P.
        Treatment decisions for complex patients: differences between primary care physicians and midlevel providers.
        Am J Manag Care. 2009; 15: 373-380
        • Larson E.B.
        • Fihn S.D.
        • Kirk L.M.
        • et al.
        The future of general internal medicine. Report and recommendations from the Society of General Internal Medicine (SGIM) Task Force on the Domain of General Internal Medicine.
        J Gen Intern Med. 2004; 19: 69-77
        • Charlson M.E.
        • Pompei P.
        • Ales K.L.
        • MacKenzie C.R.
        A new method of classifying prognostic comorbidity in longitudinal studies: development and validation.
        J Chronic Dis. 1987; 40: 373-383
        • Landis J.R.
        • Koch G.G.
        The measurement of observer agreement for categorical data.
        Biometrics. 1977; 33: 159-174
        • Gillard J.N.
        • Szoke A.
        • Hoff W.S.
        • Wainwright G.A.
        • Stehly C.D.
        • Toedter L.J.
        Utilization of PAs and NPs at a level I trauma center: effects on outcomes.
        JAAPA. 2011; 24 (40-43): 34
        • Sherwood K.
        • Sugerman S.
        • Bossart P.
        • et al.
        EDOU staffing by PAs: what are the effects on patient outcomes?.
        JAAPA. 2011; 24 (37): 31-34
        • Moote M.
        • Krsek C.
        • Kleinpell R.
        • Todd B.
        Physician assistant and nurse practitioner utilization in academic medical centers.
        Am J Med Qual. 2011; 26: 452-460