Mayo Clinic Proceedings Home

Measuring Faculty Reflection on Medical Grand Rounds at Mayo Clinic: Associations With Teaching Experience, Clinical Exposure, and Presenter Effectiveness



      To develop and validate a new instrument for measuring participant reflection on continuing medical education (CME) and determine associations between the reflection instrument scores and CME presenter, participant, and presentation characteristics.

      Participants and Methods

      This was a prospective validation study of presenters and faculty at the weekly medical grand rounds at Mayo Clinic in Rochester, Minnesota, from January 1, 2011, through June 30, 2011. Eight items (5-point Likert scales) were developed on the basis of 4 reflection levels: habitual action, understanding, reflection, and critical reflection. Factor analysis was performed to account for clustered data. Interrater and internal consistency reliabilities were calculated. Associations between reflection scores and characteristics of presenters, participants, and presentations were determined.


      Participants completed a total of 1134 reflection forms. Factor analysis revealed a 2-dimensional model (eigenvalue; Cronbach α): minimal reflection (1.19; 0.77) and high reflection (2.51; 0.81). Item mean (SD) scores ranged from 2.97 (1.17) to 4.01 (0.83) on a 5-point scale. Interrater reliability (intraclass correlation coefficient) for individual items ranged from 0.58 (95% CI, 0.31-0.78) to 0.88 (95% CI, 0.80-0.94). Reflection scores were associated with presenters' speaking effectiveness (P<.001) and prior CME teaching experience (P=.02), participants' prior clinical experiences (P<.001), and presentations that were case based (P<.001) and used the audience response system (P<.001).


      We report the first validated measure of reflection on CME at medical grand rounds. Reflection scores were associated with presenters' effectiveness and prior teaching experience, participants' clinical exposures, and presentations that were interactive and clinically relevant. Future research should determine whether reflection on CME leads to better patient outcomes.

      Abbreviations and Acronyms:

      CME (continuing medical education), MGR (medical grand rounds)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Mayo Clinic Proceedings
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Mansouri M.
        • Lockyer J.
        A meta-analysis of continuing medical education effectiveness.
        J Contin Educ Health Prof. 2007; 27: 6-15
        • Davis D.
        • O'Brien M.A.
        • Freemantle N.
        • Wolf F.M.
        • Mazmanian P.
        • Taylor-Vaisey A.
        Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes?.
        JAMA. 1999; 282: 867-874
        • Davis D.A.
        • Thomson M.A.
        • Oxman A.D.
        • Haynes R.B.
        Changing physician performance: a systematic review of the effect of continuing medical education strategies.
        JAMA. 1995; 274: 700-705
        • Forsetlund L.
        • Bjorndal A.
        • Rashidian A.
        • et al.
        Continuing education meetings and workshops: effects on professional practice and health care outcomes.
        Cochrane Database Syst Rev. 2009; : CD003030
        • Mamede S.
        • Schmidt H.G.
        The structure of reflective practice in medicine.
        Med Educ. 2004; 38: 1302-1308
        • Van Hoof T.J.
        • Monson R.J.
        • Majdalany G.T.
        • Giannotti T.E.
        • Meehan T.P.
        A case study of medical grand rounds: are we using effective methods?.
        Acad Med. 2009; 84: 1144-1151
        • Mezirow J.
        Transformative learning: theory to practice.
        in: Cranton P. New Directions for Adult and Continuing Education. Vol 74. Jossey-Bass, San Francisco, CA1997: 5-12
        • Wittich C.M.
        • Reed D.A.
        • McDonald F.S.
        • Varkey P.
        • Beckman T.J.
        Transformative learning: a framework using critical reflection to link the improvement competencies in graduate medical education.
        Acad Med. 2010; 85: 1790-1793
        • Moon J.
        Using reflective learning to improve the impact of short courses and workshops.
        J Contin Educ Health Prof. 2004; 24: 4-11
        • Lockyer J.
        • Gondocz S.T.
        • Thivierge R.L.
        Knowledge translation: the role and place of practice reflection.
        J Contin Educ Health Prof. 2004; 24: 50-56
        • Armson H.
        • Kinzie S.
        • Hawes D.
        • Roder S.
        • Wakefield J.
        • Elmslie T.
        Translating learning into practice: lessons from the practice-based small group learning program.
        Can Fam Physician. 2007; 53: 1477-1485
        • Sandars J.
        The use of reflection in medical education: AMEE Guide No. 44.
        Med Teach. 2009; 31: 685-695
        • Kember D.
        • Leung D.
        • Jones A.
        • et al.
        Development of a questionnaire to measure the level of reflective thinking.
        Assess Eval Higher Educ. 2000; 25: 381-395
        • Kirkpatrick D.
        Revisiting Kirkpatrick's four-level model.
        Training Dev. 1996; 50: 54-59
        • Beckman T.J.
        • Cook D.A.
        Developing scholarly projects in education: a primer for medical teachers.
        Med Teach. 2007; 29: 210-218
        • Mann K.
        • Gordon J.
        • MacLeod A.
        Reflection and reflective practice in health professions education: a systematic review.
        Adv Health Sci Educ. 2009; 14: 595-621
        • Wittich C.
        • Lopez-Jiminez F.
        • Decker L.
        • et al.
        Measuring faculty reflection on adverse patient events: development and initial validation of a case-based learning system.
        J Gen Int Med. 2010; 26: 293-298
        • Boud D.
        • Keogh R.
        • Walker D.
        Reflection: Turning Experience Into Learning.
        Kogan Page, London, England1985
        • Landis J.R.
        • Koch G.G.
        The measurement of observer agreement for categorical data.
        Biometrics. 1977; 33: 159-174
        • DeVellis R.F.
        Scale Development: Theory and Applications.
        Sage, Newbury Park, CA1991
        • Beckman T.J.
        Determining the validity and reliability of clinical assessment scores.
        in: Henderson M. A Textbook for Internal Medicine Education Programs. Association of Program Directors in Internal Medicine and Association of Specialty Professors, Washington, DC2007: 139-146
      1. Accreditation Council for Continuing Medical Education. Accessed October 5, 2012.

        • Wittich C.M.
        • Mauck K.F.
        • Mandrekar J.N.
        • et al.
        Improving participant feedback to continuing medical education presenters in internal medicine: a mixed-methods study.
        J Gen Int Med. 2012; 27: 425-431
        • Leung K.H.
        • Pluye P.
        • Grad R.
        • Weston C.
        A reflective learning framework to evaluate CME effects on practice reflection.
        J Contin Educ Health Prof. 2010; 30: 78-88
        • Lowe M.
        • Rappolt S.
        • Jaglal S.
        • Macdonald G.
        The role of reflection in implementing learning from continuing education into practice.
        J Contin Educ Health Prof. 2007; 27: 143-148
        • Beckman T.J.
        • Cook D.A.
        • Mandrekar J.N.
        Factor instability of clinical teaching assessment scores among general internists and cardiologists.
        Med Educ. 2006; 40: 1209-1216