Abstract
Objective
To explore the relationship of electronic health record (EHR)–based audit log data with physician burnout and clinical practice process measures.
Methods
From September 4 to October 7, 2019, we surveyed physicians in a larger academic medical department and matched responses to August 1 through October 31, 2019, EHR-based audit log data. Multivariable regression analysis evaluated the relationship between log data and burnout and the interrelationship between log data and turnaround time for In Basket messages and percentage of encounters closed within 24 hours.
Results
Of the 537 physicians surveyed, 413 (77%) responded. On multivariable analysis, number of In Basket messages received per day (each additional message: odds ratio, 1.04 [95% CI, 1.02 to 1.07]; P<.001) and time spent in the EHR outside scheduled patient care (each additional hour: odds ratio, 1.01 [95% CI, 1.00 to 1.02]; P=.04) were associated with burnout. Time spent doing In Basket work (each additional minute: parameter estimate, −0.11 [95% CI, −0.19 to −0.03]; P=.01) and in the EHR outside scheduled patient care (each additional hour: parameter estimate, 0.04 [95% CI, 0.01 to 0.06]; P=.002) were associated with turnaround time (days per message) for In Basket messages. None of the variables explored were independently associated with percentage of encounters closed within 24 hours.
Conclusion
Electronic health record–based audit log data of workload relate to odds of burnout and responsiveness to patient-related inquiries and results. Further study is needed to determine whether interventions that reduce the number of and time spent doing In Basket messages or time spent in the EHR outside scheduled patient care reduce physician burnout and improve clinical practice process measures.
Abbreviations and Acronyms:
EHR (electronic health record), NVC (non-visit care)Physician well-being has an impact on the patient’s experience. Professional burnout, a consequence of chronic, unmitigated work stress,
1
is associated with medical errors, malpractice litigation, impaired professionalism, lower commitment to patient care, and reduced patient satisfaction.World Health Organization
International Classification of Diseases for Mortality and Morbidity Statistics.
International Classification of Diseases for Mortality and Morbidity Statistics.
https://icd.who.int/browse11/l-m/en#/http://id.who.int/icd/entity/129180281
Date accessed: October 4, 2022
2
,3
In addition, as the severity of burnout increases among physicians, so does their intent to reduce their clinical hours or to leave their current job.2
Previous studies have reported that physicians with burnout are twice as likely as physicians without burnout to leave their organization during the next 2 years.4
, 5
, 6
Such reductions in clinical effort and turnover negatively affect patients’ access to care as well as continuity of care. Long-term patient-physician relationships are associated not only with higher patient satisfaction but also with better health care outcomes and lower costs.7
,8
Leaders in medicine have called for efforts to improve the patient experience, to optimize health care outcomes, and to reduce health care expenditures to include system-level approaches to mitigate work stress and to reduce burnout.
2
,9
, 10
, 11
, 12
A critical part of such efforts—and a key recommendation from the National Academy of Medicine—is routine measurement and monitoring of physician burnout and potential contributory factors.2
Although pragmatic strategies can be employed to reduce responder burden and to guide institutional action,13
an upstream approach could be a useful complementary strategy that could signal work units at risk of high levels of burnout if prompt action is not taken, with potential to alter trajectory and to mitigate risk of burnout consequences. One well-recognized contributor to burnout is workload, which can be tracked, at least partially, through electronic health record (EHR)–based audit log data.14
,15
Electronic health record–based audit log data (eg, time spent in the EHR, number of In Basket messages received) have been proposed as a metric to be included in system improvement efforts intended to reduce work load or to improve work efficiency.16
,17
National organizations, such as the American Medical Association, are setting the expectation that health care organizations monitor and respond to EHR-based audit log data in their efforts to improve physician well-being and to reduce burnout.18
American Medical Association
Joy in Medicine Health System Recognition Program.
Joy in Medicine Health System Recognition Program.
https://www.ama-assn.org/practice-management/sustainability/joy-medicine-health-system-recognition-program
Date accessed: January 31, 2023
Previous studies have reported relationships between EHR-based audit log data (ie, total In Basket volume, patient call volumes, and time spent entering notes) and symptoms of burnout among physicians.
19
, 20
, 21
, 22
Most of these studies aggregated physicians with other health care professionals, included only primary care physicians, used a measure of burnout likely to underestimate the prevalence of burnout, did not link survey data with EHR-based audit log data, or did not adjust analysis for personal or professional characteristics that relate to risk of burnout.19
, 20
, 21
, 22
, 23
, 24
In addition, no previous studies have explored interrelationships between EHR-based audit log data variables that could potentially provide insight into efficiency or reflect other clinical practice process measures to augment traditional practice analytics usually leveraged to identify patient experience and practice improvement opportunities. For example, turnaround time for In Basket messages and percentage of clinic encounters completed within 24 hours (of the appointment) are EHR-based audit log data reflecting timeliness of response to patient inquiries and test results and clinical notes being electronically available to patients, other health care workers, and the billing office. Such information may provide insights into areas of the practice to be improved to optimize the patient experience, team-based care, and timely collection of revenue.The aim of our study was to evaluate the relationship of EHR-based audit log data of workload with occupational burnout and to explore the interrelationship between various EHR-based audit log data and responsiveness to patient-related inquiries and results and timeliness of completed outpatient documentation. The overarching goal is to explore whether EHR-based audit log data, in their current form and function, could serve as leading indicators for physician burnout and clinical practice process measures that identify practice improvement opportunities.
Methods
Participants
All physicians in the Department of Medicine, Rochester, Minnesota, were asked to complete the annual Program on Physician Well-Being survey between September 4 and October 7, 2019. The Department of Medicine has 11 divisions ranging in size from 7 to 97 physicians: endocrinology, gastroenterology, general internal medicine (a referral-based, non–primary care practice that includes executive health), hematology, hospital medicine, infectious diseases, nephrology and hypertension, preventive medicine, primary care (community-based, general internal medicine continuity practices in Rochester, Minnesota, and Mayo Clinic Health System campuses in Wisconsin and Minnesota), pulmonary and critical care, and rheumatology. The invitation email informed individuals that participation was voluntary and that responses were confidential. Responders who worked in hospital internal medicine exclusively were not included in this analysis. The Mayo Clinic institutional review board deemed this study exempt.
Electronic Health Record Workload Variables
Electronic health record–based audit log data were obtained from the Epic Signal tool (Epic Systems Corporation, 1999-2021). We selected the subset of data for evaluation and analysis from all the available metadata in audit logs based on literature, insights from the KLAS Arch Collaborative,
25
and our collective expertise (Table 1). For each physician, we obtained these data points from August through October 2019. This was 8 months after the completion of an organization-wide EHR implementation initiative and 2 months after the first upgrade and optimization effort. Most physicians had 15 months of experience with the EHR, with the maximum physician experience with the Mayo Clinic EHR being 25 months.KLAS Collaborative
https://klasresearch.com/klas-model
Date accessed: October 4, 2022
Table 1Electronic Health Record–Based Log Audit Data Variables
| Definition | |
|---|---|
| Workload measures | |
| Patient appointments per day | Numerator: Number of appointments in the reporting period Denominator: Sum of scheduled half-days (0.5-4 hours of scheduled time in clinic) and full days (>4 hours of scheduled time in clinic) in the reporting period |
| No. of In Basket messages received per day (all types) | Numerator: Total messages received within the reporting period Denominator: Total number of days that the providers logged in and worked during the reporting period |
| No. of In Basket messages related to non-visit care | The following specific types of In Basket messages: patient calls, patient advice requests, and prescription refills |
| No. of In Basket messages related to results | The following specific types of In Basket messages: results messages, result notes (messages generated by a health care professional to another staff to direct results management) |
| In Basket minutes per day | Numerator: Total number of minutes spent per physician in an In Basket activity or navigator section within the reporting period Denominator: Total number of days that the physician logged in and worked during the reporting period |
| Time outside scheduled hours | Numerator: Number of minutes spent in the system outside of scheduled hours based on Cadence data with a 30-minute buffer before the first appointment and after the last appointment Denominator: Number of scheduled days in the reporting period where time was spent in the system |
| Time outside of 7 am to 7 pm | Numerator: Number of minutes spent in the system outside 7 am to 7 pm on scheduled days Denominator: Number of scheduled days in the reporting period where time was spent in the system outside 7 am to 7 pm |
| Pajama time | Numerator: Number of minutes spent in charting activities outside 7 am to 5:30 pm on weekdays and outside scheduled hours on weekends Denominator: Number of scheduled days in the reporting period where time was spent in the system |
| Time on unscheduled days | Numerator: Number of minutes spent in the system on days with no scheduled patients Denominator: Number of unscheduled days in the reporting period where time was spent in the system |
| Clinical practice process measures | |
| Turnaround time for In Basket messages | Numerator: Sum of the number of days it took for the physician to mark all messages of this type as done Denominator: Total number of messages of this type |
| Percentage of closed office encounters same day | Numerator: Number of appointments that were closed the same day as the appointment date Denominator: Number of appointments the physician had in the reporting period |
Clinical Practice Process Measures
To reflect responsiveness to patient-related inquiries and results, we selected turnaround time (average days per message) for In Basket messages (sum of turnaround time for the following types of In Basket messages: patient calls, patient advice requests, prescription refills, results messages, and result notes) from the EHR-based audit log data measures (Table 1). As a metric of timeliness of completed outpatient documentation, we chose percentage of closed office encounters the same day (ie, note was signed the same day the patient was seen in the office). We also obtained these data points from August through October 2019 for each physician.
Survey
The survey included basic demographic characteristics (age, sex), average weekly work hours, and 2 single measures from the Maslach Burnout Inventory, used under license with Mind Garden, Inc, previously demonstrated in samples of more than 10,000 physicians to stratify risk of burnout with areas under the receiver operating characteristic curve of 0.94 for the single item of emotional exhaustion and 0.93 for the single item of depersonalization.
26
,27
Each measure has a 7-item frequency response option ranging from never to every day. The positive predictive values of the single-item thresholds for high levels of emotional exhaustion and depersonalization (ie, once a week or more) are 88.2% and 89.6%, with positive likelihood ratios of 14.9 and 23.4, respectively.27
Overall burnout was defined by a high score on the emotional exhaustion or depersonalization item, as has been done in multiple prior studies.28
, 29
, 30
Survey responses were linked to EHR-based audit log data, and all identifiers were stripped before analysis.Statistical Analyses
For analysis purposes, we created 4 variables:
- 1.Time in the EHR outside scheduled patient care (a sum of time outside scheduled hours and time on unscheduled days) to reflect total time physicians spent in the EHR when not engaged in direct face-to-face care22;
- 2.Total non-visit care (NVC) In Basket messages (sum of patient calls, patient advice requests, and prescription refills);
- 3.Total results management In Basket messages (results messages and result notes); and
- 4.Total NVC plus result management In Basket result messages as these are the most common types of in-box messages (there are >100 types of In Basket messages).
We calculated basic summary statistics and explored relationships between dependent and independent variables by using Fisher exact test (for categorical variables) or Kruskal-Wallis test (for continuous variables), examining relationships overall and by primary care vs non–primary care specialties. All tests were 2 sided, with a type I error rate of .05. Collinearity diagnostics were performed, and the results did not support impactful collinearity. We then conducted 3 separate multivariable analyses to evaluate the relationship between EHR workload variables and burnout, turnaround time for In Basket messages, and percentage of office encounters closed the same day. The multivariable models included age, sex, specialty (primary care vs not), appointments per day (for each additional appointment), number of In Basket messages received per day, In Basket minutes per day (for each additional minute), and calculated measure of total time in the EHR outside of scheduled patient care (for each additional hour). Given large differences in In Basket messages received between primary care and non–primary care physicians, we re-ran the models for turnaround time for In Basket messages separately for primary care and non–primary care physicians.
Results
In aggregate, 413 of 533 physicians (77.5%) responded to the survey. Of these responders, 394 had EHR-based audit log data available from August through October 2019 and were included in the analysis. Demographic and professional characteristics of participants are shown in Table 2. Overall, 36.6% (137/374) were female, 19.5% (77/394) worked in primary care, and self-reported mean weekly work hours were 56.9. Within this cohort, 38.7% (151/392) had high emotional exhaustion, 21.8% (85/392) had high depersonalization, and 40.0% (156/392) had overall burnout.
Table 2Characteristics of Responding Physicians
| Variable | No. (%) or Mean (SD) |
|---|---|
| Age, years | |
| <30 | 3 (0.8) |
| 31-40 | 101 (26.7) |
| 41-50 | 104 (27.5) |
| 51-60 | 97 (25.7) |
| ≥61 | 73 (19.3) |
| Missing | 16 |
| Sex | |
| Male | 230 (61.5) |
| Female | 137 (36.6) |
| Other | 7 (1.9) |
| Missing | 20 |
| Specialty | |
| Allergy | 5 (1.3) |
| Endocrine | 25 (6.3) |
| Gastroenterology | 50 (12.7) |
| General internal medicine | 74 (18.8) |
| Hematology | 40 (10.2) |
| Infectious disease | 21 (5.3) |
| Nephrology and hypertension | 29 (7.4) |
| Preventive medicine, occupational health, and aerospace medicine | 14 (3.6) |
| Primary care | 77 (19.5) |
| Pulmonary and critical care medicine | 46 (11.7) |
| Rheumatology | 13 (3.3) |
| Missing | 0 |
| Average work hours per week | 56.9 (13.8) |
| Burnout | |
| High emotional exhaustion | 151 (38.7) |
| High depersonalization | 85 (21.8) |
| Overall burnout | 156 (40.0) |
a A referral-based, non–primary care practice that includes executive health.
b Community-based, general internal medicine continuity practices in Rochester, Minnesota, and Mayo Clinic Health System campuses in Wisconsin and Minnesota.
Electronic health record workload variables and patient experience measures for the 394 physicians are shown in Table 3 overall and by primary care vs non–primary care specialties. The mean number of In Basket messages received per day was 26.0 (SD 17.2). On average, physicians had 4 In Basket messages for each appointment per scheduled day in clinic. Overall, physicians spent 13.9 (SD 9.8) minutes per day completing In Basket work, 36.2 (SD 24.6) minutes per day in the EHR beyond face-to-face time with patients on clinic days (ie, time outside of scheduled hours), and an additional 39.9 (SD 29.6) minutes per day in the EHR on days not seeing patients in clinic (ie, time on unscheduled days). These EHR workload variables had large ranges, and on average primary care physicians had more In Basket messages (Supplemental Figure, available online at http://www.mayoclinicproceedings.org) and spent more time in the EHR outside time allocated for direct patient care (Table 3).
Table 3Electronic Health Record–Based Audit Log Data Among Survey Responders From August Through October 2019
| Overall (N=394) | Primary care (n=77) | Non–primary care (n=317) | P value | ||
|---|---|---|---|---|---|
| Mean (SD) | Range | Mean (SD) | Mean (SD) | ||
| EHR workload variables | |||||
| Appointments per day | 7.6 (4.3) | 0.2-58.0 | 8.7 (2.9) | 7.3 (4.6) | <.001 |
| No. of In Basket messages received per day | 26.0 (17.2) | 0.4-99.3 | 44.1 (22.0) | 21.5 (12.3) | <.001 |
| No. of messages per month (below categories) | 444.5 (327.9) | 3.0-1919.7 | 794.0 (381.3) | 359.6 (248.5) | <.001 |
| NVC encounter messages per month | 130.0 (121.4) | 0.3-595.7 | 303.6 (135.0) | 87.7 (69.1) | <.001 |
| Patient calls | 62.4 (52.7) | 0.3-351.3 | 124.3 (63.6) | 47.4 (36.4) | <.001 |
| Patient advice requests | 34.7 (34.3) | 0.0-217.0 | 61.8 (48.2) | 27.9 (25.8) | <.001 |
| Prescription refills | 35.5 (53.1) | 0.0-267.3 | 119.0 (62.6) | 13.8 (16.2) | <.001 |
| Results management messages per month | 122.1 (89.1) | 0.0-577.0 | 174.9 (75.0) | 109.2 (87.6) | <.001 |
| Results message | 117.8 (85.6) | 0.0-564.7 | 158.2 (67.5) | 107.9 (86.7) | <.001 |
| Result notes message | 5.3 (8.8) | 0.0-69.7 | 16.9 (11.8) | 1.7 (2.0) | <.001 |
| Total In Basket messages per appointment per scheduled day | 4.0 (3.1) | 0.1-37.3 | 5.2 (2.5) | 3.7 (3.2) | <.001 |
| In Basket minutes per day | 13.9 (9.8) | 0.3-74.1 | 24.9 (10.5) | 11.2 (7.5) | <.001 |
| Time outside scheduled hours, minutes per day | 36.2 (24.6) | 1.9-155.9 | 44.3 (30.0) | 33.9 (22.4) | .009 |
| Time outside of 7 am to 7 pm, minutes per day | 17.5 (18.5) | 0.1-205.9 | 18.7 (16.2) | 17.1 (19.1) | .25 |
| Pajama time, minutes per day | 12.2 (15.7) | 0.0-106.0 | 13.2 (15.3) | 11.9 (15.8) | .12 |
| Time on unscheduled days, minutes per day | 39.9 (29.6) | 0.9-184.9 | 54.7 (35.7) | 35.7 (26.2) | <.001 |
| Total time in the EHR outside scheduled patient care, total hours across 3 months | 30.3 (26.1) | 0.8-208 | 43.5 (29.7) | 26.6 (23.8) | <.001 |
| EHR patient experience variables | |||||
| Turnaround time (below In Basket message types), days | 3.5 (4.9) | 0.3-40.3 | 2.0 (4.3) | 3.9 (5.0) | <.001 |
| Turnaround time NVC encounter messages, days | 3.0 (4.8) | 0.1-54.1 | 1.6 (4.1) | 3.4 (4.9) | <.001 |
| Patient calls average TAT/message | 3.4 (5.0) | 0.1-42.9 | 1.8 (4.7) | 3.8 (5.0) | <.001 |
| Patient advice requests average TAT/message | 3.6 (9.2) | 0.0-112.1 | 3.8 (15.6) | 3.5 (6.7) | <.001 |
| Prescription authorizations average TAT/message | 0.8 (1.7) | 0.0-20.8 | 0.3 (0.3) | 0.9 (1.9) | <.001 |
| Turnaround time results management messages, days | 4.0 (5.7) | 0.0-47.3 | 2.5 (3.5) | 4.4 (6.1) | .002 |
| Results average TAT/message, days | 4.0 (5.8) | 0.0-47.6 | 2.5 (3.5) | 4.4 (6.1) | .001 |
| Result notes average TAT/message, days | 3.8 (9.8) | 0.0-132.0 | 2.6 (2.5) | 4.2 (11.1) | .006 |
| Percentage of closed office encounters the same day | 68.6 (31.5) | 0.0-100.0 | 67.3 (31.4) | 69.0 (31.5) | .70 |
a EHR, electronic health record; NVC, non-visit care; TAT, turnaround time for In Basket messages.
b Average of number of scheduled appointments divided by number of full days or half-days scheduled to see patients in August through October 2019.
c Average turnaround time for non-visit care encounter and results management messages combined.
Overall, mean turnaround time for In Basket messages was 3.5 (SD 4.9) days per message; however, responsiveness to patient inquiries was around 3 days, with prescription requests being completed within 1 day on average. Typically, patients were informed of new test results in an average of 4 days. Despite having higher volumes of In Basket messages, on average primary care physicians had shorter turnaround times on most measures. Approximately two-thirds (68.6%) of clinic notes were completed within 24 hours.
Electronic Health Record Workload Measures and Burnout
On average, physicians with burnout received more In Basket messages, had more In Basket messages per appointment per scheduled day in clinic, and spent more time each day completing In Basket messages (Table 4). Greater numbers of NVC encounter messages, in particular patient calls, and both types of result management messages were associated with burnout. Physicians with burnout spent more time in the EHR on days not scheduled to see patients and had more total time in the EHR outside scheduled patient care.
Table 4Relationship Between EHR Workload Variables and Burnout, Turnaround Time, and Office Encounters Closed the Same Day
| Burnout | Turnaround time for In Basket messages | Mean % office encounters closed the same day | |||||
|---|---|---|---|---|---|---|---|
| Burnout, mean (SD) | No burnout, mean (SD) | P value | GLM parameter estimate (95% CI) | P value | GLM parameter estimate (95% CI) | P value | |
| Appointments per day | 7.9 (5.4) | 7.4 (3.5) | .36 | 0.01 (−0.10 to 0.12) | .86 | −0.39 (−1.12 to 0.34) | .29 |
| No. of In Basket messages received per day (all types) | 30.6 (19.5) | 23.0 (14.9) | <.001 | −0.05 (−0.08 to −0.02) | <.001 | −0.27 (−0.45 to −0.09) | .004 |
| No. of In Basket messages per month (below categories) | 531.1 (378.4) | 390.8 (277.8) | <.001 | −0.00 (−0.00 to −0.00) | <.001 | −0.01 (−0.02 to −0.00) | .004 |
| NVC encounter messages per month | 153.7 (139.2) | 116.0 (106.0) | .01 | −0.01 (−0.01 to −0.00) | <.001 | −0.01 (−0.04 to 0.01) | .33 |
| Patient calls | 74.7 (62.7) | 55.1 (43.5) | .006 | −0.02 (−0.03 to −0.01) | <.001 | −0.02 (−0.08 to 0.04) | .44 |
| Patient advice requests | 36.9 (34.3) | 33.6 (34.4) | .07 | −0.03 (−0.04 to −0.01) | <.001 | −0.03 (−0.13 to 0.06) | .46 |
| Prescription refills | 44.9 (65.6) | 29.7 (42.1) | .12 | −0.02 (−0.02 to −0.01) | .001 | −0.03 (−0.09 to 0.03) | .32 |
| Results management messages per month | 147.7 (99.0) | 105.7 (78.4) | <.001 | −0.01 (−0.01 to −0.00) | .007 | 0.01 (−0.02 to 0.05) | .54 |
| Results message | 142.1 (94.9) | 102.1 (75.5) | <.001 | −0.01 (−0.01 to −0.00) | .014 | 0.01 (−0.02 to 0.05) | .46 |
| Result notes message | 6.6 (10.2) | 4.4 (7.5) | .03 | −0.11 (−0.17 to −0.05) | <.001 | −0.22 (−0.60 to 0.16) | .26 |
| Total In Basket messages per appointment per scheduled day | 4.6 (3.9) | 3.5 (2.4) | <.001 | −0.21 (−0.37 to −0.06) | .008 | −1.71 (−2.72 to −0.71) | <.001 |
| In Basket minutes per day | 15.7 (10.0) | 12.7 (9.6) | <.001 | −0.10 (−0.14 to −0.05) | <.001 | −0.51 (−0.82 to −0.19) | .002 |
| Time outside scheduled hours, minutes per day | 38.9 (27.0) | 34.4 (23.0) | .12 | 0.02 (−0.00 to 0.04) | .09 | −0.12 (−0.25 to 0.01) | .07 |
| Time outside of 7 am to 7 pm, minutes per day | 18.5 (23.42) | 16.4 (13.9) | .76 | 0.06 (0.03-0.08) | <.001 | −0.16 (−0.34 to 0.02) | .08 |
| Pajama time, minutes per day | 13.6 (17.6) | 11.2 (14.2) | .34 | 0.06 (0.02-0.09) | <.001 | −0.31 (−0.51 to −0.10) | .003 |
| Time on unscheduled days, minutes per day | 47.1 (34.9) | 34.8 (24.1) | <.001 | 0.03 (0.01-0.04) | .001 | −0.14 (−0.25 to −0.03) | .01 |
| Total hours in the EHR outside scheduled patient care, for each additional hour | 34.6 (29.4) | 27.5 (23.4) | .005 | 0.01 (−0.01 to 0.03) | .15 | −0.16 (−0.28 to −0.04) | .012 |
a EHR, electronic health record; GLM, generalized linear model; NVC, non-visit care;
b A negative number indicates slower average turnaround time for In Basket messages.
c A negative number indicates fewer charts closed the same day.
d Average of number of scheduled appointments divided by number of full days or half-days scheduled to see patients in August through October 2019.
e A calculated number representing sum of time outside scheduled hours and time on unscheduled days to reflect total time physicians spent in the EHR when not engaged in direct face-to-face care.
On multivariable analysis controlling for sex, age, specialty, appointments per day, and time spent in the In Basket per day, more In Basket messages received per day (for each additional message: odds ratio, 1.04 [95% CI, 1.02 to 1.07]; P<.001) and total time spent in the EHR outside scheduled patient care (for each additional hour: odds ratio, 1.01 [95% CI, 1.00 to 1.02]; P=.04) were independently associated with higher odds of burnout (Table 5).
Table 5Multivariable Analysis of EHRa Workload Variables and Burnout, Turnaround Time, and Office Encounters Closed the Same Day
| Burnout | Turnaround time for In Basket messages | Mean % office encounters closed the same day | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Odds ratio (95% CI) | P value | Overall P value | Parameter estimate (95% CI) | P value | Overall P value | Parameter estimate (95% CI) | P value | Overall P value | |
| Sex (vs male) | .76 | .05 | .33 | ||||||
| Female | 1.20 (0.72-2.02) | .49 | 1.37 (0.27-2.46) | .02 | −4.45 (−11.89 to 2.99) | .24 | |||
| Other | 0.86 (0.15-4.79) | .86 | 0.25 (−3.12 to 3.61) | .89 | −12.36 (−35.26 to 10.53) | .29 | |||
| Age, years (vs <40) | .43 | .53 | .86 | ||||||
| 41-50 | 1.17 (0.63-2.18) | .61 | −0.17 (−1.48 to 1.13) | .79 | −3.09 (−11.94 to 5.76) | .49 | |||
| 51-60 | 0.86 (0.45-1.64) | .64 | 0.64 (−0.70 to 1.98) | .35 | −3.61 (−12.71 to 5.50) | .44 | |||
| 61+ | 0.65 (0.32- 1.33) | .24 | 0.67 (−0.79 to 2.13) | .37 | −2.67 (−12.59 to 7.25) | .60 | |||
| Primary care vs not | 0.60 (0.30-1.22) | .16 | −1.59 (−3.05 to −0.13) | .03 | 9.32 (−0.63 to 19.27) | .07 | |||
| Appointments per day, for each additional appointment | 0.95 (0.86-1.04) | .24 | 0.12 (−0.06 to 0.30) | .19 | 0.08 (−1.13 to 1.29) | .90 | |||
| No. of In Basket messages received per day | 1.04 (1.02-1.07) | <.001 | 0.002 (−0.04 to 0.05) | .91 | −0.27 (−0.56 to 0.02) | .06 | |||
| In Basket minutes per day (for each additional minute) | 0.98 (0.94-1.02) | .37 | −0.11 (−0.19 to −0.03) | .01 | −0.29 (−0.85 to 0.27) | .31 | |||
| Total time in the EHR outside scheduled patient care (for each additional hour) | 1.01 (1.00-1.02) | .04 | 0.04 (0.01-0.06) | .002 | −0.10 (−0.25 to 0.05) | .18 | |||
aEHR, electronic health record.
b A negative number indicates slower average turnaround time for In Basket messages.
c A negative number indicates fewer charts closed the same day.
Electronic Health Record Workload Measures and Clinical Practice Process Measures
On average, physicians who received more In Basket messages (all types) and spent more time completing In Basket work had longer turnaround times for patient-related inquires and results (Table 4). The more time physicians spent in the EHR on days scheduled and not scheduled to work in the clinic, the faster their turnaround time was for In Basket messages. On multivariable analysis controlling for sex, age, specialty, appointments per day, and number of In Basket messages received per day, physicians who spent more time doing In Basket work took more days to respond to patient-related inquires and results (each additional minute: parameter estimate, −0.11 [95% CI, −0.19 to −0.03]; P=.01). Physicians who spent more time in the EHR outside scheduled patient care responded in fewer days to In Basket messages (for each additional hour: parameter estimate, 0.04 [95% CI, 0.01 to 0.06]; P=.002).
Because of the substantial differences in In Basket volume for primary care and non–primary care physicians, we examined differences in turnaround time for messages by specialty (Supplemental Table 1, available online at http://www.mayoclinicproceedings.org). For primary care physicians, more In Basket messages per patient appointment per scheduled day and more time spent completing In Basket tasks were associated with slower turnaround times. The relationship between time spent completing In Basket tasks and slower turnaround time persisted on multivariable analysis controlling for other factors (for each additional minute: parameter estimate, −0.11 [95% CI, −0.21 to −0.01]; P=.03; Supplemental Table 2, available online at http://www.mayoclinicproceedings.org). Among physicians in non–primary care, a greater volume of In Basket messages and in particular of patient calls was associated with slower turnaround time, whereas more time in the EHR was associated with faster turnaround time. On multivariable analysis, total time in the EHR outside scheduled patient care was independently associated with faster turnaround time (parameter estimate, 0.04 [95% CI, 0.02 to 0.07]; P=.003).
Physicians who received more In Basket messages and spent more time completing In Basket messages had fewer office encounters closed the same day (Table 4). The more time physicians spent charting after hours in the EHR (ie, pajama time), completing tasks in the EHR on days not scheduled to see patients, and working in the EHR outside of scheduled patient care, the fewer encounters were closed within 24 hours. On multivariable analysis, none of the explored variables were independently associated with percentage of office encounters closed the same day (Table 5).
Discussion
In this study of physicians in a large academic medical department, volume of In Basket messages, time spent doing In Basket work, and total time spent in the EHR outside scheduled patient care were independently associated with burnout and turnaround time for In Basket messages after adjustment for physician sex, age, specialty, appointments per day, and work hours. Further study is needed to determine whether tracking and responding to these measures ultimately reduce physician burnout and improve clinical practice processes.
Among this cohort of physicians, each additional In Basket message received per day was associated with a 4% higher odds of burnout. The volume of In Basket messages received per day ranged widely, with primary care physicians receiving, on average, twice as many messages as non–primary care physicians. Nearly a quarter of these messages stemmed from patients calls that often necessitate a call back to the patient, work not captured in EHR audit log data. Recent studies have reported that physicians are receiving more In Basket messages and spending more time in the EHR now relative to prepandemic levels.
31
,32
Correspondingly, a national study of US physicians approximately 21 months into the pandemic revealed a dramatic increase in the prevalence of burnout.33
Together these results imply that system-level approaches to reduce physician burnout should include new reimbursement models that enable replacement of traditional face-to-face clinic visits with blocked time during the clinical workday for In Basket work and new approaches to reduce the volume of In Basket messages received by physicians. New team-based workflows are assumed to be a solution to this problem. However, this has not been rigorously tested. Strategies are also needed that assess the efficiency of different team-based models in terms of physician workload, response time, and patient satisfaction.Consistent with previous research,
19
we also found that a particular type of In Basket message that records patient calls to which physicians need to reply was associated with increased odds of burnout. We further demonstrate that higher patient call volumes are also associated with slower responsiveness time to patient inquiries and results, possibly reflecting a prioritization of returning patient calls to completing other In Basket tasks, particularly among non–primary care physicians. Time spent returning patient calls is additional work often unaccounted for by health care organizations and traditionally not reimbursed, leaving it as an add-on activity after a busy clinical day.More time logged doing In Basket work related to burnout, slower turnaround time for In Basket messages, and fewer clinical encounters completed day of appointment. The relationship between more time recorded doing In Basket work and slower turnaround time for In Basket messages persisted on multivariable analysis after controlling for sex, age, specialty, appointments per day, volume of In Basket messages received, and total time in the EHR outside scheduled patient care. However, when examined separately by specialty, this finding persisted only for primary care physicians on multivariable analysis. Reasons for this warrant further study and may stem from differences in triage in patient-generated In Basket messages, task distribution across team members, and other factors.
Total time physicians spent in the EHR when not engaged in direct face-to-face care averaged 30 hours during the 3-month period (approximately 10 hours per month or 600 minutes per month). This calculated measure outside scheduled patient care may more completely estimate total time physicians spend in the EHR outside of direct patient care as it considers time in the EHR on clinic and nonclinic days and is not limited to certain activities (eg, pajama time estimates time spent in charting activities per day scheduled to care for patients). On average, US physicians spend more time per workday and after hours in the EHR than non-US physicians,
34
suggesting that completing EHR-related tasks outside scheduled patient care time to this extent is not a necessary consequence of clinical care with currently available health information technologies.On multivariable analysis, we found an independent dose-response relationship between each additional minute per day of being in the EHR outside scheduled patient care time and greater odds of burnout. We also found that the more time physicians spent in the EHR outside scheduled patient care time, the faster they completed In Basket tasks. However, when examined separately by specialty, the relationship between time in the EHR and turnaround time for In Basket messages persisted only among non–primary care physicians after controlling for other factors. This may be due to differences in how time is spent in the EHR outside scheduled patient care time by specialty.
This study has several limitations. This single-institution study was conducted within the Department of Medicine. The findings may not be generalizable to other specialties or nonacademic settings. The study was cross-sectional, and thus the direction of the observed relationships cannot be determined. In addition, the study included a limited number of factors likely to have an impact on work stress, turnaround time to In Basket messages, and note completion. There could also be factors moderating or mediating effect on the relationships explored. We explored interaction effects between burnout, EHR time, and measures of efficiency and did not identify consistent patterns or any interaction effects that substantially changed overall model outcomes. Vendor-available variables have several drawbacks. For example, In Basket messages received per day and time spent in the In Basket average the total messages received and time spent over the total number of days physicians log into the EHR, regardless of whether they are scheduled to care for patients, potentially underestimating the workload. In addition, the time log stops accumulating measurements with each pause in system use lasting more than 5 seconds (eg, moving the mouse, clicking, scrolling, and making keystrokes) and does not include time spent in the EHR using a mobile application (eg, Haiku or Canto). Time completing In Basket work is likely to be underestimated as the time does not include time spent reviewing other aspects of the chart (eg, chart review, medications) needed to answer the message. The study did not consider patient complexity, differences in cognitive load associated with various EHR-related tasks, physician savviness with the EHR, maturity of triage of In Basket messages by other care team members, or other factors influencing user experience or other system-related factors affecting work-related stress.
35
Electronic health record–based audit log data obtained from the Epic Signal tool in 2019 are limited to clinical work conducted in the outpatient setting, and many of the physicians in this cohort are in specialties with considerable hospital and procedural practices, affecting the metrics on time in the EHR. Tracking and responding to EHR-based audit log data will likely need to be complementary to other approaches to successfully affect the physician and patient experience.36
Future integration of data from procedure-based platforms (eg, Provation for gastroenterology, Solus Endoscopy suit for pulmonary medicine) as well as other unmeasured efforts (voice recording software, eg, M Modal, Nuance) into a metadata interface could allow more comprehensive information about work effort. Last, Epic Signal data are limited to those who are actively practicing and delivering direct patient care on a minimum number of days. As such, this may exclude low-volume physicians and select for higher volume physicians who are at increased risk for burnout. Alternatively, this may exclude erroneous events, null measurements, and false log-ins. Therefore, whereas using vendor-defined limits does introduce selection bias, doing so limits data corruption and creates consistencies across studies and organizations that may facilitate progress.Conclusion
This study suggests that the volume of In Basket messages, the time spent doing In Basket work, and the calculated measure total time spent in the EHR outside scheduled patient care relate to risk of burnout and markers of the patient experience. Given the relationship between burnout and quality of care, turnover, and suicidal ideation risk, health system leaders should consider tracking and addressing factors contributing to high In Basket volume and time physicians spend in the EHR outside scheduled patient care time. Future studies are needed to explore whether these vendor-defined or vendor-derived EHR-based audit log data can serve as useful targets for proactive identification of subgroups of physicians at risk for having excessive work-related stress and whether acting on these data ultimately reduces burnout and improves patient satisfaction and outcomes. In addition, it may also be worth exploring whether EHR-based audit log data may be useful as part of future models that identify work units with high engagement and efficiency. With identification of those work units, internal team dynamics, strategies, role delineations, and tool configurations could be shared and emulated throughout an organization for the betterment of all.
Potential Competing Interests
Dr Dyrbye is co-inventor of the Well-being Index instruments (outside submitted work). Mayo Clinic holds the copyright for these instruments and has licensed them for use outside of Mayo Clinic. Dr Dyrbye receives a portion of any royalties paid to Mayo Clinic. Dr Dyrbye receives honoraria for lectures and other presentations related to health care professional well-being and health care trainee well-being. Dr O’Horo has received grants from Nference, Inc and the MITRE corporation as well as personal consulting fees from Bates College outside of the submitted work.
Supplemental Online Material
- Supplementary Figure 1, Tables 1 and 2
References
- International Classification of Diseases for Mortality and Morbidity Statistics.(Version 4/2019. QD85 Burnout)https://icd.who.int/browse11/l-m/en#/http://id.who.int/icd/entity/129180281Date accessed: October 4, 2022
- Taking Action Against Clinician Burnout: A Systems Approach to Professional Well-Being.National Academies Press, 2019
- Physician burnout: contributors, consequences and solutions.J Intern Med. 2018; 283: 516-529
- Estimating institutional physician turnover attributable to self-reported burnout and associated financial burden: a case study.BMC Health Serv Res. 2018; 18: 851
- Burnout and health care workforce turnover.Ann Fam Med. 2019; 17: 36-41
- Correlates and outcomes of physician burnout within a large academic medical center.JAMA Intern Med. 2018; 178: 856-858
- Physician wellness in rural America: a review.Int J Psychiatry Med. 2013; 46: 303-313
- Continuity of care: still important in modern-day general practice.Br J Gen Pract. 2016; 66: 396-397
- From triple to quadruple aim: care of the patient requires care of the provider.Ann Fam Med. 2014; 12: 573-576
- To care is human—collectively confronting the clinician-burnout crisis.N Engl J Med. 2018; 378: 312-314
- Two decades since to err is human: progress, but still a “chasm.”.JAMA. 2020; 324: 2489-2490
- Preventing a parallel pandemic—a national strategy to protect clinicians' well-being.N Engl J Med. 2020; 383: 513-515
- A pragmatic approach for organizations to measure health care professional well-being. NAM Perspectives.Discussion Paper, 2018
- Metrics for assessing physician activity using electronic health record log data.J Am Med Inform Assoc. 2020; 27: 639-643
- Implementing measurement science for electronic health record use.JAMA. 2021; 325: 2149-2150
- EHR audit logs: a new goldmine for health services research?.J Biomed Inform. 2020; 101103343
- How to measure progress in addressing physician well-being: beyond burnout.JAMA. 2021; 326: 2129-2130
- Joy in Medicine Health System Recognition Program.https://www.ama-assn.org/practice-management/sustainability/joy-medicine-health-system-recognition-programDate accessed: January 31, 2023
- Are specific elements of electronic health record use associated with clinician burnout more than others?.J Am Med Inform Assoc. 2020; 27: 1401-1410
- Electronic health records and burnout: time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians.J Am Med Inform Assoc. 2020; 27: 531-538
- Physicians' well-being linked to in-basket messages generated by algorithms in electronic health records.Health Aff (Millwood). 2019; 38: 1073-1078
- Clinician burnout associated with sex, clinician type, work culture, and use of electronic health records.JAMA Netw Open. 2021; 4e215686
- Validation of single-item screening measures for provider burnout in a rural health care network.Eval Health Prof. 2015; 39: 215-225
- Establishing crosswalks between common measures of burnout in US physicians.J Gen Intern Med. 2022; 37: 777-784
- https://klasresearch.com/klas-modelDate accessed: October 4, 2022
- Concurrent validity of single-item measures of emotional exhaustion and depersonalization in burnout assessment.J Gen Intern Med. 2012; 27: 1445-1452
- Single item measures of emotional exhaustion and depersonalization are useful for assessing burnout in medical professionals.J Gen Intern Med. 2009; 24: 1318-1321
- Changes in burnout and satisfaction with work-life integration in physicians and the general US working population between 2011 and 2017.Mayo Clin Proc. 2019; 94: 1681-1694
- Characterization of nonphysician health care workers’ burnout and subsequent changes in work effort.JAMA Netw Open. 2021; 4e2121435
- Association of racial bias with burnout among resident physicians.JAMA Netw Open. 2019; 2e197457
- Trends in electronic health record inbox messaging during the COVID-19 pandemic in an ambulatory practice network in New England.JAMA Netw Open. 2021; 4e2131490
- Assessing the impact of the COVID-19 pandemic on clinician ambulatory electronic health record use.J Am Med Inform Assoc. 2021; 29 (Published correction appears in J Am Med Inform Assoc. 2022;29(4):749): 453-460
- Changes in burnout and satisfaction with work-life integration in physicians and the general US working population between 2011 and 2020.Mayo Clin Proc. 2022; 97: 491-506
- Assessment of electronic health record use between US and non-US health systems.JAMA Intern Med. 2021; 181: 251-259
- The complex case of EHRs: examining the factors impacting the EHR user experience.J Am Med Inform Assoc. 2019; 26 (Published correction appears in J Am Med Inform Assoc. 2019;26(11):1424): 673-677
- Conceptual considerations for using EHR-based activity logs to measure clinician burnout and its effects.J Am Med Inform Assoc. 2021; 28: 1032-1037
Article info
Footnotes
Dr Dyrbye is now Senior Associate Dean and Chief Well-being Officer at University of Colorado, Denver.
Grant Support: Funding for this study was provided by the Mayo Clinic Program on Physician Well-Being and was based on work partially supported by a National Science Foundation grant (No.2041339). Any opinions, findings, and conclusions or recommendations expressed in this article are those of the authors and do not necessarily reflect the views of the National Science Foundation. Funding sources had no role in study design; in the collection, analysis, and interpretation of data; or in the writing and publication of this article.
Identification
Copyright
© 2023 Published by Elsevier Inc on behalf of Mayo Foundation for Medical Education and Research
ScienceDirect
Access this article on ScienceDirectLinked Article
- In the Limelight: March 2023Mayo Clinic ProceedingsVol. 98Issue 3
