Mayo Clinic Proceedings Home
MCP Digital Health Home
Brief report| Volume 98, ISSUE 3, P445-450, March 2023

A Clinician’s Guide to Running Custom Machine-Learning Models in an Electronic Health Record Environment


      We recently brought an internally developed machine-learning model for predicting which patients in the emergency department would require hospital admission into the live electronic health record environment. Doing so involved navigating several engineering challenges that required the expertise of multiple parties across our institution. Our team of physician data scientists developed, validated, and implemented the model. We recognize a broad interest and need to adopt machine-learning models into clinical practice and seek to share our experience to enable other clinician-led initiatives. This Brief Report covers the entire model deployment process, starting once a team has trained and validated a model they wish to deploy in live clinical operations.

      Abbreviations and Acronyms:

      AI (artificial intelligence), AUC (area under the receiver-operator curve), ED (emergency department), EHR (electronic health record), IT (information technology), ML (machine-learning), UI (user interface)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Mayo Clinic Proceedings
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Schafermeyer R.W.
        • Asplin B.R.
        Hospital and emergency department crowding in the United States.
        Emerg Med. 2003; 15: 22-27
        • Kelen G.D.
        • Wolfe R.
        • D’Onofrio G.
        • et al.
        Emergency department crowding: the canary in the health care system.
        N Engl J Med Catalyst. 2021; 5
        • Cavallo J.J.
        • Donoho D.A.
        • Forman H.P.
        Hospital capacity and operations in the coronavirus disease 2019 (COVID-19) pandemic: planning for the nth patient.
        JAMA Health Forum. 2020; 1e200345
      1. News and Publications. The Johns Hopkins Hospital launches capacity command center to enhance hospital operations. Johns Hopkins Medicine. Published online October 26, 2016. Accessed November 16, 2020.

        • MedTech Impact on Wellness
        Improving patient-care with hospital command centers: AI/ machine learning insights medical technology.
        • Ryu A.J.
        • Romero-Brufau S.
        • Qian R.
        • et al.
        Assessing the generalizability of a clinical machine learning model across multiple emergency departments.
        Mayo Clin Proc Innov Qual Outcomes. 2022; 6: 193-199

      Linked Article

      • Implementing Machine Learning in the Electronic Health Record: Checklist of Essential Considerations
        Mayo Clinic ProceedingsVol. 98Issue 3
        • Preview
          Machine learning (ML) holds significant promise for improving clinical care.1 To facilitate their appropriate and effective use, it is important that clinical guidance based on these ML models is provided automatically to end users as a part of routine clinical care processes,2 in particular through integration with electronic health record (EHR) systems. However, there is still relatively little literature on the actual deployment of ML models in EHRs to facilitate their appropriate use.
        • Full-Text
        • PDF