Mayo Clinic Proceedings Home

Developing the Surveillance Algorithm for Detection of Failure to Recognize and Treat Severe Sepsis

Published:January 06, 2015DOI:



      To develop and test an automated surveillance algorithm (sepsis “sniffer”) for the detection of severe sepsis and monitoring failure to recognize and treat severe sepsis in a timely manner.

      Patients and Methods

      We conducted an observational diagnostic performance study using independent derivation and validation cohorts from an electronic medical record database of the medical intensive care unit (ICU) of a tertiary referral center. All patients aged 18 years and older who were admitted to the medical ICU from January 1 through March 31, 2013 (N=587), were included. The criterion standard for severe sepsis/septic shock was manual review by 2 trained reviewers with a third superreviewer for cases of interobserver disagreement. Critical appraisal of false-positive and false-negative alerts, along with recursive data partitioning, was performed for algorithm optimization.


      An algorithm based on criteria for suspicion of infection, systemic inflammatory response syndrome, organ hypoperfusion and dysfunction, and shock had a sensitivity of 80% and a specificity of 96% when applied to the validation cohort. In order, low systolic blood pressure, systemic inflammatory response syndrome positivity, and suspicion of infection were determined through recursive data partitioning to be of greatest predictive value. Lastly, 117 alert-positive patients (68% of the 171 patients with severe sepsis) had a delay in recognition and treatment, defined as no lactate and central venous pressure measurement within 2 hours of the alert.


      The optimized sniffer accurately identified patients with severe sepsis that bedside clinicians failed to recognize and treat in a timely manner.

      Abbreviations and Acronyms:

      CPOE (computerized physician order entry), CVP (central venous pressure), DNR/DNI (do-not-resuscitate/do-not-intubate), ED (emergency department), EGDT (early goal-directed therapy), EMR (electronic medical record), ICU (intensive care unit), LOS (length of stay), NPV (negative predictive value), PPV (positive predictive value), SIRS (systemic inflammatory response syndrome), SSC (Surviving Sepsis Campaign)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Mayo Clinic Proceedings
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Martin G.S.
        • Mannino D.M.
        • Eaton S.
        • Moss M.
        The epidemiology of sepsis in the United States from 1979 through 2000.
        N Engl J Med. 2003; 348: 1546-1554
        • Poeze M.
        • Ramsay G.
        • Gerlach H.
        • Rubulotta F.
        • Levy M.
        An international sepsis survey: a study of doctors' knowledge and perception about sepsis.
        Crit Care. 2004; 8: R409-R413
        • Miller III, R.R.
        • Dong L.
        • Nelson N.C.
        • et al.
        • Intermountain Healthcare Intensive Medicine Clinical Program
        Multicenter implementation of a severe sepsis and septic shock treatment bundle.
        Am J Respir Crit Care Med. 2013; 188: 77-82
        • Torio C.M.
        • Andrews R.M.
        National Inpatient Hospital Costs: The Most Expensive Conditions by Payer, 2011.
        Agency for Healthcare Research and Quality, Rockville, MD2013 (HCUP Statistical Brief 160)
        • Rivers E.
        • Nguyen B.
        • Havstad S.
        • et al.
        • Early Goal-Directed Therapy Collaborative Group
        Early goal-directed therapy in the treatment of severe sepsis and septic shock.
        N Engl J Med. 2001; 345: 1368-1377
        • ProCESS Investigators
        A randomized trial of protocol-based care for early septic shock.
        N Engl J Med. 2014; 370: 1683-1693
        • ARISE Investigators, ANZICS Clinical Trials Group
        Goal-directed resuscitation for patients with early septic shock.
        N Engl J Med. 2014; 371: 1496-1506
        • Dellinger R.P.
        • Levy M.M.
        • Rhodes A.
        • et al.
        • Surviving Sepsis Campaign Guidelines Committee including the Pediatric Subgroup
        Surviving Sepsis Campaign: international guidelines for management of severe sepsis and septic shock: 2012.
        Crit Care Med. 2013; 41: 580-637
        • Sawyer A.M.
        • Deal E.N.
        • Labelle A.J.
        • et al.
        Implementation of a real-time computerized sepsis alert in nonintensive care unit patients.
        Crit Care Med. 2011; 39: 469-473
        • Nelson J.L.
        • Smith B.L.
        • Jared J.D.
        • Younger J.G.
        Prospective trial of real-time electronic surveillance to expedite early care of severe sepsis.
        Ann Emerg Med. 2011; 57: 500-504
        • LaRosa J.A.
        • Ahmad N.
        • Feinberg M.
        • Shah M.
        • Dibrienza R.
        • Studer S.
        The use of an early alert system to improve compliance with sepsis bundles and to assess impact on mortality.
        Crit Care Res Pract. 2012; 2012: 980369
        • Hooper M.H.
        • Weavind L.
        • Wheeler A.P.
        • et al.
        Randomized trial of automated, electronic monitoring to facilitate early detection of sepsis in the intensive care unit.
        Crit Care Med. 2012; 40: 2096-2101
        • Klein Klouwenberg P.M.
        • Ong D.S.
        • Bonten M.J.
        • Cremer O.L.
        Classification of sepsis, severe sepsis and septic shock: the impact of minor variations in data capture and definition of SIRS criteria.
        Intensive Care Med. 2012; 38: 811-819
        • Silber J.H.
        • Williams S.V.
        • Krakauer H.
        • Schwartz J.S.
        Hospital and patient characteristics associated with death after surgery: a study of adverse occurrence and failure to rescue.
        Med Care. 1992; 30: 615-629
        • Silber J.H.
        • Rosenbaum P.R.
        • Schwartz J.S.
        • Ross R.N.
        • Williams S.V.
        Evaluation of the complication rate as a measure of quality of care in coronary artery bypass graft surgery.
        JAMA. 1995; 274: 317-323
        • Levy M.M.
        • Dellinger R.P.
        • Townsend S.R.
        • et al.
        • Surviving Sepsis Campaign
        The Surviving Sepsis Campaign: results of an international guideline-based performance improvement program targeting severe sepsis.
        Crit Care Med. 2010; 38: 367-374
        • Donchin Y.
        • Gopher D.
        • Olin M.
        • et al.
        A look into the nature and causes of human errors in the intensive care unit.
        Crit Care Med. 1995; 23: 294-300
        • Herasevich V.
        • Kor D.J.
        • Subramanian A.
        • Pickering B.W.
        Connecting the dots: rule-based decision support systems in the modern EMR era.
        J Clin Monit Comput. 2013; 27: 443-448
        • Afessa B.
        • Keegan M.T.
        • Hubmayr R.D.
        • et al.
        Evaluating the performance of an institution using an intensive care unit benchmark.
        Mayo Clin Proc. 2005; 80: 174-180
        • Alberti C.
        • Brun-Buisson C.
        • Burchardi H.
        • et al.
        Epidemiology of sepsis and infection in ICU patients from an international multicentre cohort study.
        Intensive Care Med. 2002; 28: 108-121
        • Annane D.
        • Aegerter P.
        • Jars-Guincestre M.C.
        • Guidet B.
        • CUB-Réa Network
        Current epidemiology of septic shock: the CUB-Réa Network.
        Am J Respir Crit Care Med. 2003; 168: 165-172
        • Herasevich V.
        • Pickering B.W.
        • Dong Y.
        • Peters S.G.
        • Gajic O.
        Informatics infrastructure for syndrome surveillance, decision support, reporting, and modeling of critical illness.
        Mayo Clin Proc. 2010; 85: 247-254
        • Schramm G.E.
        • Kashyap R.
        • Mullon J.J.
        • Gajic O.
        • Afessa B.
        Septic shock: a multidisciplinary response team and weekly feedback to clinicians improve the process of care and mortality.
        Crit Care Med. 2011; 39: 252-258
        • Gaudard M.
        • Ramsey P.
        • Stephens M.
        Interactive Data Mining and Design of Experiments: The JMP® Partition and Custom Design Platforms.
        North Haven Group, LLC, Brookline, NH2006
        • Herasevich V.
        • Pieper M.S.
        • Pulido J.
        • Gajic O.
        Enrollment into a time sensitive clinical study in the critical care setting: results from computerized septic shock sniffer implementation.
        J Am Med Inform Assoc. 2011; 18: 639-644
        • Viera A.J.
        • Garrett J.M.
        Understanding interobserver agreement: the kappa statistic.
        Fam Med. 2005; 37: 360-363
        • Sim J.
        • Wright C.C.
        The kappa statistic in reliability studies: use, interpretation, and sample size requirements.
        Phys Ther. 2005; 85: 257-268
        • Ferrer R.
        • Artigas A.
        • Levy M.M.
        • et al.
        • Edusepsis Study Group
        Improvement in process of care and outcome after a multicenter severe sepsis educational program in Spain.
        JAMA. 2008; 299: 2294-2303
        • Jaimes F.
        • Garcés J.
        • Cuervo J.
        • et al.
        The systemic inflammatory response syndrome (SIRS) to identify infected patients in the emergency room.
        Intensive Care Med. 2003; 29: 1368-1371
        • Kaukonen K.M.
        • Bailey M.
        • Suzuki S.
        • Pilcher D.
        • Bellomo R.
        Mortality related to severe sepsis and septic shock among critically ill patients in Australia and New Zealand, 2000-2012.
        JAMA. 2014; 311: 1308-1316
        • Stevenson E.K.
        • Rubenstein A.R.
        • Radin G.T.
        • Wiener R.S.
        • Walkey A.J.
        Two decades of mortality trends among patients with severe sepsis: a comparative meta-analysis.
        Crit Care Med. 2014; 42: 625-631
        • Gaieski D.F.
        • Edwards J.M.
        • Kallan M.J.
        • Carr B.G.
        Benchmarking the incidence and mortality of severe sepsis in the United States.
        Crit Care Med. 2013; 41: 1167-1174
        • Artenstein A.W.
        • Higgins T.L.
        • Opal S.M.
        Sepsis and scientific revolutions.
        Crit Care Med. 2013; 41: 2770-2772
        • Chopra V.
        • Govindan S.
        • Kuhn L.
        • et al.
        Do clinicians know which of their patients have central venous catheters? a multicenter observational study.
        Ann Intern Med. 2014; 161: 562-567
        • Nguyen L.
        • Patrick H.
        Unexpected relationship between central venous pressure (CVP) and mortality in patients with severe sepsis.
        Chest. 2014; 146 ([abstract]): 231A