Original articleSepsis surveillance from administrative data in the absence of a perfect verification
Introduction
Sepsis is a major public health problem and is one of the leading causes of death in the United States [1]. The high morbidity of sepsis results in $20.3 billion in annual hospital costs in the United States [2], in addition to the potential costs associated with permanent organ damage, long-term cognitive impairment, and functional disability [3]. The Agency for Healthcare Research and Quality reported that sepsis was involved in 2.8% of all hospitalizations in 2011 [2].
Sepsis was defined in 1991 by a consensus conference of the American College of Chest Physicians and the Society of Critical Care Medicine as a syndrome of dysregulated inflammatory response to severe infection [4]. The consensus group recognized (and reaffirmed in 2001) the host response, called the systemic inflammatory response syndrome, as a result of suspected or confirmed infection, for the definition, as opposed to the presence of a specific infection [4]. The most recent revision (i.e., Sepsis-3) to the consensus definition defines sepsis as a life-threatening organ dysfunction as a result of a dysregulated response to an infectious insult [5]. The diverse causes and clinical manifestations of sepsis such as pneumonia or urinary tract infection accompanied with organ dysfunctions or shock have created difficulty for surveillance and assessment of quality of care.
Several multicenter studies and national reports in the literature that relied on administrative billing data, suggested that the incidence of sepsis has been increasing by about 10% annually [6], [7], [8], [9], [10], [11], [12], [13], [14]. Similarly, a recent 5-year study at our tertiary-care center reported a 9.7% annual percent change in hospitalizations with a discharge diagnosis of sepsis [15]. The results of a study at our institution also did not find an increase in sepsis incidence when we used patient-level data to adjust for the coinciding improvement in the clinical diagnosis of sepsis, its documentation, and administrative coding of sepsis during the same period [16]. These studies demonstrated a lack of a temporal trend in the apparent prevalence (or incidence) of sepsis; however, there has not been an attempt to estimate the true prevalence of sepsis by adjusting for the misclassification bias due to the imperfect accuracy of current sepsis detection using administrative data.
In this study, we developed criteria, referred to as surveillance-aimed sepsis detection (SASD) criteria to estimate the true prevalence of sepsis from administrative data. In specifying the criteria, we considered some fundamental concepts of a surveillance system such as simplicity of implementation, accuracy (diagnostic sensitivity and specificity), precision (repeatability and reproducibility), timeliness (quick implementation), utility (flexibility and extensibility of methods to evolving settings and conditions), and value (low or no cost compared to accrued value) [17]. In devising SASD, we intended the criteria to be applied to aggregate-level data for surveillance purposes, rather than in a clinical setting for an individual patient. Unlike some published studies [14], [18], [19], [20], [21], [22], [23], [24], we did not assume that our criteria or any other reference or validation method has perfect accuracy. We adapted appropriate analytical techniques to adjust for the misclassification bias due to imperfect verification and to estimate the true prevalence of sepsis, while we coherently incorporated all uncertainties regarding the unknown quantities in our inference [25], [26]. Finally, we illustrated the use of methods for surveillance using an imperfect diagnostic criterion and provided an open-source program code that can be readily adapted for surveillance of conditions of interest using administrative data or electronic health records.
Section snippets
Study setting and population
The study population included all inpatient stays for patients, who were 18 years of age or older, admitted to Barnes-Jewish Hospital, an academic tertiary-care center affiliated with Washington University School of Medicine in St. Louis, Missouri, between January 1, 2008 and December 31, 2012. Administrative data and electronic health records containing clinical, pharmacy, and laboratory data for Barnes-Jewish Hospital were available from the BJC HealthCare's Center for Clinical Excellence and
Results
We examined a total of 273,126 hospitalizations. The apparent prevalence of sepsis hospitalizations based on explicit ICD-9-CM codes was 1.5% (808/53,291), 1.4% (783/54,293), 1.6% (888/55,090), 2.2% (1182/54,284), and 2.5% (1422/56,168) from 2008 to 2012, respectively. Table 2 presents cross-classified results of the SASD criteria for the study period.
Estimates of the true prevalence, sensitivities, and specificities of the SASD criteria are presented in Table 3. The results suggested that the
Discussion
Our surveillance-aimed criteria estimated the true prevalence of sepsis to be about 18%, which remained stable during the study period at our institution. This study follows the results of two previous studies at the our institution that suggested an uptrend in the apparent prevalence of hospitalizations with a discharge diagnosis code for sepsis [15], [16]. Our findings are similar to those from Iwashyna et al. [43], who reported an apparent prevalence of sepsis to be 13.5% based on an
Acknowledgments
This work was supported by the Prevention Epicenters Program from the Centers for Disease Control and Prevention (CDC) (grants U54 CK000162 and U54 CK000172) and the Washington University Institute of Clinical and Translational Sciences from the National Center for Advancing Translational Sciences (NCATS) (grant UL1 TR000448).
References (50)
- et al.
Nationwide trends of severe sepsis in the 21st century (2000-2007)
Chest
(2011) - et al.
Epidemiology of severe sepsis: 2008-2012
J Crit Care
(2016) - et al.
Quantifying the improvement in sepsis diagnosis, documentation, and coding: the marginal causal effect of year of hospitalization on sepsis diagnosis
Ann Epidemiol
(2016) - et al.
Positive predictive value of the AHRQ Patient Safety Indicator “Postoperative Sepsis”: implications for practice and policy
J Am Coll Surg
(2011) - et al.
Epidemiologic issues in the validation of veterinary diagnostic tests
Prev Vet Med
(2000) - et al.
Estimation of sensitivity and specificity of diagnostic tests and disease prevalence when the true disease state is unknown
Prev Vet Med
(2000) - et al.
Bayesian modeling of animal- and herd-level prevalences
Prev Vet Med
(2004) - et al.
Estimation of diagnostic-test sensitivity and specificity through Bayesian modeling
Prev Vet Med
(2005) - et al.
On the interpretation of test sensitivity in the two-test two-population problem: assumptions matter
Prev Vet Med
(2009) - et al.
Conditional dependence between tests affects the diagnosis and surveillance of animal diseases
Prev Vet Med
(2000)