Using NSQIP to calculate mortality risk from NSTIs
Development and Validation of a
Necrotizing Soft-Tissue Infection
Mortality Risk Calculator Using NSQIP
Iris Faraklas, RN, BSN, Gregory J.
Stoddard, MPH, Leigh Neumayer MD, FACS, Jeffrey
Saffle, MD, FACS, Amalia Cochran, MD, FACS
University of Utah
Necrotizing Soft Tissue Infections
• Aggressive infections requiring prompt surgical
debridement and systemic support
– May be mono- or polymicrobial
• Incidence in US: 500-1,500 cases per year
• Previous work from our group showed a
mortality rate of 12%
Identifying Risk Factors
• Focus on correctly
• Anaya et al (2009)
created a mortality
– 2 centers
– 350 patients in
National Surgical Quality Improvement
• Created by the VA
– Managed by ACS
• Prospective, multicenter database
• Patient data
– Pre-, intra-, and postoperative variables
– 30-day postoperative mortality and morbidity
Receiver Operating Characteristic (ROC)
• Risk estimate to 1
from non-cases .8
– Non-survivors from
• ROC=1.0 if .4
• ROC=0.50 if 0.50
ROC area = 0.85
discrimination is no 0
0 .2 .4 .6 .8 1
better than chance 1 - Specificity
Methods: Statistical Analysis
• Univariate exploratory analysis on each variable
compared to mortality
• All variables showing significance (p<0.05) were
included in stepwise multiple logistic regression
• Preoperative laboratory values >400
observations were included:
– Hct, BUN, Crt, Plt
• The bootstrap method was used to validate the
• 1,392 patients identified
– 82% (1,142) Necrotizing fasciitis
– 8.6% (119) Gas gangrene
– 9.4% (131) Fournier’s gangrene
Number of NSTI cases vs. Total NSQIP Cases per Year
Year NSTI Cases Total NSQIP Cases
2005 47 34,099
2006 140 118,391
2007 212 211,407
2008 281 271,368
2009 352 336,190
2010 360 363,431
Results: Demographics n=1,392
• Median Age: 55 (IQR: 46-63)
• 58% Male
• Median BMI: 32 (IQR: 26-40)
• 51% Were either partially or totally dependent
• 49% Diabetic
• 54% Hypertension requiring medication
Necrotizing soft-tissue infections (NSTI) are a group of uncommon, aggressive infections, requiring prompt surgical debridement and systemic support ; these infections may be monomicrobial or polymicrobial. Varying studies have reported the incidence in the US ranges from 500-1,500 cases per year. A previous large patient series from our group showed a mortality rate of 12%.
In the past decade investigators have focused on correctly identifying NSTI cases. Some studies have tried to delineate specific risk mortality factors . To date Anaya (An I yah) et al created the only mortality risk scoring system. Shown in this table are their 6 variables. It should be noted that this scoring system was created from only 2 centers with a total of 350 patients over 9 years.
Large national databases with validated data obtained by trained collectors have emerged as valuable sources of high-quality data, creating a unique opportunity to evaluate mortality risk factors for NSTI. The National Surgical Quality Improvement Program (NSQIP) is perhaps the best of these databases. The VA had the foresight to create NSQIP- currently it is managed by the American College of Surgeons. nsqip is a prospective, multicenter database with greater than 500 hospitals participating. Over a 130 data points are collected, including pre, intra and post operative variables, 30-day mortality and morbidity outcomes are also collected for patients undergoing surgical procedures. With this large program we were able to develop and validate a mortality risk calculator for NSTI patients.
Before diving into our study…a little statistical back story A Receiver Operating Characteristic (or the ROC) is a plot of the true positive against the false positive rate at different points in a diagnostic test. It is the ability of the risk estimate to discriminate cases from noncases (or in our study: nonsurvivors from survivors).15, 16 The closer the curve follows the left-hand border and then the top border, the more accurate the test. The ROC area equals 1 if discrimination is perfect <<click>>and .5 if it is no better than chance.
Among statisticians a rule of thumb is that a ROC of >.7 shows acceptable discrimination in clinical studies , while a ROC >.8 is considered excellent discrimination.
The objective of this study was to develop and validate a 30-day postoperative mortality risk calculator for NSTI patients using the NSQIP database
After receiving IRB approval, data were extracted from 2005 through 2010. The Participant Use Data Files do not identify hospitals, health care providers or patients. The files are HIPAA compliant. Patients discharged with an ICD-9 diagnosis of necrotizing fasciitis; Fournier’s gangrene, or gas gangrene were included in the analysis.
Due to our time constraint The full definition of all nsqip variables are delineated elsewhere. 23 Some of the pre- and intraoperative variables included were: Demographic and lifestyle variables, comorbidities and previous surgeries Other factors considered were the patient’s ASA classification and presence of septic shock. If available within 2 days prior to surgery, preoperative laboratory variables were reported as number of abnormal per reported. Missing laboratory variables were not assumed to be normal and missing values were excluded from regression.
The primary outcome variable was surgical mortality. Outcomes were tracked for 30 days postoperatively. If the patient was still inpatient at 30 days they were tracked until discharge or until NSQIP locked the data file at 120 days after their first surgery.
Univariate exploratory analysis was performed on each preoperative variable compared to mortality. All variables showing significance were included in the stepwise multiple logistic regression analysis. Only the following preoperative laboratory values had > 400 observations and were included in the regression: hematocrit, BUN, creatinine and platelet count. Bootstrap method was used to validate this model
A total of 1,392 patients were identified from 2005 through 2010, 82% had a diagnosis of necrotizing fasciitis , 8.6% were diagnosed with gas gangrene and the remaining 9.4% were diagnosed with Fournier’s gangrene. As shown in this Table , the number of NSTI admissions reported to NSQIP increased progressively from 47 patients in 2005 to 360 in 2010. Please note that the total number NSQIP cases reported increased more than 10X over this same period as centers were added to the program
This is a much abbreviated general demographics slide. Patients were mostly male in their 50s, and obese. More than half were considered partially or totally dependent, and required medication to treat their diabetes and or hypertension.
The majority were admitted from home. Over 40% were ASA classified as a 4 or greater; a quarter were in septic shock, while a third had had previous surgery in the past month and the majority of the surgeries were considered emergent in nature meaning they were within 12 hours of admission
Median length of stay was 16 days; however there were 9 patients that were still in the hospital at time that NSQIP locked their data file (>120 days from first surgery). And the Thirty-day mortality was 13%.
Multivariable logistic regression identified the following 7 independent variables that affected mortality: patients older than 60, totally dependent functional status, requiring dialysis prior to surgery, having a ASA class 4 or greater, considered as an emergent surgery (meaning <12 hours post admission), septic shock and low platelet count stratified between those <50,000 and those (<150,000/mm3 but more than 50,000
The ROC for our model was .85 which indicates a strong predictive model. Using boot-strap validation the ROC curve area was .83, which represents how the model will behave in future patients.
The model was used to develop an interactive risk calculator. Once the model was validated, an interactive spreadsheet was created that uses the demonstrated risk factors from a specific patient to return a probability of mortality expressed as a percent. This calculator is available upon request. It calculates a patient’s estimated risk of mortality, after clinicians enter patient data based on the variable definitions that are listed on spreadsheet. Definitions are the same as NSQIP definitions and if the user hovers over the variable the specific definition pops up.
As with any study, limitations exist: due to the rarity of disease the number of patients included in this analysis is relatively small. Although NSQIP is probably one of the best databases available- it doesn’t include the microbial picture or which patients required serial debridements. Nor are we able to control for diagnosis variablity that might occur at different facilities which is ultimately only as good as the ICD-9 coding. Also tertiary facilities might participate in NSQIP more frequently thus a selection bias might exist.
As stated earlier an ROC greater than .8 is considered an excellent predictive model this calculator is based on a strong predictive model (with an ROC of .85). The variables included in the model are available early in the hospital course. However risk models should not dictate management but are just one added tool in the clinican’s toolbox. This simply provides extra information or a helpful communication tool to help patients and their families make informed decisions.