Predicting mortality in pediatric sepsis: A laudable but elusive goal

“Science is not, despite how it is often portrayed, about absolute truths. It is about developing an understanding of the world, making predictions, and then testing these predictions.”

Brian Schmidt, Nobel Laureate in Physics, 2011

Sepsis is a syndrome with protean clinical manifestations and unpredictable responses to therapy and, hence, predicting sepsis outcomes is difficult. Attempts to improve early and rapid diagnosis of sepsis, follow response to therapies and predict outcomes have led to a multitude of studies pursuing biomarkers.¹ However, despite considerable investment, biomarker research has yielded few biomarkers useful in clinical practice.² The complex pathophysiology of sepsis has led some to investigate a panel of biomarkers with limited success.³

In this issue of the Jornal de Pediatria, Tonial et al. take the approach of predicting pediatric sepsis prognosis using a panel of biomarkers and an existing pediatric mortality prediction score, the Pediatric Index of Mortality 2 (PIM2).⁴ PIM2 is a composite score representing patient risk factors, patient comorbidities, key laboratory data, and vital sign measurements. Tonial et al. sought to test the prognostic performance of each of the following alone and in combination: biomarkers reflecting the host response (CRP, ferritin, and leukocyte count), a marker of perfusion deficit (lactate) and the PIM2 score.^4,5 The researchers retrospectively studied children aged 6 months to 18 years admitted to the pediatric intensive care unit (PICU) of a tertiary care center in Brazil with a diagnosis of sepsis defined by systemic inflammatory response syndrome (SIRS) criteria and in whom all biomarkers were measured (N = 294 of 350 admissions).^4,6 For prognostication, the authors used the highest ferritin and CRP levels within 48 h and the highest lactate level and leukocyte count within 24 h of PICU admission.⁴

Overall, the researchers found that the PIM2 score had the best discriminatory power for mortality (area under the curve [AUC] 0.815; 95% CI 0.766–0.858), better than any single biomarker (ferritin AUC 0.785, 95% CI 0.733–0.830; lactate AUC 0.762, 95% CI 0.709–0.810; and CRP AUC 0.648, 95% CI 0.590–0.702).⁴ Leukocyte count stood out as the sole biomarker with poor discriminatory power; it was as good at predicting mortality as a coin toss (AUC 0.508; 95% CI 0.450–0.567).⁴ It is not surprising that leukocyte count was uninformative when discriminating between septic patients given that SIRS criteria, which includes a criterion for abnormal leukocyte count, was used to diagnose sepsis.⁶

The authors calculated cutoff values for the PIM2 score (>14%), ferritin (>135 ng/mL), lactate (>1.7 mmol/L), and CRP (>6.7 mg/mL) using Youden’s Index, and found that all cutoff values were positively associated with mortality.⁴ Using these cutoffs, the researchers then calculated the positive predictive value (PPV) for mortality: the PIM2 score had a PPV of 38.5%; the combination of ferritin, lactate, and CRP had a PPV of 43%; and the combination of the PIM2 score with all three biomarkers had a PPV of 76%, which was far better than any single biomarker but still likely to be on shaky grounds for clinical use.⁴ Interestingly, negative predictive values (NPV) for PIM2 and all biomarkers alone and in combination were >95%. This prognostic approach was better at predicting who did not die as compared to those who did.

A logical question is how best to utilize these findings at the bedside. When interpreting these results and applying the findings, it is critical to consider the prevalence of mortality in any given PICU at baseline, as prevalence impacts PPV, NPV and accuracy (prevalence of mortality in this cohort was 8.5%). A higher prevalence of mortality would increase the PPV and decrease the NPV and accuracy (until prevalence reached >50%), while a lower prevalence would decrease the PPV and increase the NPV and accuracy compared to the reported values.⁷ Therefore, these values are highly context-dependent. Moreover, while many clinicians would like an unambiguous “yes” or “no” answer to the question, “Will my septic patient die?”, dichotomizing a continuous test, such as biomarker concentrations or the PIM2 score, significantly reduces its value. Furthermore, it creates a false construct whereby if prognostic biomarkers are highly specific for death, the implication is that the final outcome is inevitable and, more importantly, not modifiable. A more realistic approach would be to employ prognostic biomarkers to obtain a probability of death, which would serve as one consideration in clinical decision-making.

We applaud Tonial et al. for seeking to develop a pediatric sepsis prognostic approach that is relevant to PICUs throughout low- and middle-income countries. We recognize that identifying children at high-risk of mortality using non-specific clinical signs alone is fraught with difficulty and existing clinical scores intended for resource-limited settings have poor discriminative ability for in-hospital mortality.^8–13 Tonial and team expanded on an existing prognostic score (PIM2) by intentionally including prognostic markers that could be “useful in low-resource settings where human or financial resources” are limited and resource prioritization may need to be targeted to children most likely to survive.⁴

While we agree with Tonial et al.’s goal to develop a broadly applicable prognostic tool, including the PIM2 score is a questionable approach. The PIM2 score is not designed or intended to predict the probability of death real-time and, hence, we agree with the authors that its relevance in the “early prediction of patients at risk of deterioration or death” is problematic and its use “to estimate mortality in an individual patient is limited”.^4,5 The PIM2 score is useful in calculating the probability of mortality across large patient populations with variable disease pathologies and severities.⁵ However, the PIM2 score has not been adapted for resource-constrained settings; notably absent from the PIM2 “high risk diagnoses” are cerebral malaria, bacterial meningitis, and dengue shock syndrome.⁵ In addition, resource availability (such as the ability to measure base excess and PaO₂ and provide mechanical ventilation) contributes to the absolute PIM2 score.⁵ As an example, a patient in respiratory failure in a setting without ventilators has a lower predicted probability of death than a patient in a setting with ventilators, though the opposite is likely true. Finally, underlying Tonial et al.’s prognostic approach is the assumption that laboratory support is available to not only measure biomarker concentrations and leukocyte count, but also base excess and PaO₂; such laboratory capacity is far from universal. Prognostic approaches incorporating clinical pediatric scoring systems like the Lambaréné Organ Dysfunction (LOD), the Pediatric Early Death Index for Africa (PEDIA), or the Signs of Inflammation that Can Kill (SICK) scores and point-of-care diagnostics that do not rely on advanced medical or laboratory capacity for calculation will gain wider acceptance in low resource settings.^9–11

We agree with the authors that biomarker data combined with clinical data can improve prognostication. Biomarkers can help to distinguish severe from mild disease; CRP and lactate are used to risk-stratify patients, and the combination of an elevated CRP and ferritin greatly increased the odds of mortality in acutely ill children.^14–16 Biomarker performance and, hence, prognostic approaches are influenced by the epidemiology of disease and the patient population under study; because disease prevalence, comorbid conditions and illness severity can influence biomarker concentrations, generalizability to other populations is therefore limited.¹ For instance, Tonal et al. astutely pointed out that the higher prevalence of iron deficiency anemia in this cohort may have resulted in lower measured ferritin values (135 ng/mL) as compared to a previously published study (373 ng/mL).^4,17

The timing of biomarker sampling is also important: levels may vary depending on the natural trajectory of the disease and likely on the patient’s response to therapeutic interventions. For example, an elevated serum lactate level obtained before fluid resuscitation has a different clinical implication than an elevated level after fluid resuscitation; the former reflects tissue hypoperfusion, while the later reflects washout after perfusion is restored. Thus, the utility of biomarkers drawn throughout a 24–48 h window assumes that the clinical state is constant, invalidates the potential effect of clinical interventions, and renders interpretation difficult. Similar to the PIM2 scoring system, the ideal window for collecting prognostic biomarkers would be at the time of admission to the PICU (or shortly after).

Underlying Tonial et al.’s prognostic approach is the assumption that all sepsis is equal, regardless of etiology–bacterial, viral, parasitic, or fungal–, and that the pathogen is either not contributing or unimportant in determining the final outcome. Yet, even in this small cohort, we observe that 44.2% of survivors compared to 20% of non-survivors (p = 0.020) had a suspected viral infection, while 3.7% of survivors compared to 24% of non-survivors (p = 0.001) had a suspected fungal infection. Given the heterogeneity of sepsis, sub-group analyses testing prognostic power by etiology of sepsis may have been very informative.

Overall, these results are exciting and encouraging, but there are several important limitations that must be considered when interpreting the results. Firstly, there is potential for selection bias; 16% (N = 46) of eligible sepsis patients were not included in the final analyses due to missing measurements. In general, excluded patients were more likely to be readmitted within 72 h, have one or more complex chronic conditions, receive vasoactive drugs less often, have lower peak CRP levels, and die (16.1% of excluded patients died compared to 8.5% of included patients, p = 0.080). Given these differences, the concern is that, at worst, excluded patients may in fact have been the sicker patients with higher mortality, and, at best, the reported results for CRP are not representative of the underlying population. Secondly, this study was conducted in a private, tertiary care, teaching hospital in Brazil with PICU capabilities, and hence its relevance to other settings and patient populations is unknown. Thirdly, while the authors tested and reported the discriminatory power of the PIM2 score and individual biomarkers for mortality, one is left to wonder whether the combination of PIM2 and biomarkers would further amplify discriminatory power. Notably, the AUC is not impacted by prevalence, and thus would be a more context-independent value for reporting overall predictive ability than the PPV, NPV or accuracy.⁷ Finally, to develop a new prognostic score, as alluded to by the authors, the sample size would need to be significantly larger, potentially leveraging big data sources, and require external validation. Thus, these findings need to be tested in a larger cohort to gain wider acceptance.

Still, this is a well-executed and thoughtful study that adds to the mounting evidence that no single biomarker is sufficient to predict pediatric sepsis mortality; however, when biomarker data are combined with clinical data in the form of the PIM2 score, we observe superior prognostic performance. While the current findings are not ready for primetime and should be tested in a larger cohort, there are several other clinical applications for this prognostic approach that could be highly valuable once validated. First, a combined PIM2-biomarker prognostic tool could be used as a quality improvement metric or benchmark for comparing pediatric sepsis outcomes between institutions and over time.^1,3 By comparing actual mortality to predicted, clinicians could objectively assess whether implemented care processes improve mortality.¹⁸ Second, such a prognostic tool could aid with more specific stratification of clinical data compared to stratification by the PIM2 score or individual biomarkers alone for analyses of clinical trials and treatment outcomes.^1,3 Finally, a combined PIM2-biomarker tool could be used to inform patient enrollment in future clinical trials.¹⁹

Despite decades of research, tens of thousands of publications, and hundreds of millions of dollars, we have yet to identify and adopt into clinical practice a groundbreaking diagnostic approach, therapeutic agent or prognostic model for sepsis.²⁰ Predicting mortality due to sepsis is exceedingly difficult as it entails a complicated, dynamic interplay between host sociodemographic and biological factors, as well as access to timely and high-quality medical care, and pathogen characteristics and resistance patterns.²⁰ Given this complexity, a clinically relevant prognostic sepsis tool will likely need to be context- and population-specific and include inputs for patient risk factors (comorbidities, delay in presentation), host response (symptoms, signs, biomarkers), and suspected pathogen characteristics (organism, virulence). This important work by Tonial et al. raises many key questions regarding what should be included in the optimal pediatric sepsis prognostic tool. As we continue to unravel the complexities of sepsis, the testing of tools in various settings and patient populations should be a focus of future research.

Conflicts of interest

The authors declares no conflicts of interest. Dr. Kortz receives salary support from the National Institute of Allergy and Infectious Diseases, USA (K23AI1440.29).