Emergency services are subject to periods of excessive patient demand, which exceed their capacity to respond at an appropriate time. In the pediatric population, especially in Low and Middle-Income Countries (LMIC), an early assessment and approach reduce morbidity and mortality due to the high prevalence of time-sensitive medical emergencies.1 Over the years, several triage systems have been developed and improved to reduce long waiting queues for assistance, optimize flows, improve the quality of care for customers and the satisfaction of the multidisciplinary team.2 Currently, some validated pediatric triage systems are originating in different countries, such as the Manchester Triage System (MTS, United Kingdom), the Canadian Triage and Acuity Scale (CTAS, Canada), the Emergency Severity Index (ESI, United States of America) and the Classification of Risk in Pediatrics (CLARIPED, Brazil).3,4 However, the validity of these systems varies according to the patient load, the severity of illness case mix, available infrastructure, and available resources.5 The greater the socioeconomic and epidemiological diversity, the greater the difference in the performance of these tools. Although the most commonly used systems have shown good validity in the pediatric population in their countries of origin, evidence shows that this does not occur in LMIC without an adequate cross-cultural adaptation.1,2,4,5

The validity studies carried out in LMIC are limited and of low to moderate quality and include heterogeneous instruments, such as the Pediatric Early Warning Score – PEWS,1,6 the Emergency Triage Assessment and Treatment – ETAT,1,7-9 a three-level system advocated by WHO for low-income countries, and the South African Pediatric Screening Scale (pSATS), a four-level system validated in South Africa.10 Thus, there is a lack of evidence to support the use of the more traditional triage systems in the pediatric population in this group of countries.

The CLARIPED (Classification of Risk in Pediatrics), until now the only pediatric triage system developed in Brazil, was validated in its first version, for manual use, in a sample of 1416 patients from 0 to 15 years old, in the emergency department (ED) of a tertiary hospital in Rio de Janeiro, Brazil, in 2013. It proved to be a valid, reliable, and very safe tool, with high sensitivity (89%; 95% CI: 78%-95%) to identify very urgent patients, a low percentage of undertriage (7.4%), a consistent association between urgency categories and ED outcomes (hospitalization, number of diagnostic or therapeutic resources used, and length of stay in ED) and substantial interobserver agreement [kw2 = 0.75 (95% CI: 0.74-0.79)].11 Since then, a computerized version of CLARIPED has been developed, adding more operational efficiency to the triage process, being adopted in public and private pediatric services, of secondary and tertiary complexity, in southeastern Brazil. Also, some modifications were included based on the demands arising from the use of the system. Thus, this study aimed to evaluate the validity of the computerized version of the CLARIPED pediatric triage system.

The authors of the present study included 19,122 patients out of the 38,321 patients seen in the pediatric ED in the study period. The most frequent age groups were 1 to 4 years (42.8%) and 5 to 11 years (34.2%); periods with the highest number of visits were 12 to 18h (34.2%) and 18 to 0h (32.8%); the percentage of visits on each day of the week ranged from 13.5% (Friday) to 15.8% (Sunday); respiratory diseases accounted for 42% of the reasons for consultation, followed by general and nonspecific causes (40.7%). The distribution of the urgency categories was: 2.6% BLUE (no urgency), 58.5% GREEN (low urgency), 35.7% YELLOW (urgency), 3.2% ORANGE (high urgency), and 0.02% RED (emergency). The hospitalization rate was 1.8%. There was no relevant difference between the total population's characteristics and the studied population (Table 1).

The frequency of emergency marker outcomes, admission to the emergency observation room, hospitalization, and use of two or more resources significantly increased according to the gradually greater level of urgency assigned by the CLARIPED (p < 0.001). On the other hand, the frequency of outcomes such as being discharged home after medical consultation and the use of fewer than two resources decreased significantly with the increase in urgency level (p < 0.001) (Table 2). The median length of stay in the emergency department was 35 minutes (IQR 25;75), with a significant increase according to the increase in urgency level (p-value < 0.001).

The odds ratio (OR) for the use of two or more resources, from the most urgent levels (YELLOW, ORANGE, and RED) to the least urgent (BLUE and GREEN, the reference), adjusted for age, assistance time, and day of the week, was 2.85 (95% CI: 2.63-3.06); for hospitalization, it was 7.88 (95% CI: 5.35 11.6) (Table 3). The sensitivity and specificity of CLARIPED to identify the most urgent patients, based on the outcome “hospitalization” were, respectively, 0.82 (95% CI: 0.77-0.85) and 0.62 (95% CI: 0.61 0.62). The negative predictive value (NPV) was 0.99 (95% CI: 0.99-1.00), the overtriage rate was 4.28%, and the undertriage was 18.41% (Table 4).

The computerized CLARIPED proved to be a safe triage system, with good sensitivity to identify very urgent patients, a low rate of undertriage, and a strong association with clinical outcomes. The present study results are consistent with those found in a previous study of the first manual version of CLARIPED.11 These findings corroborate the tool's validity and support its implementation in pediatric emergency care with similar populations.

The distribution of levels of urgency in the study population is similar to that found in other validity studies on different triage systems, conducted in tertiary hospitals of developed countries, using consecutive samples, which describe 70 to 90% of the visits in the levels 3 or 4.12-17

The strong association between increased urgency levels and increased frequency of severity outcomes (use of two or more resources, admission to the emergency observation room, length of stay in the emergency department, and hospitalization) demonstrated the convergent construct validity of the CLARIPED system. These results are like those observed in other triage systems and superior in some respects. This study noted a decreasing and discriminative gradient in the hospitalization percentage (25% to 0.4%), from the highest to the lowest urgency level. Studies on PaedCTAS and ESI in their countries of origin have also shown discrimination gradients. Still, the percentages are higher at all levels of urgency, ranging from 100 to 60% at the most urgent patients to 2 to 1% at the least urgent.12-17 Even higher percentages of hospitalization were observed in Israel with PaedCTAS, reaching 12.32% at the least urgent level18). In Iran, ESI showed even higher percentages at levels 1 and 2 (the most urgent patients) but with low discrimination capacity among 3, 4, and 5 (less urgent patients).19 A 3.5% hospitalization percentage was found among less urgent patients in a study on MTS safety,20 while the SATS non-urgent category had a hospital admission rate of 4.7%.10 In all these studies, the percentages of hospitalization at low urgency levels are not negligible and may impair these instruments’ safety. It is noteworthy that this study found only 0.4% to 0.6% of hospitalizations at the less urgent levels (BLUE and GREEN, respectively). However, populations with more severe profiles, as suggested by the highest rates of general hospitalization, 8 to 8.6% in Canadian studies and 24.4% in Israel, compared to the 1.8% rate in the present study, may explain, in part, these differences in results. Different hospitalization policies, different application of urgency criteria in populations with varying degrees of severity, level of team understanding, and expertise in using the triage tool are also possible explanations. Considering all these aspects, the comparative analysis of the hospitalization percentages at different levels of urgency between the present study and similar studies from other systems shows that CLARIPED has a good discriminative capacity and a better safety profile. It also showed a very high negative predictive value and a percentage of hospitalization of less urgent patients (BLUE and GREEN) significantly below that found in other systems validity studies.

In this study, the odds of hospitalization in the most urgent categories (RED, ORANGE, and YELLOW) were almost eight times higher than in the less urgent categories (BLUE and GREEN). Studies evaluating CTAS and MTS13,20 found odds ratios of 4.94 and 2.5 to 3.5 (depending on the subgroup), respectively, comparing the high urgency levels (level 1 and 2) to level 3 (urgent), considered as the reference. In this study, the YELLOW category (urgent) showed admission rates to the emergency observation room and hospitalization rates considerably higher than the GREEN and BLUE categories (less urgent and non-urgent). The YELLOW and GREEN categories (levels 3 and 4) correspond to the two largest care groups in most pediatric emergency services globally, accounting for about 70 to 90% of the visits. In this study, the percentages of visits YELLOW and GREEN amounted to 94.15%. Thus, although the main focus of triage systems is given to the rapid identification of high-urgency patients (RED and ORANGE, levels 1 and 2), the safety of the triage process needs to have safe discrimination between urgent patients (YELLOW) and less urgent patients (GREEN), which determined the use of this cutoff for the OR, sensitivity, specificity, PPV, VPN, overtriage and undertriage rates in this study.

The ED length of stay showed a discriminative gradient between the highest and lowest levels of urgency, ranging from 260 to 18 minutes. In this calculation, the authors did not consider the time between the patient's arrival and the start of medical care, as the waiting time after triage is inversely proportional to the urgency and could unbalance the calculation of this outcome. Studies on PedCTAS12,13 and ESI v.415 measured the time from arrival to discharge from ED and found less discriminatory distributions.

There is no recommendation for safe limits of sensitivity, undertriage, and overtriage. However, as a priority, a triage tool should present good sensitivity and a low undertriage rate to ensure its safety.11 A high rate of overtriage means that many patients with low urgency were incorrectly classified as high urgency, which reduces the effectiveness of the triage tool, increasing the contingent of patients to be admitted and the waiting time for the high urgency patients.4 A high undertriage rate showed that many high-urgency patients were incorrectly classified as low-urgency, reducing the process's safety. In the present study, the CLARIPED tool showed high sensitivity to discriminate between most urgent patients (YELLOW, ORANGE, and RED) from less urgent patients (GREEN and BLUE), based on the hospitalization outcome. The undertriage and overtriage rates were moderate, and the VPN was high. Using the same cutoff point, the CLARIPED previous validity study showed more moderate values (sensitivity 0.74, specificity 0.62, undertriage rate 7.4%, and overtriage rate 59.1%), showing the progress of the current computerized version. The sensitivity to identify high urgency (levels 1 and 2) of other triage systems, using the outcome “admission to the intensive care unit” were: 0.67 to 0.93 with PaedCTAS,4 0.71 with MTS,4 and 0.91 with SATs.10

The present study has some limitations. Although the convenience sample was huge (n = 19122) and higher than most validity studies in pediatrics (median = 1496 visits),4 it comprised around 50% of the visits during the study period. It could not be representative of the general population. However, the comparative analysis of the study population with the general population, in the same period, did not show relevant differences regarding age, sex, day of the week, and service hours and also did not identify a systematic pattern of losses (hour or day of the week of service).

On the other hand, more important than the sample's representativeness would be ensuring enough visits at all levels of urgency. Few validity retrospective studies used sampling strategies to ensure a pre-determined number of patients at each level of urgency.2,15 Like most studies on pediatric emergency triage, the present study had very few visits at the RED level because this level is not frequent in most general pediatric EDs worldwide. This could be pointed out as another limitation. Nevertheless, the authors identified that almost 3% of the patients in this ED were seen directly by the medical team without going through the triage. They were immediately identified at the emergency reception with signs of high urgency (they would be RED and ORANGE levels in CLARIPED). The most frequent diagnoses in these patients were seizures, head trauma, burns, polytrauma, bruising wounds with active bleeding, acute laryngitis, anaphylaxis, and dehydration associated with constant vomiting. Based on this scenario, the authors believe that the present study had a lower proportion of visits at the ORANGE and RED levels than the real proportion at this ED. This loss would more likely attenuate the CLARIPED outcome gradients and sensitivity to detect more urgent patients.

Finally, one could argue that not blinding the emergency physicians to the assigned triage level could influence the outcomes. Blinding could only be achieved with a retrospective design. In this pragmatic prospective observational study, the ED professionals were unaware of the study, and the researchers did not interfere with the team's care routine. The interest of the study was to investigate what happens in real life when the authors believe that the physician's knowledge of the urgency level previously assigned does not influence his decision-making.

Another limitation of this and all other studies on the validity of triage systems would be the absence of a gold standard, which would allow an assessment of criterion validity, with greater methodological homogeneity and the possibility of a better comparison between the performance of the different tools. Some studies on MTS2,21,22 used a reference standard developed by specialists defining the “real” urgency, based on the combination of several outcomes, offering more objective and significant criteria for assessing validity. In contrast, there is no evidence of the adequacy and validity of this reference standard. Although the first CLARIPED validity study (manual version) also used an adapted reference standard, like the MTS studies, in the present study, it was decided to evaluate the construct validity through associations with each of the outcomes separately to make the results comparable to most other studies.

The results of this study emphasize the validity of CLARIPED, characterizing it as a Brazilian tool that performs well in pediatric emergency services, with an excellent discriminative capacity between the levels of urgency. It is the only pediatric risk classification system with validity evidence in the Brazilian pediatric population, supporting its implementation in pediatric emergency services in this country and others with similar characteristics.