Evaluation of national health‐care related infection criteria for epidemiological surveillance in neonatology

Abstract

Objective

to assess the use of the Brazilian criteria for reporting of hospital‐acquired infections (HAIs) in the neonatal unit and compare them with the criteria proposed by the National Healthcare Safety Network (NHSN).

Methods

this was a cross‐sectional study conducted from 2009 to 2011. It included neonates with HAI reporting by at least one of the criteria. Statistical analysis included calculation of incidence density of HAIs, distribution by weight, and by reporting criterion. Analysis of sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for the national criteria was performed considering the NHSN as the gold standard, with agreement assessed by kappa.

Results

a total of 882 newborns were followed, and 330 had at least one infection notified by at least one of the criteria. A total of 522 HAIs were reported, regardless of the criteria. An incidence density of 27.28 infections per 1,000 patient‐days was observed, and the main topographies were sepsis (58.3%), candidiasis (15.1%), and conjunctivitis (6.5%). A total of 489 (93.7%) were notified by both criteria, eight infections were notified only by the national criteria (six cases of necrotizing enterocolitis and two cases of conjunctivitis), and 25 cases of clinical sepsis were reported by NHSN criteria only. The sensitivity, specificity, PPV, and NPV were 95.1%, 98.6%, 98.4%, and 95.7%, respectively, for all topographies, and were 91.8%, 100%, 100%, and 96.3% for the analysis of sepsis. Kappa analysis showed an agreement of 96.9%.

Conclusion

there was a high rate of agreement between the criteria. The use of the national criteria facilitates the reporting of sepsis in newborns, and can help to improve the specificity and PPV.

Resumo

Objetivo

avaliar a aplicação dos critérios nacionais para notificação de infecções relacionadas à assistência à saúde (IRAS) em Unidade Neonatal e comparar com os critérios propostos pelo National Healthcare Safety Network (NHSN).

Métodos

estudo transversal realizado de 2009 a 2011. Forma incluídos os neonatos que apresentaram notificação de IRAS por pelo menos um dos critérios. Análise estatística incluiu cálculo de densidade de incidência de IRAS e distribuição por peso e por critério de notificação. Foi realizada análise da sensibilidade, especificidade, valor preditivo positivo (VPP) e valor preditivo negativo (VPN) para os critérios nacionais, considerando o NHSN como padrão‐ouro e a concordância avaliada pelo Kappa.

Resultados

foram acompanhados 882 neonatos, e 330 apresentaram pelo menos uma infecção notificada por, no mínimo, um dos critérios. Foram notificadas 522 IRAS, independentemente do critério. Observou‐se densidade de incidência de 27,28 infecções por 1.000 pacientes‐dia, e as principais topografias foram sepse (58,3%), monilíase (15,1%) e conjuntivite (6,5%). Um total de 489 (93,7%) notificações foram por ambos os critérios; oito infecções foram notificadas apenas pelo critério nacional (duas conjuntivites e seis enterocolites necrosantes); e 25 casos de sepse clínica foram notificadas apenas pelo NHSN. A sensibilidade, especificidade, VPP e VPN foram de 95,1%, 98,6%, 98,4%, 95,7%, respectivamente, para todas as topografias, e para análise de sepse foram 91,8%, 100%, 100% e 96,3%. O Kappa revelou concordância de 96,9%.

Conclusão

verificou‐se uma elevada concordância entre os critérios. A utilização dos critérios nacionais facilita a notificação da sepse em neonatologia e podem contribuir para melhorar a especificidade e VPP.

Introduction

Control of hospital‐acquired infections (HAIs) represents a challenge for health care workers and hospital administrators, as HAIs are severe infectious events, responsible for increased morbidity and mortality, costs, and social impact.^1,2

For the effective control of these diseases, the Hospital Infection Control Commissions (HICCs) must structure and maintain a system of epidemiological surveillance of HAIs, especially in critical sectors such as adult, pediatric, and neonatal intensive care units.^3–5

The organization of epidemiological surveillance of HAIs was started in the United States in 1986, when the National Nosocomial Infections Surveillance (NNIS), the surveillance system of the Centers for Disease Control (CDC), was organized and subdivided into components that include global monitoring, intensive care units (ICUs), a surgical component, and neonatal units.⁶

The method proposed by the NNIS was expanded and revised in 2005, and received the designation of the National Healthcare Safety Network (NHSN),⁷ with the establishment of criteria for infection associated with invasive devices (mechanical ventilation, central venous catheter, and urinary catheter), a surgical surveillance component, the introduction of a surveillance component in dialysis, and the use of the safety component protocol for the health care professional. The criteria are revised and new versions of NHSN are published periodically.⁷

In Brazil, the control of HAIs was regulated in 1983 by Decree 196/1983, revoked by Decrees 930/92 and 2616/1998.⁵ Since 1994, the Brazilian Ministry of Health has adopted the use of the method proposed by NNIS as a recommendation for epidemiological surveillance of HAIs in Brazilian hospitals, with the publication of the manual's translation. Thus, the NHSN protocols constitute the reference guide for epidemiological surveillance and reporting of HAIs in Brazilian hospitals.⁶ The NNIS method was officially adopted by the HICC of the Hospital das Clínicas da Universidade Federal de Minas Gerais (HC/UFMG), Belo Horizonte, Brazil, in 1994, controlling data collection, diagnosis, and reporting of HAIs.

Since 2008, the Brazilian National Health Surveillance Agency (Agência Nacional de Vigilância Sanitária ‐ ANVISA) has published manuals that address definitions, indicators, and strategies related to the prevention of major HAIs. The national criteria for surveillance of HAIs in neonatology were published in 2008 and updated in 2010, with the goal of standardizing epidemiological surveillance, targeting control actions, and allowing for the comparison of indicators between Brazilian institutions.⁸

However, due to the scarcity of published consolidated national data on the incidence of HAIs, especially in neonatology, the monitoring of these indicators in Brazilian institutions is usually performed through comparisons with U.S. data, obtained by NHSN reports, published annually. The use of epidemiological surveillance criteria defined by the CDC does not represent the current infection control situation in Brazilian institutions.^2,8

Considering the introduction of ANVISA criteria for epidemiological surveillance in neonatology and the service routine at HC/UFMG using the CDC/NHSN method since 1994, it is important to assess the progress and contributions of the national proposal.

Thus, this study aimed to assess the implementation of national criteria for the reporting of HAIs⁸ to the epidemiological surveillance, and to compare them with the international criteria proposed by the NHSN⁷ used in a neonatal unit of a referral university hospital.

Methods

This was a cross‐sectional study conducted from January 1, 2009 to December 31, 2011, at the Neonatal Progressive Care Unit (NPCU) of HC/UFMG.

HC/UFMG is a university, general public referral hospital that belongs to the municipal and state public healthcare system. Surveillance is performed, according to national legislation, in at‐risk patients, including newborns with low birth weight, those in mechanical ventilation, with central venous catheter, receiving antimicrobials for treatment of infections with systemic manifestations, and those submitted to surgical procedures.^5,7,8

The variables were collected through active search in records by a qualified and trained HICC professional and were recorded in the Epidemiology Center database of the aforementioned commission. The sample included all neonates at risk admitted at the NPCU, considering NHSN criteria⁷ and/or ANVISA criteria⁸ for HAI reporting.

Patient data and HAI notifications were entered in a computer software (Microsoft Excel, 2003, USA) spreadsheet and analyzed using the SPSS software (release 13.0, 2008, USA).

Statistical analysis included calculation of incidence density (ID) of infections (number of infections per 1,000 patient‐days), distribution of HAIs by weight range, and by surveillance criteria and notification. The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated for the criteria proposed by ANVISA⁸ considering the criteria established by NHSN⁷ as the gold standard for the total number of HAIs, and an exclusive analysis for reported sepsis cases, as they represent the majority of infections in which only one of the criteria was used for HAI notification.

The analysis of agreement between the methods was used to measure the kappa index, which is based on the number of concordant responses beyond what would be expected by chance.⁹

Kappa ranges from ‐1 (total absence of agreement) to +1 (complete agreement). Kappa values above 0.75 are considered as excellent agreement; between 0.4 and 0.75 as good, and below 0.4, the agreement is considered weak.

The project was approved by the Research Ethics Committee of HC/UFMG, according to COEP: ETIC 312/08. There were no procedures that could interfere with the healthcare activities and routines of the NPCU of HC/UFMG.

Results

A total of 882 patients were admitted at the NPCU, totaling 19,137 patient‐days, and 330 newborns had at least one episode of infection reported by at least one of the criteria.

A total of 522 episodes of HAIs were reported regardless of the criteria, with incidence density (ID) of 27.28 infections/1,000 patient‐days. Of the patients followed by the epidemiological surveillance (n = 882), 73.8% were in the weight range > 1,501 g; however, the ID and the percentage of patients with HAIs were more elevated in the weight range < 750 g when compared to the others (p < 0.001). The total number of patients, number of HAIs, and incidence density are presented by weight range in Table 1.

Table 1.

HAI stratification by birth weight, NPCU, HC/UFMG, 2009 to 2011.

Weight	No. of Patients at risk	No. of HAIs	Patients with HAIs (%)	IR of HAIs (/1,000 patient‐days)	p‐value	Risk (95% CI)
Up to 750 g	20	35	85.00	44.76	< 0.001	1.94 (1.32–2.84)
751 g to 1,000 g	65	88	73.85	28.23	0.16	1.22 (0.92–1.62)
1,001 g to 1,500 g	146	95	42.47	23.04	0.99	1.00 (0.76–1.32)
1,501 g to 2,500 g	348	198	34.77	30.37	0.02	1.32 (1.04–1.67)
> 2,500 g	303	106	26.73	23.06	‐	1
TOTAL	882	522	37.30	27.28

HAIs, hospital‐acquired infections; IR, incidence rate of HAIs; CI, confidence interval; NPCU, Neonatal Progressive Care Unit.

A total of 522 notifications of HAIs in 17 different topographies were filled, regardless of the surveillance criteria used. Sepsis was the most frequent infectious complication, with 305 cases (58.3%), of which 122 (40%) were early‐onset and 183 (60%) were late‐onset cases.

Clinical sepsis (n = 197) corresponded to 37.7% of total reported infections, and sepsis with laboratory confirmation (n = 108) to 20.6% of the total HAIs, regardless of the criteria used. After sepsis, the most frequent infections were skin infections (candidiasis), with 15.1%, followed by conjunctivitis (6.5%) and pneumonia with clinical diagnosis (6.1%).

Of the 522 notifications of HAIs, 489 (93.7%) were filled using both criteria, eight infections were reported only by the national criteria, and 25 only by NHSN criteria. The 25 cases of infection that employed only the NHSN criteria accounted for episodes of early‐ or late‐onset clinical sepsis, while those that used only the national criteria included two cases of conjunctivitis and six of necrotizing enterocolitis (Table 2).

Table 2.

HAI distribution by surveillance criterion, NPCU, HC/UFMG, 2009 to 2011.

Topography	Both	ANVISA	NHSN	Total	%
Sepsis
Sepsis with laboratory confirmation – late onset	102	0	0	102	19.5
Clinical sepsis – early	94	0	22	116	22.2
Clinical sepsis – late‐onset	78	0	03	81	15.5
Sepsis with laboratory confirmation – early	6	0	0	6	1.1
Subtotal – sepsis	280	0	25	305	58.3
Skin infection – candidiasis	79	0	0	79	15.1
Conjunctivitis	32	2	0	34	6.5
Pneumonia with clinical diagnosis	32	0	0	32	6.1
Meningitis or ventriculitis	21	0	0	21	4.0
Necrotizing enterocolitis	11	6	0	17	3.3
Symptomatic urinary tract infection	10	0	0	10	1.9
Oral cavity infection – candidiasis	5	0	0	5	1.0
Skin infection – except candidiasis	3	0	0	3	0.6
Pneumonia with laboratory confirmation	3	0	0	3	0.6
Intra‐abdominal infection	3	0	0	3	0.6
Skin infection (undefined)	3	0	0	3	0.6
Tracheal and bronchial infection	2	0	0	2	0.4
Artery or vein infection	2	0	0	2	0.4
Endocarditis	1	0	0	1	0.2
Asymptomatic bacteriuria	1	0	0	1	0.2
Other urinary tract infection	1	0	0	1	0.2
TOTAL	489	8	25	522	100.0

HAIs, hospital‐acquired infections; ANVISA, Brazilian National Health Surveillance Agency; NHSN, National Healthcare Safety Network; NPCU, Neonatal Progressive Care Unit.

The results of the analysis of national criteria for HAI notification in all topographies show high sensitivity, specificity, PPV, and NPV, in addition to excellent agreement, and are shown in Table 3.

Table 3.

Comparison between the national and the NHSN criteria in the reporting of HAIs in all topographies, NPCU, HC/UFMG, 2009 to 2011.

Test	Index	95% CI
Sensitivity	95.1%	92.8 ‐ 96.8
Specificity	98.6%	97.1 ‐ 99.3
PPV	98.4%	96.7 ‐ 99.2
NPV	95.7%	93.6 ‐ 97.1
Kappa	96.9%	92.6 ‐ 97.4

HAIs, hospital‐acquired infections; PPV, positive predictive value; NPV, negative predictive value; CI, confidence interval; NPCU, Neonatal Progressive Care Unit.

It was observed that the sensitivity of the national criteria for notification of sepsis had a relatively lower proportion of notifications, but higher specificity when compared to notification by the NHSN criteria, when the criteria were employed for all topographies. The sensitivity, specificity, PPV, NPV, and Kappa for sepsis notifications using the national criteria are shown in Table 4.

Table 4.

Comparison between the national and the NHSN criteria for the reporting of sepsis, NPCU, HC/UFMG, 2009 to 2011.

Test for sepsis	Index	95% CI
Sensitivity	91.8%	88.0–94.5
Specificity	100.0%	99.3–100.0
PPV	100.0%	98.3–100.0
NPV	96.3%	94.5–97.6
Kappa	97.3%	96.1–98.6

HAIs, hospital‐acquired infections; PPV, positive predictive value; NPV, negative predictive value; CI, confidence interval; NPCU, Neonatal Progressive Care Unit.

Discussion

The incidence density of HAIs in this study, considering the two surveillance criteria, was 27.28 infections per 1,000 patient‐days, and was higher in the weight range below 750 g, when compared to the others. The results reinforce the literature data indicating that low birth weight is a risk factor for infections, reflecting the greater need for invasive procedures and prolonged hospitalization in newborns weighing less than 750 g.^10,11

Topographies with the highest incidence of reported HAIs were sepsis (58.3%), skin infections (15.1%), and conjunctivitis (6.5%). These results were similar to that of a study conducted in a neonatal intensive care unit (NICU) in Belo Horizonte, state of Minas Gerais, Brazil, which identified sepsis as the main infectious complication in newborns (46%), followed by conjunctivitis (12.1%), and skin infections (9.5%).¹² Another study, carried out for five years in Germany, identified sepsis (78%), ventilator‐associated pneumonia (11%), and necrotizing enterocolitis (10%) as the major sites of HAI in neonates weighing less than 1,500 g.¹³

Among all HAI topographies, clinical sepsis was the neonatal infectious complication with the highest number of notifications, accounting for 37.7% of the total, regardless of the criteria used for notification. The highest incidence of clinical sepsis in neonates can be explained by the difficulty of laboratory confirmation in this population. Confirmation of sepsis is attained by the growth of pathogens in blood culture, or in case of a commensal microorganism, growth in two blood cultures, which is considered the gold standard for the diagnosis of bloodstream infections. However, this does not always occur in newborns, due to several factors, such as insufficient blood sample and the administration of antibiotics to the mother before delivery.^14–17

Some authors consider that certain factors complicate the diagnosis of sepsis in the neonatal period, due to the nonspecific clinical signs, which may be mistaken as some characteristic conditions of the period, such as transient tachypnea and apnea of prematurity. Clinical abnormalities in newborns are considered as having low predictive value for sepsis, and other parameters are necessary for the confirmation of infection. Therefore, the diagnosis of neonatal sepsis is considered difficult from both the clinical and laboratory perspective. Moreover, negative blood culture results do not imply in the absence of sepsis in the neonate, as this test has low sensitivity.^16,17

Some studies have shown the variability and low predictive value of clinical signs of infection in neonates. A Brazilian study conducted in a NICU, which followed 55 neonates with suspected infection, demonstrated that clinical signs of neonatal sepsis may vary, with manifestations such as hypothermia and hyperthermia (33%), tachycardia (11%), and hypoactivity (94%).¹⁸

Another study, which analyzed 220 episodes of clinical sepsis in a NICU in India, identified hypoactivity (46.4%), apnea (33.6%), and gastric stasis (33.6%) as the most frequent clinical signs. With the analysis of the clinical score for sepsis proposed in this study, it was observed that clinical signs are reduced from the time of suspected infection up to the following 24 hours, with a sensitivity of 90% (0 h) and 75% (24 h), and low PPV of 30.3% (0 h) and 41.7% (24 h).¹⁹ This result is similar to another study performed in Bangladesh²⁰ with analysis of clinical predictors of sepsis in premature neonates with positive cultures (n = 105), in which the application of the clinical score showed low sensitivity of 56.6%, specificity of 52.1%, and PPV of 78.1%. Another study, by Singh et al.,²¹ analyzed 16 clinical signs of sepsis and identified a sensitivity of 47%, 40%, and 30% for apnea, lethargy, and tachycardia, respectively, as the best results.

As signs and symptoms of sepsis in the newborn are nonspecific, it is apparent that the combination of clinical signs and laboratory abnormalities may contribute to neonatal sepsis diagnosis and notification. In 1988, Rodwell et al.²² proposed the use of the hematologic score to predict sepsis in neonates with a compatible picture, with standardized parameters for global leukocytes, total neutrophils, immature neutrophils, immature/mature neutrophil ratio, as well as platelet count and degenerative alterations.

When applying the score, it was observed that 96% of neonates with proven sepsis and 100% of those with probable sepsis had scores altered by greater than three‐fold. In a study carried out in NICUs in Belgium, a score with clinical and laboratory variables (CRP, neutrophil fraction, thrombocytopenia, fever, and prolonged exposure to parenteral nutrition) was considered useful in identifying newborns with signs of sepsis, with a sensitivity of 95%, specificity of 43%, PPV of 54%, and NPV of 93%.²³

Another study, by Sankar et al.,²⁴ in which the hematological score was analyzed, including total leukocytes, total neutrophils, immature neutrophil fraction, and CRP, demonstrated a sensitivity of 93% to 100% and specificity of 83% in the presence of two abnormal parameters for neonates with suspected sepsis. CRP has also been considered useful for the monitoring of neonates with signs of sepsis due to high NPV marker. The decrease in this marker, together with clinical improvement, has been used as a parameter for the discontinuation of antibiotic therapy.¹⁸ A prospective study performed for over three years in a NICU in Rio de Janeiro showed altered leukogram results in 64.3% of cases of probable sepsis with appropriate compatible picture, and the C‐reactive protein (CRP) measurement was elevated in patients with proven sepsis.¹⁷

In the present study, the correlation between national criteria regarding NHSN for HAI reporting and sepsis in newborns was considered high (kappa = 96.9% and 97.3%, respectively). However, the HAI notifications that showed the biggest difference between the methods were of clinical sepsis. It is noteworthy that the notification of clinical sepsis is still recommended by ANVISA,⁸ although it has been excluded from the NHSN⁷ protocol version from July 2011, when data collection for this study had already been finished.

All 25 reports of clinical sepsis that did not meet the ANVISA criteria and thus were solely reported based on NHSN criteria occurred due to the requirement of a larger number of parameters, including the need for laboratory abnormalities: Complete Blood Count (CBC) with three or more abnormal parameters and/or altered CRP measurement, which may contribute to obtain a more reliable notification. ANVISA also adds a larger number of clinical signs for the reporting of clinical sepsis, such as food intolerance, hemodynamic instability, worsening of breathing, glucose intolerance, and lethargy.⁸

The proposal made by ANVISA⁸ is close to the current clinical management and literature,^18,23,24 which tend to consider laboratory findings for the diagnosis of sepsis and the start or maintenance of antibiotic therapy.

In a study previously performed in the same service, which exclusively used the national criteria for the reporting of infections in neonatology, it was observed that the main indicators of HAIs and topographies were similar to other previous studies based on the international NHSN criteria.²⁵

In a study conducted for two years in the NICU of Hospital Universitário do Rio de Janeiro, the national HAI criteria⁸ were employed, aiming to improve the diagnosis and reporting of sepsis in low‐birth weight neonates. This study showed that, with the implementation of clinical and hematological criteria, as well as CRP measurements proposed by ANVISA, there was a reduction in the diagnosis and reporting of early‐onset sepsis, with a consequent decrease in the use of antimicrobials.²⁶ Kaufman comments and describes the importance of implementing protocols containing criteria for the diagnosis of early‐ and late‐onset neonatal sepsis, aiming at the discontinuation of antimicrobial therapy based on hematologic data and CRP measurement.²⁷

Therefore, it is considered that the lower number of notifications by the national criteria that is observed in relation to NHSN criteria, with a sensitivity of 95.1%, might occur due to the addition of laboratory parameters required for notifications of sepsis by ANVISA's criterion, with sensitivity of 91.8% when considering only the reports of bloodstream infections.

The larger number of laboratory parameters for the reporting of clinical sepsis helps to exclude possible non‐infectious conditions, increasing the specificity and PPV, as observed in this study, with a specificity of 98.6% for HAI reporting in all topographies, and 100% when only sepsis is analyzed, in addition to the high NPV of 98.4% and 96.3%, respectively.

Necrotizing enterocolitis was the second topography in which there was a greater difference between the criteria for HAI surveillance and reporting. Of the 17 notifications of enterocolitis, six met only the ANVISA criteria and 11 met both criteria. It is observed that, for notifications of necrotizing enterocolitis, differently from the NHSN criteria, ANVISA does not require the need for the presence of blood in the stool as a condition for infection reporting. In this case, there is an increase in the number of enterocolitis notifications, which increases the sensitivity of the national criterion.

The signs of enterocolitis in neonates show a wide variation in terms of presentation, from the non‐specific, such as mild abdominal distension, increased gastric residuals, lethargy, and temperature instability, to the most evident cases, with abdominal distension, rectal bleeding, and presence of blood in the stool, which does not always occur.^28,29

Some authors consider that the absence of predetermined CDC criteria for the characterization of infections exclusively in the neonatal period and the rapid evolution of these processes in newborns hinder the classification of some infections. This becomes evident in the classification of sepsis, which shows a wide variation among studies in the area.^2,17,18,30

In addition to all these difficulties for the diagnosis of sepsis in this age group, the Neonatal Unit is considered a critical unit with priority and mandatory surveillance, which demands specialized care for critically‐ill patients and those at greater risk of exposure. The prevention and control of HAIs in this population represent a challenge for hospitals, and to define interventions, it is necessary to maintain an active surveillance system for infections.^9,10,12

Conclusion

There was a high correlation between the use of national criteria proposed by ANVISA in relation to the NHSN method for reporting of HAIs in neonatal units. The use of national criteria for epidemiological surveillance of HAIs in these units provides tools to help the reporting of infections, as the association of laboratory data to clinical data proposed by ANVISA may help to improve the specificity and PPV of the diagnosis and reporting of neonatal infections, considering the low sensitivity and predictive value of clinical signs in the presence of suspected infection.

Conflicts of interest

The authors declare no conflicts of interest.