The Brazilian Portuguese version of the Pediatric Trigger Toolkit is applicable to measure the occurrence of adverse drug events in Brazilian pediatric inpatients

Abstract

Objective

To describe the process of translation, adaptation, and validation of the Brazilian Portuguese version of the Pediatric Trigger Toolkit: Measuring Adverse Drug Events in the Children's Hospital.

Methods

The validation process for the research instrument was carried out in five steps: (I) translation from English into Brazilian Portuguese by two independent translators; (II) synthesis of the translated versions; (III) adaptation of the triggers to the Brazilian context using a modified Delphi method carried out in two rounds; (IV) application of the research instrument in the review of 240 hospitalizations during 2014; and (V) back-translation from Brazilian Portuguese to English.

Results

The translation step resulted in two versions, which were then synthesized to obtain equivalence within the Brazilian cultural context. The process of adapting the research instrument allowed the inclusion and adaptation of some triggers, as well as the exclusion of others, based on the consensus reached among experts during the Delphi step. The positive predictive value of the research instrument was 13.51%, and the antihistamine trigger presented the highest positive predictive value.

Conclusion

The research instrument can be used to track the occurrence of adverse drug events in pediatric patients in Brazilian hospitals. Despite the adjustments made, triggers that were not identified or that had low positive predictive value can suggest the need for additional changes. However, more studies are needed before modifying or removing a trigger.

Resumo

Objetivo

Descrever o processo de tradução, adaptação e validação para o português do Pediatric Trigger Toolkit: Measuring Adverse Drug Events in the Children's Hospital.

Métodos

O processo de validação do instrumento de pesquisa foi realizado em 5 etapas: (I) tradução do inglês para o português por dois tradutores independentes; (II) síntese das versões traduzidas; (III) adaptação dos rastreadores para realidade brasileira por meio do método Delphi modificado, em duas rodadas; (IV) aplicação do instrumento de pesquisa na revisão de 240 internações de 2014; e (V) retrotradução do português para o inglês.

Resultados

A etapa de tradução resultou em duas versões traduzidas, que foram sintetizadas de forma a obter equivalência ao contexto cultural brasileiro. O processo de adaptação do instrumento de pesquisa possibilitou a inclusão e adaptação dos rastreadores, bem como a exclusão de rastreadores segundo consenso obtido entre os especialistas na etapa Delphi. O valor preditivo positivo do instrumento de pesquisa foi de 13,51% e o rastreador anti-histamínico foi o que apresentou maior valor preditivo positivo.

Conclusão

O instrumento de pesquisa pode ser utilizado para rastrear a ocorrência de eventos adversos a medicamentos em pacientes pediátricos nos hospitais brasileiros. Apesar das adaptações realizadas, rastreadores que não foram identificados ou com baixo valor preditivo positivo podem indicar, ainda, a necessidade de modificações. Entretanto, mais estudos são necessários antes de modificar ou excluir algum rastreador.

Introduction

Patient safety in health care has become increasingly prominent worldwide in recent years, especially after the publication of the report To Err is Human.¹ Nevertheless, consistent identification of patient harms due to adverse events (AE) remains a challenge for healthcare institutions.² In order to assist in AE monitoring and to provide healthcare professionals with the proper knowledge about potential AE that can occur in health systems, different identification methods have been developed.³

In AE detection, the utility of retrospective and manual review of medical records using triggers has been demonstrated.^4,5 A trigger is defined as data or a clue in the patient record that alerts the reviewer to the possibility of harm. Triggers related to medication use, for example, include abrupt interruption of medication use, prescription of an antagonist, or an abnormal laboratory test result.⁶

Although most research instruments that use triggers are focused on adults,⁵ efforts have been made to develop specific research instruments to be used in children.² Thus, there are triggers for use in neonatology,⁷ pediatric critical care,⁸ and pediatric general care,^3,9–11 as well as specific types of harms, such as those involving medication use.¹²

Among all research instruments using triggers to identify adverse drug events (ADEs), the one developed by Takata et al.¹² includes a list of triggers with specifications, instructions for review of randomly selected medical records, and reviewer training instructions. That study found the occurrence of 11.1 ADE per 100 pediatric patients and 15.7 ADE per 1000 patient-days.

This study aimed to describe the steps of translation into Brazilian Portuguese, adaptation, and validation of the Pediatric Trigger Toolkit: Measuring Adverse Drug Events in the Children's Hospital for use in Brazilian hospitals.

Methods

The authors obtained permission to translate the research instrument into Brazilian Portuguese and adapt its use for the purpose of the research aims.

The validation process was based on the recommendations proposed by Beaton et al.¹³ and Mendes et al.¹⁴ The research was carried out according to the steps described in Fig. 1.

Figure 1.

Schematic representation of the study's development steps. aPTT, activated partial thromboplastin time; INR, international normalized ratio.

The first step consisted of translation from English into Brazilian Portuguese of the research instrument. Translation was performed independently by two translators who were native speakers of Brazilian Portuguese. The first is a professional translator, without technical knowledge in AE, and the second is a healthcare professional with basic knowledge of the area under study.

In the second step, a team of six healthcare professionals synthesized both translated versions of the research instrument by comparing them with the original version. This stage aimed to achieve semantic equivalence (idiomatic, cultural, and conceptual) between both languages.^13,14

The third step was carried out to adapt the ADE triggers to the Brazilian context by using an expert panel in a modified Delphi approach with two rounds.¹⁵ For the first round, experts were selected by searching the Lattes Platform (a database of Brazilian researchers’ curriculum), using the keywords “adverse events,” “adverse drug events,” “pharmacovigilance,” “trigger,” and “pediatrics,” as well as recommendations from professionals with experience on AE. Selection criteria for the Delphi panel were healthcare professionals (nurses, pharmacists and physicians) involved in research activities on pharmacoepidemiology; coordinators and researchers in projects related to pharmacovigilance, ADE, and use of triggers; and members of patient safety groups and hospital pharmacovigilance committees.

Experts received an invitation to join the study by e-mail and instructions for completing the questionnaire. The form was composed of 24 ADE triggers: 13 from the original instrument without modifications, two from the original instrument with modifications, and nine inserted by the research team based on other studies.^6,10,11 Other triggers, in addition to those from the original instrument, were included to increase the scope of ADE identification.

Each trigger was assessed according to three criteria: (i) if the trigger was a good ADE indicator, (ii) if the trigger was applicable to the reality in Brazilian hospitals, and (iii) if the trigger was described in clear language and appropriate terms, using a Likert scale ranging from 1 (extremely negative evaluation) to 5 (extremely positive evaluation). Results were calculated as means, so that triggers with a mean score greater than or equal to 4 were approved, less than 4 but greater than or equal to 3 were evaluated, and less than 3 were not included.¹⁴ Experts were asked to make suggestions on triggers according to their analysis.

Experts who answered the first-round questionnaire were invited to join the second round. Based on first round responses and further analysis by the research team, a new instrument consisting of 17 triggers was sent to the experts. Each trigger was considered as adequate or inadequate according to the following criteria: Are the trigger and its description well defined? Is the trigger to be investigated from patient records well described? Trigger suitability was calculated in terms of percentage, with a cut-off point of 75% of acceptability among the experts.

The fourth step consisted of testing the research instrument by reviewing patient records. The study was undertaken in a pediatric department of a teaching and tertiary care hospital with approximately 290 beds in Brazil. Medical records of patients aged 0–18 years who remained hospitalized for a period longer than 48 h were randomly selected. Patients with only non-medication orders were excluded.

A pharmacist with clinical experience and a pharmacy student retrospectively reviewed 20 patient records from each month during 2014, totaling 240 admissions. Several training sessions with the research instrument were performed as a pretest before the patient record review. After reviewing the medical records, ADE were presented and discussed with healthcare professional (nurses, pharmacists, and physicians) for data evaluation. The team intensively discussed each possible ADE based on the drug's properties, the patient's clinical condition, and the time course of the ADE occurrence.

Data validated during consensus meetings were analyzed using Epi Info 3.5.4 (Epi Info^®, Centers for Disease Control and Prevention, GA, USA) and Stata 12.0 (StataCorp, 2011. Stata Statistical Software: version 12. College Station, TX, USA). Trigger performance was evaluated based on the ability to identify ADE, and the analysis established this in terms of positive predictive value (PPV) with 95% confidence intervals (CI) of the exact binomial distribution. Each trigger's PPV was calculated as the number of times each trigger identified an ADE divided by the number of times each trigger was found.

The fifth step consisted of back-translation from Brazilian Portuguese to English. This was performed by a group of professional translators who had no knowledge of the original document. The back-translated version was compared to the original version and to the Brazilian Portuguese version, to assess the equivalence of the instruments in both languages and to identify terms that still needed adjustment for equivalence. Finally, the modified back-translated version was sent to the creators of the original version for approval.

This research was conducted with approval from the Research Ethics Committee of the Hospital, protocol number 1007134/2015.

Results

Translation and synthesis of the translated versions

The two translated versions of the research instrument were compared with each other, and some disagreements were observed. During the synthesis process, the team debated the best expressions to use in Brazilian Portuguese and adopted them according to the Brazilian cultural context. For example, the term trigger was translated as gatilho ou desencadeador (trigger) and as indicador (trigger) by the translators. The synthesis team suggested the term rastreador (trigger), because it was most commonly found in the national literature to refer to the word trigger.

Research instrument adaptation

Delphi panel

The authors invited 58 experts, of whom 22 replied to the invitation and answered the first questionnaire. First-round participants included four nurses (31.8%), 14 pharmacists (63.6%), and one doctor (4.5%); these participants had professional experience in the Southeast (55%), South (25%), Midwest (15%), and Northeast (5%) of Brazil. A total of 8.7% of participants had an undergraduate degree, 4.3% broad expertise, 34.8% a master's degree, and 52.2% a doctorate.

In the second round, the 22 experts who responded to the first questionnaire were asked to answer a second questionnaire on triggers. Ten experts answered the second questionnaire.

Delphi method rounds

During the first round, the triggers antihistamine, flumazenil, naloxone, rising serum creatinine (>2× baseline), and abrupt medication stop achieved a mean score greater than or equal to 4 in the experts’ evaluation. Therefore, they were approved for inclusion in the research instrument. Suggestions made by experts to improve these item descriptions were considered. To the trigger antihistamine, drug names from the therapeutic class were added; to naloxone, signs and symptoms of overdose caused by opioids were included; to rising serum creatinine (>2× baseline), drugs that can lead to renal failure were described, and to abrupt medication stop, criteria to consider it as a trigger were defined.

All other triggers that had a mean score higher than 3 but less than 4 in the experts’ assessment were reviewed by the research team. Vitamin K; anti-emetics; activated partial thromboplastin time (aPTT) > 100 s or international normalized ratio (INR) > 5; over-sedation, lethargy, falls, or hypotension; glucose > 150 mg/dL; laxatives or stool softeners; administration of glucagon or glucose ≥10%; abrupt drop in hemoglobin or haematocrit (>25%); and glucose < 50 mg/dL were approved after consideration by the research team. To the above mentioned triggers, the following information was added: associated drugs from the therapeutic classes, signs and symptoms, and changes in laboratory parameters.

The triggers ion exchange resin and electrolyte abnormalities were approved after modification. For the first one, only use of calcium polystyrene sulfonate was added in parentheses, which is used in Brazil, and the second one was divided into two separate triggers, one related to abnormalities in levels of potassium and the other regarding sodium levels.

The triggers partial thromboplastin time >100 s, rising serum creatinine, rising urea or creatinine (>2× baseline), rash, and hyperkalemia were excluded, as they had already been covered in other triggers. The triggers called codes, drug level out of range, and Clostridium difficile positive stool were excluded, as they were considered unlikely to be applicable in many Brazilian hospitals.

In the second round, the experts assessed 17 items. All triggers were considered adequate by at least 80% of them, and no item was excluded.

Research instrument application

A total of 407 triggers were found among 240 selected admissions, with 65% (n = 156) of hospitalizations having at least one trigger and 59.58% of them having between one and five (Table 1). The highest number of triggers found in a medical record was 14, and the mean number of triggers per patient was 1.70.

Table 1.

Distribution of adverse drug events triggers according to the number of triggers identified in each hospitalization evaluated.

Number of triggers	Number of hospitalizations	Proportion (%)
0	84	35.00
1–5	143	59.58
6–10	11	4.58
11–14	2	1.67

Among the triggers that identified an ADE, PPV ranged from 3.03% (95% CI 0.08–5.76%) to 38.24% (95% CI 22.17–56.44%) for use of laxatives or stool softeners and antihistamine, respectively. The PPV of the research instrument was 13.51% (95% CI 10.35–17.22%; Table 2). Triggers with better performance were use of antihistamine, ion exchange resin, and abrupt medication stop. Table 3 shows examples of identified ADE.

Table 2.

Frequency and positive predictive value of adverse drug event triggers in pediatric patients.

Trigger	Number of triggers	Number of trigger that identified ADE	PPV (%) (95% CI)
Antihistamine (dexchlorpheniramine, loratadine, promethazine)	34	13	38.24 (22.17–56.44)
Vitamin K (phytonadione)	18	0	0
Flumazenil	^a	–	–
Anti-emetics (bromopride, metoclopramide, and ondansetron)	106	7	6.60 (2.70–13.13)
Naloxone	^a	–	–
Ion exchange resin (calcium polystyrene sulfonate)	4	1	25.00 (0.63–80.59)
Laxatives or stool softeners (lactulose, bisacodyl, mannitol, sodium docusate, mineral oil)	33	1	3.03 (0.08–5.76)
Administration of glucagon or glucose ≥10%	22	3	13.64 (2.91–34.91)
Activated partial thromboplastin time (aPTT > 100 s) or international normalized ratio (INR > 5)	5	1	20.00 (0.51–71.64)
Abrupt drop in hemoglobin or hematocrit (>25%)	8	0	0
Rising serum creatinine (>2× baseline)	7	1	14.29 (0.36–57.87)
Potassium: K^+ < 3.0 mmol/L or K^+ > 6.0 mmol/L	19	2	10.54 (1.30–33.14)
Sodium: Na^+ < 130 mmol/L or Na^+ > 150 mmol/L	11	2	18.18 (2.28–51.78)
Glucose > 150 mg/dL (hyperglycemia)	18	2	11.11 (1.37–34.71)
Glucose < 50 mg/dL (hypoglycemia)	15	1	6.67 (0.17–31.95)
Over-sedation, lethargy, falls, or hypotension	31	5	16.13 (5.54–33.73)
Abrupt medication stop	76	16	21.05 (12.54–31.92)
Total	407	55	13.51 (10.35–17.22)

ADE, adverse drug event; PPV, positive predictive value; CI, confidence interval.

a

Triggers were not identified in study sample.

Table 3.

Description of adverse drug events identified by triggers in pediatric inpatients.

Trigger	Example
Abrupt medication stop	6-Year-old patient, female, developed tremor, tachypnea, fever and chest pain after albumin use.
Antihistamine; laxatives or stool softeners	9-Year-old patient, female, developed two adverse drug events: allergic reaction after cefepime and constipation after continuous tramadol use.

Back-translation

The comparison between all versions (back-translated, original, and Brazilian Portuguese) prompted modifications in some items in the Brazilian Portuguese version. For example, the expression medical errors translated as erros médicos (medical errors) was substituted with the expression erro de medicação (medication errors) to encompass concepts regarding patient safety culture.

Discussion

The search for research instruments to aid in the assessment of ADE occurrence is one of the strategies to enhance patient safety as this health care movement grows.¹⁶ The development and adaptation of ADE tracking methodologies in several countries^{3,11,14,17,18} highlights the need for increased knowledge about AE. This study was one of the first to adapt ADE triggers specifically for Brazilian pediatric units.

The adaptation process considers a research instrument to be capable of measuring that for which it was developed; aspects regarding the cultural context need to be included, such as terminology, taxonomy, and ambiguities. The adequacy of the instrument's content in relation to the Brazilian context was contemplated in terms of how much would be gained from the cultural approach and how much would be lost to generalizability and comparability. The meetings to synthesize the two translated versions were important to define which terms and expressions could offer the greatest clarity and appropriateness to the Brazilian context and remain as close as possible to the original meaning.

The Delphi method is internationally recognized because it gives equal value to each participant and allows experts from different locations to reach consensus more easily. There is no standard for establishing the validity of the results obtained through the Delphi method, but it is assumed to be valid because of its multiple rounds and the fact that a consensus is reached among a group of people who have knowledge about a topic.¹⁵

Brazil is a large country with specific regional needs; it was possible for professionals from various parts of the country to consider characteristics of triggers from different locations. However, it must be remembered that the Delphi panellists’ responses were given based on their experience with ADE; in some cases, this may refer to a specific type of ADE or specific trigger.

All triggers generated by consensus among the experts at the end of the second round evaluations are likely to lead to identification of ADE that causes patient harm. The definition and implementation of an expert panel consensus can vary from study to study, depending on research objectives.¹⁹ A third round of assessment was considered unnecessary, as an acceptable level of consensus had been reached in the second round.

In the case of vitamin K, since its use is indicated in newborns for prophylaxis against bleeding,²⁰ by restricting the definition of the trigger use of vitamin K to non-neonates, it was possible to reduce the time for record review, since there were 12 cases of neonates receiving vitamin K for whom there was no need for a detailed medical record review.

Among the triggers that identified harm, the PPV ranged from 3.03% to 38.24%, and the research instrument's PPV was 13.51%. The PPV of triggers varies in relation to ADE identification from study to study¹⁰; for example, Takata et al.¹² found a PPV of 3.73%. Consistent with previous studies,^7,12,21 trigger PPV varied widely, and the PPV of the research instrument was relatively low in most of the cases, which means that many triggers are detected, but few of them are related to ADE.

PPV is influenced by multiple factors. First, the number of triggers used, their definitions, and how to consider several events related to the same trigger, interfere with the method's ability to identify an ADE. Second, hospital and population characteristics should be considered, as research carried out in different regions can lead to different results, even if the methodology employed is the same.^3,22

For instance, some triggers are easily identified manually from prescriptions, such as use of antihistamines or anti-emetics. Other triggers, such as over-sedation, lethargy, falls or hypotension, and abrupt medication stop are more complex and require more judgment ability to be identified.²³

Moreover, the results may be underestimated due to problems related to availability of information in medical records. Health professionals often do not report the occurrence of AE due to guilt, fear of punishment, and fear of retaliation. Therefore, encouraging a safety culture is important to promote patient safety and knowledge about AE in healthcare institutions.²⁴

Of the 17 triggers used in the research instrument, 13 led to ADE identification, which indicates the usefulness of the method.²³ Thus, screening for triggers is an important method to use in the tracking of ADE. The research instrument can be improved by removing triggers of lower clinical and epidemiological relevance in the site survey or by changing parameters of triggers that had a low frequency and very low PPV, decreasing the number of false positives.²⁵

Using this methodology, all information collected is influenced by the reviewers’ skills and their experience with the method.³ Thus, the effectiveness of the methodology was ensured by the application of a pretest and standardization of the medical record review, definitions, and manual instructions, aiming to establish a reliable and consistent strategy for ADE research.^6,7,26

The Brazilian Portuguese version of the research instrument can be considered valid and applicable to measure the occurrence of ADE in pediatric inpatients. This study showed that the research instrument can be used in Brazilian hospitals for tracking ADE to ensure the quality of information obtained in future research. The research instrument can also support the evaluation of conduct implemented to improve patient safety related to medication use and the establishment of the harm epidemiology.

Strengths and limitations

Regarding this study's strengths, the present instrument was validated using a multidisciplinary approach. Employing a research instrument that can be used by different healthcare professionals allows more people to be involved with patient safety regarding medication use in healthcare institutions. Moreover, the study serves as a basis for this research instrument to be used in ADE detection in pediatric inpatients and to implement triggers to monitor patient care in Brazilian hospitals.

Regarding this study's limitations, in both rounds of the trigger adaptation step, fewer than 50% of the experts invited to participate answered the questionnaire. In research using electronic questionnaires, a known problem is the low rate of return by experts,²⁷ but researches with a small number of Delphi participants have obtained good results.²⁸ The response rate was considered satisfactory, mostly because the healthcare professionals who participated were engaged in patient safety.

The methodology may include a retrospective study bias, in which the interpretation of the reviewer tends to introduce bias. The team sought to standardize the application of the methodology, including training using a detailed description of study triggers and involving teams at all stages of the research with regular meetings to reduce the occurrence of bias.

The data collection phase was carried out in a teaching hospital, which limits the generalizability of the study results. Some triggers not found in patient records during the study would perhaps be found if a longer time period were analyzed. It is possible that triggers identified in this sample would be important data for ADE detection on in a larger sample.

Finally, inter-rater reliability was not assessed, and it was not possible to establish the consistency of the research instrument with alternate users. The lack of inter-rater reliability assessment was reduced with research team meetings that included discussions concerning triggers and ADE identified.

Funding

Lunara Teles Silva was a fellow of the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (Capes).

Conflicts of interest

The authors declare no conflicts of interest.

Acknowledgement

The authors would like to thank Luciana Resende Prudente for assistance in translating the research instrument. We would also like to thank the creators of the research instrument for allowing it to be used in this study and the Delphi panel for contributions to the triggers’ adaptation. We would like to thank Fapeg and Capes (AUXPE 1665/2016).