Validation of the Brazilian version of the Pediatric Obstructive Sleep Apnea Screening Tool questionnaire

Abstract

Objective

To validate the Pediatric Obstructive Sleep Apnea Screening tool for use in Brazil.

Materials and methods

The Brazilian version of this questionnaire, originally validated and tested in the United States, was developed as follows: (a) translation; (b) back-translation; (c) completion of the final version; (d) pre-testing. The questionnaire was applied prior to polysomnography to children aged 3–9 years from October 2015 to October 2016, and its psychometric properties (i.e., validity and reliability) were evaluated. The accuracy was assessed from comparisons between polysomnographic results and corresponding questionnaire scores.

Results

Sixty patients were enrolled, and based on polysomnographic findings, 48% patients had normal apnea-hypopnea index, while the remaining 52% met the criteria for obstructive sleep apnea. Minimum O₂ saturation level was significantly lower among obstructive sleep apnea children (p = 0.021). Satisfactory concordance was found between individual apnea-hypopnea index and questionnaire scores. Bland–Altman plot-derived bias was 0.1 for the difference between measures, with 5.34 (95% CI: 4.14–6.55) and −5.19 (95%CI: −6.39 to −3.98) for the upper and lower agreement range. Internal consistency derived from Cronbach's alpha was 0.84 (95%CI: 0.78–0.90).

Conclusion

The questionnaire was translated to and validated into Brazilian-Portuguese version, and showed good reliability and concordance with apnea-hypopnea index. This questionnaire offers a reliable screening option for sleep-disordered breathing in children.

Resumo

Objetivo

Validar o questionário Pediatric Obstructive Sleep Apnea Screening Tool para o seu uso no Brasil.

Materiais e métodos

A versão brasileira desse questionário, originalmente validado e testado nos Estados Unidos, foi desenvolvida a partir das seguintes etapas: a) tradução; b) retro-tradução; c) conclusão da versão final; d) pré-teste. O questionário foi aplicado previamente ao início da polissonografia em crianças de 3 a 9 anos incluídas no estudo no período de outubro de 2015 a outubro de 2016. As propriedades psicométricas avaliadas foram validade e confiabilidade. A acurácia foi avaliada pela comparação entre os resultados da polissonografia com o escore do questionário.

Resultados

Foram incluídos no estudo 60 pacientes. Conforme a polissonografia, 48% dos pacientes apresentaram índices de apneia e hipopneia normais e 51% apresentaram resultados alterados. A SpO₂ mínima foi significativamente menor (p = 0,021) nas crianças com diagnóstico de síndrome de apneia obstrutiva do sono. O índice de apneia e hipopneia apresentou concordância satisfatória com os resultados do questionário. O viés médio de Bland-Altman foi de 0,1 para a diferença entre as medidas, com um limite superior de 5,34 (IC95%4,14 a 6,55) e um limite inferior de -5,19 (IC95%-6,39 a -3,98). A consistência interna do questionário avaliada pelo α de Cronbach foi de 0,84 (IC95%0,78 a 0,90).

Conclusão

O questionário foi traduzido e validado adequadamente para a versão em português brasileiro, apresentando boa confiabilidade e concordância com o índice de apneia e hipopneia. Esse questionário oferece uma opção confiável de triagem de distúrbios respiratórios do sono em crianças.

Introduction

Sleep-disordered breathing (SDB) encompass a set of altered respiratory patterns during sleep that include primary snoring, upper airway resistance syndrome, and obstructive sleep apnea syndrome (OSAS).¹ The prevalence of SDB is estimated at around 4–11% in the pediatric population.² The main risk factor for the development of this pathology is adenotonsillar hypertrophy.³

SDB diagnosis and referral to treatment aim to minimize associated morbidities, mainly cardiological,⁴ cognitive,⁵ and metabolic dysfunctions.⁶ The American Academy of Pediatrics recommends the performance of overnight polysomnography (PSG) in the laboratory as the gold standard for the diagnosis of OSAS in children with clinical manifestations of SDB.^7,8 However, in countries where access to PSG is precarious, the evaluations performed from questionnaires become of great clinical importance, with a low operational cost.^9,10

Currently, in Brazil there are five validated questionnaires for evaluation of sleep disorders in pediatrics,^11–15 and only one¹⁵ has a specific validation for sleep-disordered breathing. However, it is indicated for an age group that includes adolescents.¹⁵ To date, there is no validated questionnaire for SDB that is specific for use in preschoolers and schoolchildren, phases known to show an increase in the incidence of these disorders.

The pediatric obstructive sleep apnea screening tool (PosaST) is a questionnaire developed and validated by Gozal et al.,^16,17 which has high sensitivity and moderate specificity for the diagnosis of moderate to severe OSAS in preschoolers and school children. This tool can discriminate children at greater risk for OSAS and, consequently, indicates those with greater urgency to undergo PSG and subsequent treatment of the underlying pathology.^16,17

The aim of this study was to translate, culturally adapt, and validate the PosaST for use in the pediatric population of Brazil, since this tool has high potential when SDB is suspected, a fact that is even more important due to the patients’ difficulty in access to PSG exams.

Materials and methods

Study participants

Children aged 3–9 years referred to sleep laboratories by their attending physicians to undergo polysomnography for suspected SDB were invited to participate in the study. The participating centers in the study were: Laboratório de Neurofisiologia Clínica, of the Neurology Service of Hospital São Lucas da PUCRS, Laboratório do Sono, of the Pneumology Service of Hospital de Clínicas de Porto Alegre (HCPA), and Laboratório do Sono, of the Hospital Mãe de Deus Center (HMDC). Recruitment was carried out consecutively in each center from October 2015 to October 2016. Previously, the children's parents or guardians signed the consent form, and children older than 6 years old signed the assent form. The use of the questionnaire for validation in the Brazilian Portuguese language was authorized by the author of the original article.^16,17 The study was approved by the Research Ethics Committee of PUCRS in September 2015, CAAE 46804215.0.0000.5336.

Patients with a previous diagnosis of OSAS, patients with craniofacial malformations, neurological diseases, genetic syndromes, and those who did not agree to participate in the study were excluded.

In addition to applying the PosaST questionnaire, now known in Portuguese as the Ferramenta de triagem de apneia obstrutiva do sono em pediatria (FASP), patients were also evaluated regarding their demographic data and clinical history, such as sleep time per night, time of symptom onset, presence of associated chronic diseases, presence of sinonasal disorders, and use of medications. The anthropometric evaluation was performed in all patients, and the nutritional status was classified according to the BMI z-score with weight and age.¹⁸ Those with z-score values ≥−2 and <+1 were considered as having normal weight; overweight if ≥+1 and <+2, and obese if ≥+2 and ≤+3. Patients were considered low weight if the z-score was ≥−3 and <−2.¹⁸

The questionnaire was applied during an interview that lasted approximately 10 min, using a standardized technique. After the interview, patients were referred for the PSG originally requested by their attending physician.

Tool validation

PosaST was translated from its original version (Fig. 1A) into Brazilian Portuguese by two independent bilingual translators, fluent in the English language. The two versions were evaluated and compared by two reviewers. Disagreements regarding the questions were modified to reach a consensus.

Figure 1.

Questionnaires. (A) Pediatric obstructive sleep apnea screening tool; (B) Ferramenta de triagem de apneia obstrutiva do sono em pediatria questionnaire.

The final version in Portuguese was back-translated into English by two other bilingual translators who did not participate in the initial stage, and the divergences were analyzed. When necessary, the questions in Portuguese were rewritten until an agreement was achieved. The final version was approved by the main author of the original version, since he knows the Portuguese language.

The final version of the FASP in Brazilian Portuguese was tested in a group of ten parents of patients with SDB symptoms, randomized in three polysomnography laboratories. The six questions had to be understood by at least 80% of the interviewed adults.¹³ Since there was an adequate understanding of the final version in Portuguese, no changes had be made, concluding the final version for hetero-evaluation (Fig. 1B).

Polysomnography

The polysomnographies were performed at the Laboratório de Neurofisiologia Clínica of Hospital São Lucas da PUCRS, at the Laboratório do Sono of HCPA, and at the Laboratório do Sono of HMDC using Enza Brain-Net 36^® with software Poliwin 53, 36 channels (EMSA Equipamentos Médicos Ltda, RJ, Brazil); Brain Wave III^® with software BW analysis version 1.85, 16 channels (MF Neurovirtual, SP, Brazil); Alice 5^® with software Sleepware G3 version 3.4.1, 16 channels (Philips Respironics^®, PA, USA). In all the laboratories, the examination was conducted according to the norms of the American Academy of Sleep Medicine (AASM)¹⁹ using: electroencephalogram, electro-oculogram, electrocardiography, electromyography, nasal pressure transducer, piezo-crystal sensors for identification of thoracic and abdominal respiratory effort, snoring sensor, position sensor, and pulse oximetry. Blinded professionals reported the scores of the examinations following the international guidelines, with the distribution of sleep stages in NREM sleep (N1, N2, N3) and REM sleep.¹⁹ PSGs were performed at night in a dark and silent room with a mean ambient temperature of 24 °C. Sleep was spontaneous, without previous deprivation. Simultaneously, the patient's were videotaped to aid in the identification of clinical events.

Respiratory events were identified according to the criteria recommended by the AASM.¹⁹ Obstructive apneas were classified as a cessation of airflow, associated with a drop of ≥90% in the basal signal of the oronasal sensor, for two or more respiratory cycles, in the presence of paradoxical movements of the abdomen and thorax. Hypopnea was defined as a reduction in the signal of the nasal pressure thermistor of ≥30%, with a duration of two respiratory cycles. The event should be associated with oxygen desaturation of at least 3% or arousal.¹⁹ Central apnea was defined as absence of airflow through both the nose and mouth, as well as absence of inspiratory effort, with a duration of at least 20 s or lasting two respiratory cycles, associated with arousals or oxygen desaturation of at least 3%.¹⁹ Mixed apnea was classified when it met the criteria of apnea for at least two respiratory cycles during basal breathing, associated with the absence of respiratory effort during part of the event, with the presence of respiratory effort in another part of the event, regardless of which part occurs first.¹⁹

The apnea-hypopnea index (AHI) was defined as the number of apneas and hypopneas per hours of total sleep time. Patients with AHI ≥ 1.5 and <5 were considered as having mild OSAS; those with AHI ≥ 5 and <10, as moderate OSAS; and those with AHI ≥ 10 events/h, as having severe OSAS.¹⁷

Oxygen desaturation was defined as a decrease in SpO₂ greater than or equal to 3% in sustained values lower than 90%. In the study, SpO₂ was determined during wakefulness, its mean during sleep, its lowest point, and the percentage of total sleep time were determined during which SpO₂ remained below 90%. The desaturation index was determined as ≥3% of the basal value per hour of sleep.¹⁹

Psychometric property assessment

The assessed psychometric properties were validity and reliability. Validity was assessed using the Bland–Altman method of agreement between the questionnaire and the AHI. Reliability was assessed using the Cronbach's alpha coefficient. The acceptable value of Cronbach's alpha was set at >0.7.

Accuracy assessment

Patients with normal and altered questionnaire results were compared with the results of PSG, aiming to evaluate their ability to detect the disease.

Regarding the questionnaire, the cumulative score of the answers was considered for the probable diagnosis of OSAS. The cumulative score is represented by the average of all six questions according to the following formula (in which Q1 corresponds to question 1, Q2 corresponds to question 2, and so on): A = (Q1 + Q2)/2; B = (A + Q3)/2; C = (B + Q4)/2; D = (C + Q5)/2 and the cumulative score = (D + Q6)/2. According to the original evaluation,^16,17 a score ≥2.72 was used and considered to be indicative of a high risk for OSAS. All questions were answered using the Likert scale: “never” (0), “rarely” (once a week; 1), “occasionally” (twice a week; 2), “frequently” (three to four times a week; 3) and “almost always” (more than 4 times a week; 4). The question that considers the intensity of snoring was scored as follows: very low: 0, a little high: 1, high: 2, very high: 3 and extremely high: 4.¹⁶

Statistical analysis

The sample size was estimated at 46 patients (23 with a diagnosis of OSAS and 23 without). The significance level was set at 0.05, with an effect power of 90%. The software used to calculate the sample estimate was Winpepi, version 11.65 (PEPI-for-Windows, computer programs for epidemiologists).

Continuous variables were described as mean and standard deviation, and those with asymmetric distribution as median and interquartile range (IQR). Categorical variables were described as absolute and relative frequencies. To calculate the differences between groups, Student's independent t-test was used for variables with normal distribution and the Mann–Whitney test was used for variables with non-normal distribution. The study accuracy assessed the proportion of correct tests based on the sum of the truly positive and the truly negative results over all the obtained results.

The definition of cutoff points for the questionnaire indexes was obtained by the receiver operating characteristic (ROC) curve. Sensitivity and specificity for the disorder assessment were also recorded. The area under the curve was considered significant when values ≥0.5 were obtained.

Data analysis and processing were performed using SPSS software version 17 (Released 2008. SPSS Statistics for Windows, Chicago, USA).

Results

Study participants

A total of 86 patients were recruited for the present study, of whom 21 were excluded (19 due to concomitant diagnosis of neurological diseases and two due to craniofacial malformations). Of the 65 children included, five were later excluded because they did not complete the PSG. The mean age was 5.6 ± 2.0 years in these 60 patients; most of them were males, 36 (60%) and Caucasians, 55 (91%). Most of the patients who participated in the study were from HMDC (66%), followed by PUCRS (26%) and HCPA (8%). Half of the sample had normal weight, 15 (25%) were overweight, and 15 (25%) were diagnosed as obese. The clinical and demographic data of patients without and with OSAS are shown in Table 1.

Table 1.

Clinical and demographic data of patients without and with obstructive sleep apnea syndrome (OSAS).

	Total	Without OSAS	With OSAS	p
Patients, n (%)	60	29 (48)	31 (51.7)	N/A
Age (years)	5.6 ± 2.0^a	6.4 ± 1.6^a	4.8 ± 2.0^a	0.002^c
Male gender, n (%)	36 (60)	16 (26.6)	20 (33.3)	0.318^c
BMI	17.57 ± 3.3^a	18.0 ± 4.0^a	17.2 ± 2.4^a	0.436^c
Morning headache, n (%)	12 (20)	7 (11.6)	5 (8.3)	0.326^c
Difficulty learning and aggressiveness, n (%)	26 (43.3)	14 (23.3)	12 (20)	0.313^c
Sleep latency	19.6 (3.9–40.8)^b	19.5 (10.2–48.5)^b	20 (0.0–39.4)^b	0.202^d
Sleep efficiency	80.57 ± 12.2^a	77.7 ± 12.6^a	83.2 ± 11.6^a	0.088^c
Mean O2 saturation	96.8 ± 0.87^a	96.8 ± 0.75^a	96.7 ± 0.99^a	0.816^c
Minimum O2 saturation	88.0 ± 5.84^a	89.7 ± 2.5^a	86.2 ± 7.5^a	0.021^c
Questionnaire score	2.63 ± 1.0^a	2.70 ± 1.0^a	2.57 ± 1.0^a	0.623^c

SD, standard deviation; BMI, body mass index; AHI, apnea and hypopnea index.

Data: a, mean ± SD; b, median (25–75% interquartile range).

p-values based on the tests: c, Student's t-test; d, Mann–Whitney test.

The most frequent indications for PSG were snoring and suspected sleep apnea 47(78.3%) and restless sleep 13(21.7%). The most common clinical complaints were predominant oral breathing, learning difficulties, aggressiveness, and morning headache (Table 1).

Polysomnography results

Twenty-nine (48%) patients had normal results and 31 (51.7%) had altered results; 22 (36.6%) were diagnosed with mild OSAS, 7 (11.7%) with moderate OSAS, and two (3.3%) with severe OSAS. The sleep onset latency (p = 0.161), sleep efficiency (p = 0.088), and mean O₂ saturation (p = 0.816) did not differ between groups with and without OSAS. The minimum SpO₂ was significantly lower in the OSAS group (89.7 ± 2.5 vs. 86.2 ± 7.5, p = 0.021). When comparing the two groups of patients, a significant difference (p = 0.046) of the minimum SpO₂ between patients without OSAS and patients with moderate to severe OSAS (AHI ≥ 5) was identified. There was no significant difference between patients without OSAS and patients with mild OSAS.

Psychometric properties

The concordance analysis between the questionnaire results and the apnea hypopnea index identified a mean bias of 0.1 for the difference between the measures, with an upper limit of 5.34 (95% CI: 4.14–6.55) and a lower limit of −5.19 (95% CI: −6.39 to −3.98), as assessed by the Bland–Altman plot (Fig. 2). These results indicate that the validity was acceptable. The internal consistency of the questionnaire assessed by Cronbach's α was 0.84 (95% CI: 0.78–0.90).

Figure 2.

Bland–Altman plot for the mean of the questionnaire score and apnea and hypopnea index. AHI, apnea and hypopnea index.

ROC curve

Using the cumulative score ≥2.72 of the original scale, the sensitivity for the diagnosis of moderate or severe OSAS (AHI ≥ 5) was 67% (95% CI: 29.9–92.5) and the specificity was 53% (95% CI: 38.5–67.1). The analysis of the ROC curve showed an area under the curve of 0.598 (Fig. 3).

Figure 3.

ROC curve. ROC curve for the diagnosis of moderate or severe OSAS (AHI ≥ 5). Area under the curve (AUC): 0.59. Sensitivity of 67% (95% CI: 29.9–92.5) and specificity of 53% (95% CI: 38.5–67.1).

Discussion

The PosaST questionnaire was translated and validated from the original questionnaire in the English language into the Brazilian Portuguese version, FASP, which showed good reliability and agreement with the apnea and hypopnea index in a population with a high probability of OSAS.

Reliability was not calculated in the original study¹⁶; therefore, there is no basis for comparison between the studies, but it can be affirmed that the reliability in the current study was high, considering the internal consistency of 0.84. The presence of basic elements, such as good reliability, shows the ability to reproduce the results over time in stable individuals, with sensitivity to possible changes in sleep patterns of individuals. These are important requirements to consider when choosing an OSAS screening questionnaire.

This study evaluated the concordance rates between the questionnaire cumulative score and the AHI scores, using the statistical analysis proposed by Bland–Altman.²⁰ Satisfactory concordance rates were found between the questionnaire and the AHI results for subjects with mild OSAS. These results are relevant, because they show that the tool can be used to identify patients with mild OSAS, differently from the initial proposal, in which the questionnaire would have shown better performance in the screening of moderate to severe patients.

This broader use indicates the possibility of a more comprehensive use of the tool, since the management of mild OSAS is often different from that of moderate and severe OSAS. Current studies suggest that the option of cautiously waiting/following symptom evolution may be a reasonable option in children with mild OSAS, especially in the absence of obesity. It has been observed that after seven months of follow-up, up to 42% of patients previously diagnosed with mild OSAS symptoms show resolution of the clinical condition through the normalization at the PSG examination.²¹ It is suggested that the use of intranasal corticosteroids (INCs) or leukotriene antagonists in patients classified as having mild OSAS during this follow-up period may help reduce OSAS symptoms.⁸ Another possibility for the initial therapy for the treatment of mild OSAS is the association of an INC with montelukast for at least three months. The concomitant use of these medications has already been shown to be an effective alternative to surgery, especially in non-obese patients and in children under 7 years of age, with a success rate of 80%.²² Meanwhile, in patients with moderate to severe OSAS, referral for surgical treatment is usually the best option.¹⁰

Other studies have also used the Bland–Altman concordance analysis for sleep evaluation. Werner et al., in 2008, concluded that actigraphy and sleep diaries can be used to assess sleep latency, time to wake up, and total sleep time in 4- to 7-year-olds.²³ Another more recent study, from 2016, used the Bland–Altman analysis to assess test-retest agreement in the Brazilian version of the Pittsburgh Sleep Quality Index questionnaire in adolescents with good agreement in the two questionnaire applications.²⁴ The use of the Bland–Altman method in out data indicated that the FASP has satisfactory characteristics for the screening of symptomatic patients who require referral for polysomnography.

As in the original study, the ROC curve for the questionnaire validation was calculated, considering the cumulative score ≥2.72. When using AHI cutoff points of 1.0 or 1.5 for the calculation, good diagnostic performance was not observed. When using the cumulative score ≥2.72 and the AHI ≥ 5 as the cutoff point, slightly worse results were found than those of the original scale, with an AUC of 0.59 (versus AUC of 0.64). Lower sensitivity (67%) and specificity (53%) values were found for the diagnosis of moderate to severe OSAS than in the reference study. The choice of a cumulative score ≥2.72 is based on references that used this same cutoff point.¹⁶ This regular diagnostic performance of the cumulative score of the questionnaire with the PSG assessment is probably related to the small number of patients diagnosed with moderate to severe OSAS in the sample included in the study. In more recent studies validating the questionnaire in other languages, the cutoff point had to be modified to better adapt to the clinical characteristics of the population.²⁵

One study limitation was that the data collection was restricted to patients referred to sleep laboratories, creating the bias of not adequately representing the overall pediatric population. However, the development and validation of the original questionnaire was performed in a primary service and, afterwards, its applicability was validated in a reference center for pediatric sleep.^16,17 Even though population samples are different, as the questionnaire had already been validated in a tertiary service, it was decided to collect the patients’ data in the PSG laboratories, as did other previous studies.²⁵

In Brazil, it is difficult to include questionnaires in research, due to the small number of these tools that have undergone a process of validation with high academic rigor. There is an increasing number of tools validated in the literature, but the quality of these validations is not always in accordance with the recommended protocols.^13,26 The presence of a test for hetero-evaluation and SDB screening in preschoolers and school children is of extreme clinical relevance, mainly in developing countries, where there is a reduced availability of more sophisticated and expensive exams.

The FASP was validated for the identification of pediatric patients with SDB. Because it consists of six hierarchical questions about pediatric sleep, it has shown to be a brief and simple-to-apply screening tool, which facilitates its use in everyday life. Additionally, it showed good concordance for the identification of patients with mild OSAS, indicating which children should initiate clinical treatment and whether they should be referred for diagnostic exams.

The Brazilian version of the PosaST questionnaire showed good performance. Although it has been validated for use in Brazil, it still needs to be further explored in the sleep field, by applying it to a greater number of patients, especially in those with more severe disease. Nevertheless, it has shown to be a rapid tool, with a simple approach and satisfactory agreement in patients with mild OSAS, which can be easily implemented for clinical use.

Funding

CAPES.

Conflicts of interest

The authors declare no conflicts of interest.