Journal Information
Vol. 97. Issue 3.
Pages 315-320 (May - June 2021)
Share
Share
Download PDF
More article options
Visits
2666
Vol. 97. Issue 3.
Pages 315-320 (May - June 2021)
Original article
Open Access
Postural balance, handgrip strength and mobility in Brazilian children and adolescents with osteogenesis imperfecta
Visits
2666
Giovana Coêlhoa,
Corresponding author
giovanacoelho3@gmail.com

Corresponding author.
, Lívia Cocato Luiza, Luiz Claudio Castrob, Ana C. de Davida
a Universidade de Brasília, Faculdade de Educação Física, Brasília, DF, Brazil
b Universidade de Brasília, Faculdade de Medicina, Departamento de Pediatria, Brasília, DF, Brazil
This item has received

Under a Creative Commons license
Article information
Abstract
Full Text
Bibliography
Download PDF
Statistics
Figures (1)
Tables (3)
Table 1. Clinical and functional locomotion data.
Table 2. Center of pressure parameters in type I osteogenesis imperfecta (OI) (mean±SD).
Table 3. Handgrip strength and mobility data.
Show moreShow less
Abstract
Objective

To describe postural balance, handgrip strength and mobility in children and adolescents with different types of osteogenesis imperfecta.

Methods

Cross-sectional study. Fifty selected subjects diagnosed with types I (n=11), III (n=21), and IV (n=18), followed up at Brazilian reference center for osteogenesis imperfecta in the Midwest region, aged 2–21 years (9.2±5.0), were enrolled in this study. Children and adolescents were evaluated for postural balance in the upright position with eyes-open and eyes-closed conditions, handgrip strength and the mobility domain (Pediatric Dysfunction Assessment Inventory). Data normality and difference between groups was verified.

Results

Handgrip strength was significantly lower in people with type III of osteogenesis imperfecta when compared to the osteogenesis imperfecta types I and IV, and to the age-specific reference data. Center of pressure length and mean velocity in the condition with eyes closed were worse compared to the open-eyes condition for children and adolescents with type I of osteogenesis imperfecta. There were worse results in the mobility domain for the participants classified with the most severe type of osteogenesis imperfecta.

Conclusions

It was observed that the severity of the osteogenesis imperfecta disease affected handgrip strength and locomotor function assessed by the mobility domain. Comparing osteogenesis imperfecta types, the higher the severity of osteogenesis imperfecta, the lower the handgrip strength. These results can contribute to new strategies of treatment focused on improving functional capacity and quality of life in people with osteogenesis imperfecta.

Keywords:
Osteogenesis imperfecta
Functional aspects
Postural balance
Mobility
Handgrip strength
Full Text
Introduction

Osteogenesis imperfecta (OI) is a connective tissue disorder,1 with an autosomal dominant inheritance, and is rarely recessive.2 It is considered a rare metabolic bone disorder estimated to affect about 1 in 10,000 individuals, with a prevalence of type I (i.e., the mildest and most common form) of approximately 1 in 28,000 births.3 OI is characterized by an abnormal mineralization and altered mechanical properties of the bone tissue, as well as less dense and more fragile bones compared to normal bone tissues.4

The most used clinical classification of OI, still valid, was proposed by Sillence et al., who divided the disease into four distinct types, categorized from mild to lethal.5,6 The mild form, type I, is related to quantitative mutations by the identification of collagen deficiency, with a reduction in the amount of collagen. Type II is the lethal form characterized by lethality in the perinatal period or shortly after birth. Type III is the most severe survivable form. Moderate, type IV, is an intermediate form between types I and III, which presents qualitative mutations, resulting in structural collagen abnormalities.7,8 In order to decrease bone fragility and pain, studies have reported the positive effects of the treatment with pamidronate in improving bone density and preventing skeletal fractures in children with OI.9 As a consequence, the treatment could be an aid to improve mobility and independence in the activities of daily living.5

Clinical manifestations in the different types of OI appear to affect children's motor and functional performance.10 Previous studies have reported reduction in functional capacity, increased fatigue, and decreased physical fitness in children with OI.10 Montpetit et al. reported that people with type III OI presented lower scores of mobility and home life activities, as well as lower levels of participation regarding study activities, sports activities, and the professional environment.11 Not only people with type III OI present physical limitations, but also people with mild OI – type I, generally considered with good motor skills, may report difficulties to perform activities of daily living such as walking and running.9 Mobility is of particular interest to the OI population as it is associated with multiple aspects of participation and quality of life.3 In addition, children and adolescents with OI often have less active lives when compared to healthy individuals, due to the frequent fractures and recurrent periods of immobilization.9 Given the bone fragility and the high risk of fractures, people with OI are generally, and inappropriately, advised to avoid physical activities and engaging in regular physical exercise.4 Although most children with OI would benefit from some type of rehabilitation program and/or exercise intervention, the degree of physical limitations are still unclear, limiting the strategies that can be adopted in clinical practice.9

Postural balance is important for motor and functional performance.12 Inferior static postural balance has been observed in youth with type I OI when compared to typically developing (TD) individuals.12 Compromising factors related to the defective protein in OI, type I collagen, probably affect the proprioceptive sensory information.12 Mechanoreceptors, located in muscles, tendons, ligaments, and skin, are mainly composed of type I collagen and play an important role in the sensory inputs for postural balance.13

In clinical practice, the evaluation of muscle strength in people with OI is typically limited to the pediatrician's subjective perception. However, a more quantitative measure of muscle strength can be obtained by determining the maximal isometric handgrip strength in the OI population.14 Lower levels of handgrip strength have also been reported in children and adolescents with OI when compared to healthy individuals within the same age, with worse results for those classified with the most severe types. Therefore, this suggests handgrip strength as an indicator of disease severity.15 However, the cause of muscle weakness in individuals with type I OI has not yet been identified; it could be a consequence of the lower levels of physical activity reported and/or the result of the poor synthesis of type I collagen in the muscles and tendons of this population.16

Despite the physical limitations of children and adolescents with OI, there is lack of information regarding the ambulatory characteristics, strength, and functional capacity of patients with OI, particularly from those classified with the more severe forms.5 Given the rarity of OI, there is a challenge in providing evidence-based answers to clinical practice. Studies comprising rare diseases can lead to a better understanding of the natural history of the disease, generating hypotheses for further research. To the best of the authors’ knowledge, no studies have so far reported data regarding the motor performance and locomotor function of Brazilian patients with OI. Thus, the aim of this study was to describe postural balance, handgrip strength and mobility of a group of children and adolescents with OI followed up at Brazilian reference center.

Material and methods

This is a cross-sectional study. After a general invitation and selection of potential participants, a convenience sample of 50 children and adolescents diagnosed with OI were included in the study. The patients were recruited from a hospital – the Reference Center for Osteogenesis Imperfecta (CROI), and have agreed to participate in the study. Thirty-eight patients were hospitalized for the use of intravenous medication and twelve patients were attending the Bone Metabolism Outpatient Clinic. They are aged between 2 and 21 years (9.24±5.06), with types I (n=11), III (n=21), and IV (n=18). Inclusion criteria were (1) age between 2 and 21 years; (2) diagnosis of types I, III, or IV of OI confirmed by a medical report (imaging exams, blood and urinary tests, and number of fractures); and (3) being hospitalized for intravenous infusion at the reference hospital or attending the Bone Metabolism Outpatient Clinic. Exclusion criteria were (1) uncorrected visual deficit; and (2) deficits in cognitive functions that prevented collaboration, understanding, and participation in the proposed activities. A questionnaire made by the authors and directed to the patient or to a referred family member was used in order to obtain general information such as family history, medication initiation/adherence, anthropometric assessment, and participation in and perspective on physical activities. The written informed consent was obtained from the participants or from a family member responsible for the participant (i.e., participants ≤18 years) prior to procedures. The study was approved by the Ethics Committee from the Department of Health/Federal District - Brazil (CAEE: 2.570.210).

Data collection was performed over one semester. Given to the range of impairments affecting the different types of OI, the participants performed the evaluations according to their difficulties and limitations, varying the number of participants in each test, as shown in Fig. 1.

Figure 1.

Flow diagram.

(0.09MB).
Postural balance

For the measurement of postural balance data, a portable force platform (AccuSway Plus – Advanced Mechanical Technologies, Inc.) was used. The protocol used for the postural balance test has been previously published.12 Participants were evaluated in two different conditions: upright position with eyes open and upright position with eyes closed. Participants were initially asked to stand upright barefoot and to maintain a comfortable (self-selected) distance between the feet, but without exceeding shoulder width, with their arms relaxed at the side of the body. No shoes or orthoses were used during the tests. In the open-eyes condition, participants were asked to keep their eyes fixed at a point ahead at eye level, at a distance of approximately two meters.

For each condition, three trials were measured with a one-minute interval between them, in which the subjects remained seated in a chair with support in the dorsal region. The sampling frequency was 100Hz and the acquisition time of 20s was used for each trial. The average of the three trials of the following static balance outputs were used in the analysis: amplitude of displacement of the center of pressure (CoP) in the anteroposterior and mediolateral directions, mean velocity, CoP length, and CoP area 95% (i.e., defined as the ellipse that contains the center of the points of the center of force, with a 95% probability).12 Only the children and adolescents with type I OI were able to perform the postural balance evaluations.

Handgrip strength

Handgrip strength was assessed using the Saehan SH5001 dynamometer, with 1kgf resolution and a maximum reading capacity of 90kgf. A familiarization test was performed prior to the execution and measurement of handgrip strength. The protocol used for the handgrip test was previously published.14 During the evaluation, the participant remained seated in a chair without armrest, with the spine erect, knees flexed at 90°, and the limb to be tested suspended in the air with the hand placed on the dynamometer (shoulder in adduction and neutral position of rotation, elbow flexed at 90°, forearm in mid-pronation, and wrist ranging from zero to 30° in extension), neutral grip. Participants were instructed to hold and keeping pressing the handgrip for five seconds. Three measurements with the dominant hand were performed, with one-minute interval between measurements. Participants who were hospitalized performed the manual grip with the limb that was not receiving the medication. For the data analysis, the average of the three trials was considered. Children under 6 years old did not perform the handgrip test due to complex instructions and attention required to perform the test. Handgrip strength data (in newtons) were transformed to age-specific z-scores by using reference data from Rauch et al.14

Locomotor function

Mobility was assessed using the Pediatric Disability Assessment Inventory (PEDI) immediately after the postural balance evaluation. The PEDI consists of a questionnaire applied in the form of an interview with the patient or with a family member, if the patient was unable to answer. It addresses issues such as self-care, mobility (locomotor function), and social function, as well as identifying the level of independence of the individual. In the current study, only the mobility domain was evaluated, which comprised transfers (i.e., in the bathroom/chairs; in the car/bus; in the shower), mobility in the bed/transfers, indoor walking, outdoor walking, and stair climbing. Each PEDI item is rated with 0 or 1 for performance capability. The compound age group was the same as in the study, considering that mobility evaluation was performed for all the participants.

Data analysis

Descriptive analyses of the anthropometric variables, postural balance parameters, handgrip strength, and mobility were performed. Normality of the data was verified using the Shapiro–Wilk test. To investigate differences between groups of OI type, the one-way ANOVA test and Student's t-test were used for the parametric data. Nonparametric data were analyzed using the Kruskal–Wallis test. Tukey's was used as the post hoc test. Associations between groups were verified using the chi-squared test. The significance level adopted was 5%. The software SPSS (v. 20) was used in the analysis.

Results

As shown in Table 1, the sample (n=50) was composed mostly of males (n=27; female, n=23), with a mean age of 9.2±5.0 years and disease types I (n=11), III (n=21), and IV (n=18). Results from the screening questionnaire showed that all the participants with type I OI and, 66.7% of the participants with type IV OI reported independent walking, while only 19% of the participants with type III OI reported being able to move independently. Functional locomotion (i.e., measured by the screening questionnaire) showed significant differences between groups.

Table 1.

Clinical and functional locomotion data.

  GroupTotal  p-value* 
  Type I (n=11)  Type III (n=21)  Type IV (n=18)     
Gender (n; %)†††
Female  3; 27.3  14; 66.7c  6; 33.3  23; 46  0.042 
Male  8; 72.7  7; 33.3  12; 66.7  27; 54   
Age (years), mean±SD  12.4±6.31a  7.42±4.01  9.43±4.58  9.24±5.06  0.027 
Weight (z-score), mean±SD††  −0.94±0.98a  −3.88±2.35b  −0.88±1.25  −2.14±2.28  0.001 
Height (z-score), mean±SD††  −2.13±1.46a  −5.77±2.14b  −1.82±1.48  −3.53±2.59  0.000 
Locomotion (n; %)†††
Independent  11; 100c  4; 19  12; 66.7  27; 54  0.001 
Walker  –  2; 9.5  3; 16.7  5; 10   
Wheelchair (independent use)  –  6; 28.6c  1; 5.6  7; 14   
Wheelchair (dependent use)  –  3; 14.3  2; 11.1  5; 10   
Can’t walk  –  6; 28.6c  –  6; 12   
Physical activity (n; %)†††
Yes  5; 45.5  3; 14.3  4; 22.2  12; 24  0.143 
No  6; 54.5  18; 85.7  14; 77.8  38; 76   

Bold value signifies significance level p<0.05.

Anova One-Way

† †

Kruskal–Wallis

†††

Chi-square test.

a

= difference between type I and type III.

b

= difference between type III and type IV.

c

= association between variables.

*

Significance level p<0.05.

Postural balance results from the participants with type I OI are presented in the Table 2. CoP length and mean velocity parameters were significantly higher in the eyes-closed condition compared to the eyes-open condition. CoP area 95% and amplitude of CoP displacement in the mediolateral and anteroposterior directions were not significantly different between conditions.

Table 2.

Center of pressure parameters in type I osteogenesis imperfecta (OI) (mean±SD).

  Condition (n=11; type I)p-valuea 
  Eyes open  Eyes closed   
Path length (cm)  31.42±10.24  40.49±17.00  0.010 
Average velocity (cm/s)  1.16±0.36  1.47±0.55  0.009 
Area 95% (cm23.18±3.87  3.85±3.09  0.287 
Anteroposterior amplitude (cm)  2.18±1.26  2.69±1.56  0.108 
Mediolateral amplitude (cm)  1.76±0.81  1.93±0.78  0.237 

Bold value signifies significance level p<0.05.

a

Paired sample t-test, significance level p<0.05.

Results regarding strength and mobility are described in the Table 3. Thirty-five participants performed the handgrip strength test. Comparisons of groups revealed that handgrip strength was significantly different among all the three OI groups (I, III, and IV) considering the age-specific z-scores for grip strength. Compared to an age-specific reference data,14 handgrip strength from the present sample presented deficits of about 50%, 78%, and 100% for the participants classified with OI type I, III, or IV, respectively. Considering absolute values, lower handgrip strength was observed in the most severe type of OI (type III) when compared to the mild form (type I).

Table 3.

Handgrip strength and mobility data.

  Groupp-valuee 
  n  Type I  Type III  Type IV   
Mobility domain (PEDI), mean±SDa  50  63.45±1.29c  23.19±18.50d  49.0±20.86  0.000 
Grip strength (Newtons), mean±SDa  35  171.65±131.55c  63.49±43.88  142.62±82.82  0.021 
Age-related z-scores (n; %)b
Low  26  3; 50.0  7; 77.8d  11; 100.0  0.001 
Adequate    3; 50.0  2; 22.2  –   

PEDI, Pediatric Disability Assessment Inventory.

Bold value signifies significance level p<0.05.

a

One-way Anova.

b

Chi-squared test.

c

Difference of type I with type III.

d

Association between variables.

e

Significance level p<0.05.

Locomotor function (i.e., measured by mobility with the PEDI) revealed to be significantly different with worse results for the participants with type III OI when compared to OI types I and IV.

Discussion

This current study described postural balance, handgrip strength and mobility aspects of a group of children and adolescents with OI followed up at a reference center for OI treatment. Results from the postural balance parameters revealed that CoP length and mean velocity in the condition with eyes closed were worse (i.e., achieved higher values) compared to the open-eyes condition for children and adolescents with OI – type I. Handgrip strength was significantly lower in people with type III OI when compared to OI types I and IV, and to the age-specific reference data. Locomotor function was impaired in children and adolescents with OI, with worse results in the mobility domain for the participants classified with the most severe type of OI (type III).

Regarding the postural balance, higher values of CoP length and velocity in eyes-closed condition compared to the eyes-open condition were expected, demonstrating greater CoP oscillation with eyes closed. When visual input is altered, a greater demand from the somatosensory system is required for postural balance during the upright position. In this context, with eyes closed, an increased body sway is caused by the postural instability, especially in more disabled individuals such as those with OI (i.e., presenting muscular and connective tissues dysfunctions), but also in healthy individuals.13 The present results corroborate the findings from Pouliot-Laforte et al., who suggested worse postural balance in children and adolescents with type I OI when compared to typically developing subjects.12 The results were associated with a proprioceptive deficit in this population. Therefore, in order to reduce the risk of falls and the frequency of fractures in children and adolescents with OI, therapies to improve postural balance are necessary.12

According to the locomotor function results, worse mobility was reported by the group with the most severe type of OI (type III) compared to the types I and IV. This information is in agreement with the findings from Montpetit et al., who observed that patients with type III OI presented lower scores of mobility, and also presented lower levels of participation in the professional environment, sports activities, and level of ambulation.11 In the case of functional abilities, the severity of OI influences the time required for such individuals to achieve different motor milestones, the level of ambulation, and the self-care abilities; it may affect the development of motor skills, delaying them or even regressing the abilities already acquired.9,17,18 For these subjects, with the most severe form of OI, intravenous bisphosphonates should be considered. The main goal of the treatment is to increase bone mineral density and, consequently, reduce fracture rate.5,19

There are some limitations that should be considered when interpreting the results of this study. Given to the motor limitations caused by the disease, each participant performed the tests within their capabilities, resulting in different sample sizes for each test. For the postural balance evaluation, only the participants with type I OI were able to perform the test. In addition, the number of fractures was not assessed, as well as the treatment time (i.e., using the intravenous bisphosphonate medications) and the concurrent treatments (i.e., other medications and/or rehabilitation) were not controlled. Finally, there was a challenge in interpreting the results from this current study due to the rarity of the OI condition and the non-availability of similar studies in order to compare the main findings.

In conclusion, it was observed that the severity of OI affected handgrip strength and locomotor function assessed by the mobility aspect. Handgrip strength was significantly lower in people with type III OI when compared to OI types I and IV, and to the age-specific reference data. In the same way, locomotor function was impaired in children and adolescents with OI, with worse results in the mobility domain for the participants classified with the most severe type of OI (type III). Regarding the postural balance, CoP length and mean velocity of the CoP were better when visual input was available.

This study presented data regarding postural balance, muscle strength and mobility that could contribute to new strategies of treatment focused on improving functional capacity and quality of life in people with OI.

Funding

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior, Brasil (CAPES), Finance Code 001.

Conflicts of interest

The authors declare no conflicts of interest.

References
[1]
E.L. Germain-Lee, F.-S. Brennen, D. Stern, A. Kantipuly, P. Melvin, M.S. Terkowitz, et al.
Cross-sectional and longitudinal growth patterns in osteogenesis imperfecta: implications for clinical care.
Pediatr Res, 79 (2016), pp. 489-495
[2]
A. Augusciak-Duma, J. Witecka, A.L. Sieron, M. Janeczko, J.J. Pietrzyk, K. Ochman, et al.
Mutations in the COL1A1 and COL1A2 genes associated with osteogenesis imperfecta (OI) types I or III.
Acta Biochim Pol, 65 (2018), pp. 79-86
[3]
C.R. Garman, A. Graf, J. Krzak, A. Caudill, P. Smith, G. Harris.
Gait deviations in children with osteogenesis imperfecta type I.
J Pediatr Orthop, 39 (2019), pp. e641-e646
[4]
J.M. Fritz, Y. Guan, M. Wang, P.A. Smith, G.F. Harris.
A fracture risk assessment model of the femur in children with osteogenesis imperfecta (OI) during gait.
Med Eng Phys, 31 (2009), pp. 1043-1048
[5]
K.M. Kruger, A. Caudill, M. Rodriguez Celin, S.C.S. Nagamani, J.R. Shapiro, R.D. Steiner, et al.
Mobility in osteogenesis imperfecta: a multicenter North American study.
Genet Med, 21 (2019), pp. 2311-2318
[6]
D. Sillence, A. Senn, D. Danks.
Genetic heterogeneity in osteogenesis imperfecta.
J Med Genet, 16 (1979), pp. 101-116
[7]
J.C. Marini, A. Forlino, W.A. Cabral, A.M. Barnes, J.D. San Antonio, S. Milgrom, et al.
Consortium for osteogenesis imperfecta mutations in the helical domain of type I collagen: regions rich in lethal mutations align with collagen binding sites for integrins and proteoglycans.
Hum Mutat, 28 (2007), pp. 209-221
[8]
R. Marom, Y.C. Lee, I. Grafe, B. Lee.
Pharmacological and biological therapeutic strategies for osteogenesis imperfecta.
Am J Med Genet Part C Semin Med Genet, 172 (2016), pp. 367-383
[9]
A. Caudill, A. Flanagan, S. Hassani, A. Graf, R. Bajorunaite, G. Harris, et al.
Ankle strength and functional limitations in children and adolescents with type I osteogenesis imperfecta.
Pediatr Phys Ther, 22 (2010), pp. 288-295
[10]
R.H. Engelbert, C.S. Uiterwaal, W.J. Gerver, J.J. Van Der Net, H.E. Pruijs, P.J. Helders.
Osteogenesis imperfecta in childhood: impairment and disability. A prospective study with 4-year follow-up.
Arch Phys Med Rehabil, 85 (2004), pp. 772-778
[11]
K. Montpetit, N. Dahan-Oliel, J. Ruck-Gibis, F. Fassier, F. Rauch, F. Glorieux.
Activities and participation in young adults with osteogenesis imperfecta.
J Pediatr Rehabil Med, 4 (2011), pp. 13-22
[12]
A. Pouliot-Laforte, M. Lemay, F. Rauch, L.N. Veilleux.
Static postural control in youth with osteogenesis imperfecta type I.
Arch Phys Med Rehabil, 98 (2017), pp. 1948-1954
[13]
A. Nardone, A.M. Turcato.
An overview of the physiology and pathophysiology of postural control.
Biosyst Biorobotics, 19 (2018), pp. 3-28
[14]
F. Rauch, C.M. Neu, G. Wassmer, B. Beck, G. Rieger-Wettengl, E. Rietschel, et al.
Muscle analysis by measurement of maximal isometric grip force: new reference data and clinical applications in pediatrics.
Pediatr Res, 51 (2002), pp. 505-510
[15]
K. Montpetit, H. Plotkin, F. Rauch, N. Bilodeau, S. Cloutier, M. Rabzel, et al.
Rapid increase in grip force after start of pamidronate therapy in children and adolescents with severe osteogenesis imperfecta.
Pediatrics, 111 (2003), pp. e601-e603
[16]
A. Pouliot-Laforte, L.-N. Veilleux, F. Rauch, M. Lemay.
Physical activity in youth with osteogenesis imperfecta type I.
J Musculoskelet Neuronal Interact, 15 (2015), pp. 171-176
[17]
C. Land, F. Rauch, K. Montpetit, J. Ruck-Gibis, F.H. Glorieux.
Effect of intravenous pamidronate therapy on functional abilities and level of ambulation in children with osteogenesis imperfecta.
J Pediatr, 148 (2006), pp. 456-460
[18]
P. Zack, L. Franck, C. Devile, C. Clark.
Fracture and non-fracture pain in children with osteogenesis imperfecta.
Acta Paediatr, 94 (2005), pp. 1238-1242
[19]
P.J. Simm, A. Biggin, M.R. Zacharin, C.P. Rodda, E. Tham, A. Siafarikas, et al.
Consensus guidelines on the use of bisphosphonate therapy in children and adolescents.
J Paediatr Child Health, 54 (2018), pp. 223-233
Copyright © 2020. Sociedade Brasileira de Pediatria
Idiomas
Jornal de Pediatria (English Edition)
Article options
Tools
en pt
Taxa de publicaçao Publication fee
Os artigos submetidos a partir de 1º de setembro de 2018, que forem aceitos para publicação no Jornal de Pediatria, estarão sujeitos a uma taxa para que tenham sua publicação garantida. O artigo aceito somente será publicado após a comprovação do pagamento da taxa de publicação. Ao submeterem o manuscrito a este jornal, os autores concordam com esses termos. A submissão dos manuscritos continua gratuita. Para mais informações, contate assessoria@jped.com.br. Articles submitted as of September 1, 2018, which are accepted for publication in the Jornal de Pediatria, will be subject to a fee to have their publication guaranteed. The accepted article will only be published after proof of the publication fee payment. By submitting the manuscript to this journal, the authors agree to these terms. Manuscript submission remains free of charge. For more information, contact assessoria@jped.com.br.
Cookies policy Política de cookies
To improve our services and products, we use "cookies" (own or third parties authorized) to show advertising related to client preferences through the analyses of navigation customer behavior. Continuing navigation will be considered as acceptance of this use. You can change the settings or obtain more information by clicking here. Utilizamos cookies próprios e de terceiros para melhorar nossos serviços e mostrar publicidade relacionada às suas preferências, analisando seus hábitos de navegação. Se continuar a navegar, consideramos que aceita o seu uso. Você pode alterar a configuração ou obter mais informações aqui.