High and low-fidelity simulation for respiratory diseases pediatric training: a prospective and randomized study

Abstract

Objective

To compare high and low-fidelity simulations for the recognition of respiratory distress and failure in urgency and emergency pediatric scenarios.

Methods

70 fourth-year medical students were randomly distributed in high and low-fidelity groups and simulated different types of respiratory problems. Theory tests, performance checklists, and satisfaction and self-confidence questionnaires were used in the assessment. Face-to-face simulation and memory retention was applied. The statistics were evaluated by averages and quartiles, Kappa, and generalized estimating equations. The p-value was considered 0.05.

Results

In the theory test there was an increase in scores in both methodologies (p < 0.001); in memory retention (p = 0.043) and at the end of the process the high-fidelity group had better results. The performance in the practical checklists was better after the second simulation (p > 0,05). The high-fidelity group felt more challenged in both phases (p = 0.042; p = 0.018) and showed greater self-confidence to recognize changes in clinical conditions and in memory retention (p = 0.050). The same group, in relation to the hypothetical real patient to be treated in the future, felt better confident to recognize respiratory distress and failure (p = 0.008; p = 0.004), and better prepared to make a systematic clinical evaluation of the patient in memory retention (p = 0.016).

Conclusion

The two levels of simulations enhance diagnostic skills. High fidelity improves knowledge, leads the student to feel more challenged and more self-confident in recognizing the severity of the clinical case, including memory retention, and showed benefits regarding self-confidence in recognizing respiratory distress and failure in pediatric cases.

Introduction

In the context of simulated medical environments, it is possible to develop clinical reasoning and to accelerate the performance of the expert.¹ Accordingly, health care simulation provides learning in a safe environment, case scenarios that involve feedback and skill repetition, promoting deliberate practice among its participants.^2,3 It is a growing field that combines innovative technologies and adult learning theory to reproducibly train medical professionals in clinical skills and practices.^4,5

The terms “modality” and “fidelity” are different elements of the simulation that can generate different impacts on the activity.⁶ The modality is related to the type of equipment used and levels (low, medium, and high-fidelity) refer to the realism provided in the selection of equipment and scenario. Fidelity refers to the degree of accuracy achieved and corresponds to the credibility of the experience.^7,8 The choice of simulation level depends on the learning objective. Similar results to compare high (HF) and low-fidelity (LF) simulations for training are published in the literature, but greater satisfaction and self-confidence are reported in the first.⁹ HF is a practical methodology for teaching skills and competencies based on learning objectives.^10,11 Its synesthetic feature allows students to learn and show their peers how to do it, and it is superior for assessing diagnostic ability in a variety of technical and non-technical skills.^7-11 Participants interact with the manikin and interpret changes more quickly, make decisions, and retain knowledge, which can benefit the future assessment of the real patient.^4,12 Another positive aspect of HF is better integration and communication among team members in emergency care and in simulated practices.¹³

Acute respiratory distress is one of the most common reasons why parents take their children to the Emergency Room (ER). Given its potential to cause life-threatening diseases, emergency physicians need to maintain and update their knowledge on recommendations for childhood respiratory diseases.¹⁴ Therefore, the development of simulated scenarios for teaching students in pediatric emergencies addressing severe respiratory illness is extremely important and rare in the literature.^15,16 Early recognition and treatment of this condition prevent progression to cardiopulmonary arrest and death.¹⁷

The present study aims to compare the ability of medical students to develop expert performance in training using HF and LF simulations, addressing respiratory discomfort and failure in pediatrics and, after three months, assess memory retention in relation to learning.

Material and methods

This is a cross-sectional, randomized, and prospective study with 70 students in their fourth year of medical school from a private institution in the city of Belo Horizonte, state of Minas Gerais, Brazil. They were taking a mandatory course called “Child and Adolescent Health III,” which aims to introduce initial knowledge regarding pediatric patient care in urgent and emergency situations. The previous use of simulation as a teaching methodology of them was based on LF. The research was conducted in the SIMULAB (the university's simulation laboratory), in the first semester of 2021, for six days in January and two days in April, with a total workload of 6 h per participant.

The courseware was prepared in accordance with internationally accepted guidelines¹⁷ and regional evidence-based protocols, widely used by pediatricians.^18,19 Eleven judges were selected by convenience sample for the construction validation²⁰ of all parts of the course.

A pilot project was carried out to provide feasibility to the study. The sample size was calculated using the results of a pilot project. The effect size adopted was 0.8 since the HF and LF groups obtained respectively average values of 16.9 (± 3.08) and 15.9 (± 3.28) in the theory pretest (total of 30 points). An average of 21.8 (± 3.23) was obtained in the post-test. The power of the test adopted was 80% and the confidence interval was 95%. The sample calculated at least 64 participants.

This research was randomized through the software (random.org)²¹ in three steps. First, the randomization process was applied to define on which day HF or LF training would be tested (three different dates for each of the modalities). In the second stage, the participants were randomized into two groups, 31 students in HF and 39 in LF, in a non-blind approach. After that, randomization was performed to select a case in which the participant would act as a leader and would be evaluated, composing a work team in the remaining 7 cases, according to functions predetermined by the leader. Two rooms were organized, one with a manikin and equipment for infants and the other for children, being prepared according to the LF or HF simulation.

The questions addressed the topics contained in the digitalized material previously made available (topics of pediatric respiratory urgency and emergency). The students filled out a form with their sociodemographic data. On the training day, they were sent to a classroom to answer a pretest consisting of 30 multiple-choice questions (each question was worth 1 point), then they took part in an expository class. The next step was to direct the students to the simulation rooms. Participants randomized into two groups were directed to act as a leader in one case of eight scenarios: severe acute viral laryngotracheobronchitis, foreign body aspiration, moderate acute asthma, severe acute viral bronchiolitis, infectious pneumonia without effusion, complicated pneumonic pleural effusion, benzodiazepine drug intoxication, hydrocephalus with obstruction of the ventriculoperitoneal shunt. Both groups received standard training with a pre-briefing, practice simulated, and a debriefing. While one group was assessed, the other waited in a separate room, without contact, and snacks were provided to reduce the stress of waiting.

In the HF group, the following high-complexity manikins were used: SimBaby® and SimJunior® (Laerdal, USA, 2015), which were designed to help health professionals to recognize and respond effectively to pediatric patients. The scenario was an interactive room, like an emergency room with unidirectional glass, sound, and image control, and with instructors and evaluators outside the simulation room. There was an observation room with unidirectional glass exclusively for the evaluators so that the students could not have eye contact with them. The case instructor remained in another room with similar characteristics. For the LF group, the low complexity Babyboy LM-026M® (Koken, Japan, 2014) and 1005808 W45085® (Gaumard, USA, 2013) manikins were used, without stipulated function. The scenario was unlike HF, a simple classroom without a realism degree, with instructors and evaluators present in the same environment. Cognitive material to aid in activities (scanned handouts, theory testing, expository class, clinical cases for simulation, performance checklist, and cognitive material) was provided for both groups.

The randomly chosen case a scenario for each participant, using the software random.org,²¹ was simulated in a team twice. In both simulations, the participant was individually assessed only as scenario leader, by two independent researchers using a performance checklist. The checklist included determining the following correct actions: team functions, monitoring, high or low flow oxygen, venous access, airway, breathing, types of respiratory and severity disorders, intubation and correct tube size, drugs, circulation, reevaluations, proceedings, and destination. The checklist was individualized according to the case scenario and contained an average of 10 items, to be marked by two evaluators independently.

A posttest with the same 30 multiple-choice questions used in the pretest was applied to the participants after the second simulated practice. A semi-structured questionnaire of participant's satisfaction and self-confidence using the Likert scale of five categories was answered. Figure 1 summarizes the data collection.

Figure 1.

Data collection flowchart. FICF, Free and informed consent form; HF, high fidelity; LF, low fidelity. Source: survey data.

The participant's satisfaction of the expositive class with knowledge of the teacher, and ability to solve doubts; satisfaction and self-confidence in acquiring knowledge about the initial assessment of patients with respiratory diseases, differentiating respiratory distress and failure, and identifying different types of respiratory problems were evaluated. The items evaluated during the simulated practices were the participants' degree of satisfaction and self-confidence in solving their doubts, decision-making, degree of realism of the simulated environment, recognition of the types of respiratory problems, differentiation of respiratory distress and failure, recognition of changes in simulated patients, the challenge provided by the environment, knowledge of team dynamics and the initial treatment of simulated cases.

After three months of training, the epidemiological data of the pandemic did not allow a face-to-face meeting, so a practical online simulation (telesimulation) using the Zoom platform was carried out to measure memory retention with a sample stratum (convenience method) with two teams of five participants in each group. In this practical assessment, the HF and LF participants were analyzed as team leaders to use only the case scenarios of asthma, chosen by lot. Performance evaluation checklists, satisfaction, and self-confidence test were applied, and tele-debriefing was performed.

The data were treated according to the outcomes of the variables. The results were analyzed using the STATA software, and were submitted to normality tests and, as a result, treated as non-parametric. The Kappa method was used to check the agreement between the two experts in the pre and post-training simulation regarding the practical performance of each participant in each group. As the LF theory pretest score does not have a normal distribution, comparisons were made using non-parametric tests and the results were displayed as quartiles. To compare the scores obtained in the theoretical pre and post-test in both groups generalized estimating equations were used to allow comparisons between groups and intra-groups in the same analysis. The p-value was considered 0.05.

An informed consent was requested to utilize the data. The project was approved by two institutions’ ethics and research committees involved, under the code CAAE: 07827319.0.0000.5149 and CAAE: 07827319.0.3001.8107).

Results

The data in Table 1 indicate that the sociodemographic characteristics of the recruited groups are similar (p > 0.05), ensuring the absence of data bias.

Table 1.

Type of capacitation (HF and BF) and student profile according to sociodemographic characteristics.

	HF (n (%))	LF (n (%))	Comparison results^a (p value)
Number of students	31 (44.3)	39 (55.7)	–
FemaleMale	27 (87.1)4 (12.9)	28 (71.8)11 (28.2)	^a (2.402;1)(p = 0.121)
Age group (in years)
20–25	22 (71.0)	28 (71.8)
25–30	5 (16.1)	8 (20.5)	χ^2 (0.707;3)
30–40	3 (9.7)	2 (5.1)	(p = 0.871)
>40	1 (3.2)	1 (2.6)
Monitoring activities
No	15 (48.4)	17 (43.6)	^a (0.160;1)
Yes	16 (51.6)	22 (56.4)	(p = 0.689)
Have scholarship
No	30 (96.8)	32 (82.1)	^a (3.669;1)
Yes	1 (3.2)	7 (17.9)	(p = 0.054)
Takes part in academic league
No	28 (90.3)	38 (97,4)	^a (1.622;1)
Yes	3 (9.7)	1 (2.6)	(p = 0.203)
SAVP Course
No	30 (96.8)	39 (100)	^a (1.276;1)
Yes	1 (3.2)	0 (0)	(p = 0.259)
Scientific initiation activity
No	28 (90.3)	33 (84.6)	^2 (0.502;1)
Yes	3 (9.7)	6 (15.4)	(p = 0.479)

a

Chi square comparison of categorical variables.

The score obtained in the theory test indicated that the data do not present normal distribution and therefore they are presented in the form of median and quartiles. The results indicate that there was an increase in scores in both methodologies (p-value < 0.001 for HF and LF). When the two methodologies are compared between themselves in phase 1 of the study (HF median 16- IQR 4 and LF median 17- IQR 3; p = 0.336) there is no statistical difference, but this difference exists in memory retention (HF median 21 - IQR 4 and LF median 20 – IQR 4; p = 0.043). Therefore, there is an improvement in both groups after applying the methodology, but the HF group, at the end of the process, had better results.

According to the Kappa index, the agreement between evaluators was significant for all items of the checklist of practical tests. The performance of the groups in the practical checklists was better after the second simulation in both groups (p > 0.05). The two evaluators agreed that most participants performed the expected functions for the simulated case scenario including the memory retention phase.

In the satisfaction and self-confidence evaluation using the Likert scale, the participants of both groups attributed high marks to the scanned handout and the expositive class. In the items: delivery on time, didactics, knowledge, and the teacher's ability, acquisition of knowledge on identifying the types of respiratory problems, differentiation between respiratory discomfort (RD) and failure (RF), acquisition of knowledge about working in a team, systematic evaluation and understanding of the text, both the HF and LF group considered themselves as satisfied (>0.05).

The results of the satisfaction and self-confidence tests, in relation to the simulation practices and a hypothetical patient to be treated in the future, are presented in Tables 2 and 3, respectively.

Table 2.

Satisfaction and self-confidence according to Likert scale – face-to-face simulated practices (n = 70) and simulated practice after three months to assess memory retention (n = 20).

Feeling more confident to making decisions
	HF	LF	p value^b
Face-to-face	4.63 (0.19)	4.54 (0.12)	0.684
Memory retention	4.80 (0.19)	4.40 (0.32)	0.285
p value^a	0.537	0.690
Feeling in an environment close to the real one
Face-to-face	4.44 (0.18)	4.16 (0.15)	0.212
Memory retention	4.70 (0.14)	4 (0.37)	0.081
p value^a	0.236	0.684
Feeling more confident to recognize the types of respiratory problems
Face-to-face	4.70 (0.14)	4.34 (0.14)	0.065
Memory retention	4.90 (0.09)	4.50 (0.21)	0.085
p value^a	0.248	0.563
Feeling more confident to differentiate RD from RF
Face-to-face	4.70 (0.16)	4.46 (0.11)	0.217
Memory retention	4.80 (0.13)	4.30 (0.25)	0.072
p value^a	0.627	0.533
Feeling challenged
Face-to-face	4.63 (0.18)	4.74 (0.08)	0.567
Memory retention	5 (0)	4.50 (0.21)	0.018
p value^a	0.042	0.269
Feeling more confident in reporting a similar case
Face-to-face	4.59 (0.17)	4.44 (0.13)	0.460
Memory retention	4.90 (0.09)	4.70 (0.14)	0.248
p value^a	0.085	0.138
Feeling more confident in recognizing changes in clinical conditions
Face-to-face	4.70 (0.16)	4.36 (0.12)	0.093
Memory retention	5 (0)	4.50 (0.25)	0.050
p value^a	0.071	0.634
Acquiring knowledge about teamwork
Face-to-face	4.63 (0.18)	4.51 (0.13)	0.602
Memory retention	4.70 (0.20)	4.78 (0.14)	0.751
p value^a	0.803	0.152
Acquiring knowledge about initial treatment
Face-to-face	4.74 (0.16)	4.46 (0.12)	0.173
Memory retention	4.80 (0.13)	4.30 (0.32)	0.144
p value^a	0.778	0.587

All the analysis was made with GEE.

a

Intra group comparison.

b

Between groups comparison. Data was presented as mean (sd).

Table 3.

Satisfaction study according to Likert scale regarding the feeling triggered in the face of a real case with training (n = 70) and after three months (n = 20).

More confident in making the initial approach
	HF	LF	p value^b
Face-to-face	4.37 (0.17)	4.08 (0.17)	0.215
Memory retention	4.70 (0.14)	4.40 (0.21)	0.239
p value^a	0.143	0.167
More confident in recognizing RD and RF
Face-to-face	4.56 (0.17)	4.41 (0.12)	0.487
Memory retention	5 (0)	4.20 (0.28)	0.004
p value^a	0.008	0.487
Better prepared to assess systematically
Face-to-face	4.52 (0.17)	4.23 (0.15)	0.204
Memory retention	4.80 (0.13)	4.10 (0.26)	0.016
p value^a	0.193	0.678
Better prepared to perform teamwork
Face-to-face	4.63 (0.17)	4.49 (0.14)	0.512
Memory retention	4.70 (0.20)	4.70 (0.20)	0.999
p value^a	0.793	0.339
Better prepared to report a similar case
Face-to-face	4.63 (0.16)	4.54 (0.11)	0.634
Memory retention	4.90 (0.09)	4.80 (0.13)	0.527
p value^a	0.111	0.047

All the analysis was made with GEE.

a

Intra group comparison.

b

Between groups comparison. Data was presented as mean (sd).

Discussion

There is no standardization of instruments for the assessment of diagnostic skills acquired in simulated practices, as assessment materials are created around learning objectives.⁴ Theoretical knowledge is normally measured by a theory test performed right after training or a few months later. The practical skills can be evaluated through performance checklists, satisfaction and self-confidence tests, memory retention, or even team dynamics like task execution time and team mental strain. Our study covered practically all the existing tests for the assessment of learning except memory knowledge and team dynamics.

Nimbalkar et al.²² and Curran et al.²³ used neonatal resuscitation cases for training in simulation and did not observe a statistically significant difference in the scores achieved in the theory posttest after training, between HF and LF participants. Agudelo et al.¹⁵ and Couto et al.²⁴ compared HF with clinical case discussion in pediatric respiratory cases of anaphylaxis and pneumonia. They found higher scores in the posttest, similar between the two groups. In this study, increased scores in both methodologies were observed, therefore the HF group, at the end of the process, had better theory results.

It is the relationship between the content of a test and the construct that is intended to be measured that consolidates the validation of the instrument used.⁴ Schmutz et al. tested the construct of 2 respiratory scenarios and concluded that only reliable and valid performance measures will allow medical educators to accurately evaluate the behavioral effects of training interventions and further enhance the quality of patient care.²⁵ In this study, every didactic and evaluation material was submitted to construct validation, including the performance evaluation checklists. Here, the participants of both groups performed well and improved after the second simulation.

Participants and educators prefer the highest levels of fidelity, believing them to be superior to lower levels, but some authors have found that all levels of fidelity are beneficial when used appropriately.^6,8,12 Meyers et al. performed a single-center, non-randomized, two-group observational pilot trial design using an adult manikin. The authors concluded that high-fidelity simulation improves first-year medical students’ performance and overall satisfaction in teaching pulmonary physiology and suggest future studies to further explore the effectiveness of use in preclinical medical students.²⁶ Participants in our study were fourth-year medical students who, having been trained in severe respiratory disease scenarios, and in face-to-face simulation, considering themselves more challenged, felt more self-confident in recognizing DR and IR, and clinical changes in simulated patients, in HF.

Simulations also allow participants to navigate the group dynamics and experience its effects on teamwork.^11,13 Team Strategies and Tools to Enhance Performance and Patient Safety (STEPPS) is a specialist approach to training health professionals in teamwork. Fagan et al. use professionals with PALS certification and the additional course STEPPS to access the effect on teamwork and report the ability to help a team make decisions that integrate brief team training and positively influence the culture of safety.¹³ A study carried out at our institution evaluating teamwork, by fourth-year medical students, used the STEPPS methodology, in cases of cardiorespiratory arrest in children and also improved performance in team behavior and group technical achievement.²⁷ In our study, the PALS guidelines regarding teamwork were followed, without additional training for such, and in the satisfaction questionnaire there was the question “Do you consider that you learned how to work in a team?” and more than 80% of the students in both groups answered the question positively.

Pediatric simulation scenarios have been shown to be effective as an assessment tool for medical students.²⁵ Despite the high prevalence of respiratory diseases among children, a study showed that medical students do not consider themselves able to deal with severe cases.¹⁵ This study showed that the training was effective as an educational resource and was able to quantitatively assess the two proposed levels of simulation in teaching, addressing topics of severe respiratory diseases in pediatrics.

Abulebda et al. in a cohort study involving simulation and enhanced pediatric readiness found an association with improvement in the processes of care in the general emergency department.²⁸ Spadaro et al. in a randomized trial compared computer-based and mannequin-based approaches for training residents on mechanical ventilation using five different scenarios of acute respiratory failure. They observed a difference regarding the ability to deal with a “real” patient, including observations of continuous variations in clinical parameters, in contrast with the simple interaction with a computer screen, and only the scores of the mannequin group improved significantly [3.0 (2.5–4.0) vs. 2.0 (2.0–3.0), p = 0.005].²⁹ Our study demonstrated, in relation to the hypothetical patient to be treated in the future, better confidence among medical students, to recognize RD and RI, and better preparation for systematic clinical evaluation of the patient, in HF.

For this research, the Covid-19 pandemic made data collection difficult due to the epidemiological moment in our city. Thus, a limiting factor that the authors should highlight is the decrease in the number of participants in the memory retention phase. In addition, there was a change in the assessment format from face-to-face to virtual. However, a systematic review study using randomized clinical trials observed high levels of overall satisfaction (0.60, 95% CI 0.38-0.83; p < .001) significantly favored distance online health education over traditional education.³⁰

Another limitation of this study was the unequal number of participants distributed in each group. This occurred because the participants offered to participate in the training on the dates of their availability and the randomization took place in the cases to be simulated and on the dates of the training type (HF or LF) and not directly between the participants. Finally, despite the inability of the HF manikins to demonstrate accessory muscle usage, they were able to reproduce respiratory auscultation, cyanosis, and lung expansion, facilitating the clinical discernment of respiratory distress and failure.

In this way, our findings allowed us to better understand two learning tools and describe the impacts of HF and LF simulations on the acquisition of diagnostic skills regarding major respiratory problems in pediatrics. An increase in the performance and acquisition of skills of the participants was noticed and both simulations are beneficial for the learning process in question. The authors can conclude also that our study goes beyond satisfaction and self-confidence with the applied technique. It contributes to specific aspects of satisfaction and self-confidence in severe respiratory scenarios. HF simulations lead the participant to feel more challenged and showed benefits regarding self-confidence in recognizing changes in clinical conditions, in recognizing respiratory distress and failure in pediatric cases, and in evaluating the real patients in the future. In addition, participants in this group had better grades in the theory test with better performance in terms of memory retention.

Continued advances in simulation pedagogy illustrate the need for better elucidation and standardization by researchers in future studies of the modality, levels, and types of adopted simulation fidelity so that one can be certain of the influence on the results found. Depending on the learning objective of the proposed case scenario, one level of fidelity may be more suitable than the other.

Funding

This study did not receive any funding.

Acknowledgments

The authors thank the coordinators and collaborators of the Simulation Laboratory at Faculdade de Minas (Faminas-BH), where the study was carried out.