Construction and validation of a questionnaire on the knowledge of healthy habits and risk factors for cardiovascular disease in schoolchildren

Abstract

Objectives

To develop and analyze the reliability and validity of a questionnaire on the knowledge of healthy habits and risk factors for cardiovascular disease (CARDIOKID) to be used in schoolchildren.

Methods

The study included 145 children aged 7 to 11 years. The measured factors were the knowledge of healthy habits and risk factors for cardiovascular disease. Cronbach's alpha and intra-class correlation coefficient (ICC) were used to verify reliability, and exploratory factor analysis was used to assess the validity of the questionnaire.

Results

The sample consisted of 60% females and 40% males. In factorial analysis, the Kaiser-Meyer-Olkin (KMO) test result was measures of sampling adequacy (MSA) = 0.81 and Bartlett's test of sphericity was X² = (66) = 458.64 (p < 0.001). In the factorial analysis with varimax rotation, two dimensions were defined. The “healthy habits” dimension was composed of five factors (ICC = 0.87 and α = 0.93) and the “cardiovascular risk factors” dimension was composed of seven factors (ICC = 0.83 and α = 0.91). In the individual factor analysis, Cronbach's alphas were between 0.93 and 0.91. Total variance was 46.87%. There were no significant differences between test and retest applications.

Conclusion

The questionnaire presented satisfactory validity and reliability (internal consistency and reproducibility), allowing for its use in children.

Resumo

Objetivos

Desenvolver e analisar a fidedignidade e a validade de um questionário sobre conhecimento de hábitos saudáveis e fatores de risco para doenças cardiovasculares (CARDIOKID), para utilização em crianças de idade escolar.

Métodos

Participaram do estudo 145 crianças de 7 sete a 11 anos de idade. Os fatores mensurados foram o conhecimento sobre hábitos saudáveis e fatores de risco para doenças cardiovasculares. O alfa de Cronbach (α) e a correlação intraclasse (CCI) foram utilizados para verificar a fidedignidade, e a análise fatorial exploratória para avaliar a validade do questionário.

Resultados

A amostra foi composta de 60% meninas e 40% meninos. Na análise fatorial, o teste de KMO teve resultado de 0,81 e o teste de esfericidade de Bartlett de X² = (66) = 458,64 (p < 0,001). Na análise fatorial com rotação varimax, foram definidas duas dimensões: a de “hábitos saudáveis” permaneceu com cinco fatores (CCI = 0,87 com α = 0,93), e a de “fatores de risco para doenças cardiovasculares” com sete fatores (CCI = 0,83 α = 0,91). Na análise de cada fator, o valor encontrado dos alfas de Cronbach manteve-se entre 0,93 e 0,91. A variância total manteve-se em 46,87%. Não houve diferença significativa entre as respostas no teste e reteste (p < 0,292).

Conclusão

O questionário demonstrou consistência interna e reprodutibilidade satisfatórias, possibilitando a sua utilização em crianças.

Introduction

The increase in childhood obesity is currently the focus of great global concern, as it is considered an important risk factor for the development of cardiovascular disease in adulthood.¹ It is estimated that 43 million children (35 million in developing countries) are overweight, and that 92 million are at risk of overweight.² In Brazil, data show that in 2008, 20% of children between 5 and 9 years were overweight.² In Rio Grande do Sul, a recent study of children and adolescents demonstrated that 30% of this population was overweight or obese.³ This fact directly contributes to the development of chronic diseases in adulthood.^4–6

A cohort study with 276,835 Danish schoolchildren aged 7 to 13 years showed that children with BMI > 85^th percentile have an increased risk of cardiovascular events in adulthood. In this same scenario, it is worth mentioning that there are other risk factors independent from weight in adulthood. These include smoking, physical inactivity, hypertension, and the individual's quality of life,⁷ factors strongly related to cardiovascular diseases.

With the increase in chronic diseases and the emergence of a proposal based on health promotion in the last 25 years, the concept of health promotion has been thoroughly discussed, and has become a challenge for the scientific community.⁸ Recently published data demonstrated that, in Brazil, 72.4% of the population has some type of chronic illness, of which 31.3% are cardiovascular diseases. Recent studies conducted in Brazil and worldwide with patients with chronic diseases have shown that increased knowledge through educational interventions lead these individuals to better self-care, decreasing the number of hospitalizations.^9–12

However, in a recently published meta-analysis on education and health, based on nutritional interventions and physical activity with obese children, the results indicated that the interventions showed good results concerning blood pressure control and decrease in waist measurements, but some did not show a positive response regarding the prevention of childhood obesity.¹³ In search for solutions to prevent obesity in childhood and adolescence, the self-care stimulus is emphasized; for that, it is necessary to develop tools that are easily applicable in children, which will be directed to verifying what knowledge they have regarding healthy habits and foods that are healthy for the pediatric population.

There are currently several tools that provide valid measures of food and nutrient consumption. However, some children have difficulties in completing some of the tools, due to cognitive problems. Therefore, brief and easily-applicable tools can contribute to the development of preventive strategies and to the evaluation of the effectiveness of intervention programs.

Considering these facts, this study aimed to develop a questionnaire on the knowledge of healthy habits and risk factors for cardiovascular disease, and assess its validity and reproducibility.

Methods

Tool design

The tool was developed in five stages: literature review, selection of variables, expert evaluation, pilot study, and evaluation of psychometric properties. The literature review was the base for the tool development, with the identification of studies on knowledge regarding healthy habits and risk factors for cardiovascular disease.^9,10,14 The tool was constructed in collaboration with health professionals (a pediatric cardiologist, an educational psychologist, a nurse, a nutritionist, and a physical educator).

The first version was sent to three experts in the areas of nursing and education who have extensive experience in maternal and child health, so they could assess the relevance of the items, the language, and the measurement scales used, and provide suggestions for the incorporation of other items.

Subsequently, the pilot study was performed with 38 students who did not participate in the final study. All were elementary school students from public schools in Porto Alegre. The questionnaire was applied in different ways, in order to determine the best technique for performing the data collection. During the questionnaire application, students were instructed to ask for the researcher's help when they had difficulty in understanding the questions. In case of doubt, the researcher talked to the student, clarified the doubt, and wrote down the question number on the tool, as well as the student's doubt, for possible reformulations.

In the final version before the factor analysis, the tool consisted of 14 questions on knowledge of healthy habits and risk factors for cardiovascular disease, distributed on six pages, including questions illustrated with pictures. Some questions had as options for answer: “yes,” “no,” and “do not know,” while others had “good for the heart,” “bad for the heart,” and “do not know,” all illustrated with faces.

One of the questions (number 14) used a picture from a previously validated study, which was adapted for the purposes of the present study, after consent from the authors.

In the final step, a cross-sectional study was applied in the classroom. The sample size calculation for this step was based on the recommendation of Hair et al.,¹⁵ who stated that ten tools should be applied for each question. Thus, in the present study, it would be necessary to apply 140 tools. Considering the number of students in the classrooms, the final sample consisted of 145 students.

First, the questions were read and the children were asked to mark the correct answer on one side, and on the last question, to mark the drawing corresponding to the correct answer.

Each correctly answered question is equivalent to one point, with the score ranging from 0 to 14. To assess measures of test and re-test, the tool was administered on two occasions, with a seven-day interval.

Data were collected in August of 2012, after approval by the ethics committee, under protocol No. 46.35/11.¹⁶ The study was conducted only in schools that agreed to participate. Only children whose parents allowed them to participate and signed an informed consent were enrolled in the study.

Statistical Analysis

Continuous variables were expressed as means and standard deviations. Cronbach's alpha coefficient (α) was used to determine the internal consistency of the tool; values > 0.7 were accepted as valid. The Student's t-test was used to compare the results of the test and re-test. The intraclass correlation coefficient (ICC) was used for the test and re-test analysis of reproducibility, with acceptable values close to 1. The item-total correlation test was used to assess reliability; values > 0.2¹⁵ were considered acceptable.

A number of tests were used to validate the tool. Load factor analysis was performing, establishing the number of factors to be extracted, using load values ≥ 0.3 as parameters. To calculate the sphericity index, Bartlett's test was used with a significance level p < 0.05. The measures of sampling adequacy (MSA) of Kaiser-Meyer-Olkin (KMO) was used to measure sample adequacy. MSA values were considered satisfactory when ≥ 0.60. Finally, scree plot analysis was performed to verify the number of factors that could be extracted.^15,17

Results

The sample consisted of 145 children aged 7 to 11 years, all belonging to a philanthropic institution educational project, some from public schools, and the minority from private schools, of whom 60% were females and 40% were males. The time to complete the questionnaire varied between 30 and 45 minutes. During the questionnaire application, it was observed that the children showed interest in answering it.

The KMO measure of sample adequacy was MSA = 0.81 and Bartlett's test of sphericity was X² = (66) = 458.64 (p < 0.001). Factor analysis with varimax rotation showed that the factors consisted of five items each, in the first dimension, and seven items in the second dimension. In the scree plot analysis, it was observed that the tool could be divided into two or three dimensions. Considering the factor analysis, the final form consisted of two dimensions, with dimension 1 as “healthy habits” and dimension 2 as “risk factors for cardiovascular disease.”

Table 1 shows the load values for different items of the questionnaire. Questions 2 (Do you think that food influences cardiac health?) and 14 (What should be the healthiest way for you to go to school?) were removed from the instrument due to loads < 0.3. During the questionnaire, it was observed that students had difficulty understanding these two questions, which also had the greatest number of blank answers. Thus, the final model had 12 questions.

Table 1.

Factorial analysis of CARDIOKID questionnaire.

Factors	Item	Load
Factor 1	1	0.45
Healthy habits	4	0.78
α = 0.93	3	0.43
	9	0.66
	8	0.52
		% variance 33.46
Factor 2	7	0.75
Risk Factors for cardiovascular diseases	10	0.58
α = 0.91	12	0.69
	5	0.67
	6	0.66
	11	0.64
		% variance 13.41

In the analysis of each factor, Cronbach's alpha values remained between 0.93 and 0.91. The total variance remained at 46.87.

Regarding the reliability of the tool at the test and re-test, an ICC = 0.87 was found, with α = 0.93 for Factor 1, and ICC = 0.83 with α = 0.91 for Factor 2. Table 2 shows the means of correct answers at test and re-test. There was no significant difference between the means of the two tests in any of the dimensions (p = 0.292 for the overall questionnaire, p = 0.123 for dimension 1, and p = 1.0 for dimension 2).

Table 2.

Results of the CARDIOKID questionnaire test and re-test.

	Test		Re-test		p
	Mean	Standard deviation	Mean	Standard deviation
Factor 1	4.22	1.80	4.34	1.81	0.123
Factor 2	5.25	1.23	5.25	1.25	1.000
Total					0.292

Discussion

This validation study demonstrated that the CARDIOKID questionnaire had good internal consistency and reproducibility.

There have been several studies in the literature related to dietary habits of children and adolescents, but very few regarding the knowledge on risk factors for cardiovascular diseases, including nutritional knowledge and knowledge of healthy activities in everyday life. One of the main difficulties of performing research in this area is the lack of validated tools to measure this knowledge in the Portuguese language that are culturally adapted to Brazil.

In other countries, there have been some studies related to the assessment of knowledge, attitudes, and behaviors for cardiovascular disease prevention, such as the Coordinated Approach To Child Health (CATCH) program linked to the Centers for Disease Control and Prevention (CDC) of the United States.^9,18–20

The main strategy of that study was the use of school-based interventions aimed at increasing the participants’ knowledge. Since 1990, this program has been researching what knowledge the students have about nutrition and healthy habits, with good results related to changes in habits and attitudes, and later in weight control of the children participating in the study. In a Brazilian study, there was an association between dietary habits, knowledge and childhood obesity.¹⁰

The validity of a tool is determined by the load value of each item, representing the correlation the variables have among themselves to provide an origin to common factors.^17,21 According to the literature, load values ≥ 0.3 are considered ideal for tool validation. In this study, all items that remained in the tool had values within the suggested range. Regarding internal consistency, Cronbach's alpha measure ranges from 0 to 1, and the minimum value recommended to consider a test as having good internal consistency is > 0.6¹⁵; the present study showed values > 0.9. The analysis of reproducibility showed satisfactory results. The coefficients were higher than those obtained in the study that evaluated the KAB (knowledge, attitudes, and behaviors) questionnaire in Native American children.⁹

The total variance remained at 46.87%. Some authors have suggested that item extraction should continue until the researcher attains at least 60% variance.¹⁵ However, other studies suggest that the criterion of cumulative variance should not be stringently used when extracting items, because analyses that use load values would be enough to define the variables that should be part of the tool.^15,22

Comparing the mean of correct answers between the two applications of test and re-test, the study showed no significant statistical difference. According to the literature, one of the factors to establish construct reliability is the stability of test results, that is, the degree of score accuracy. The more homogeneous the test, the more reliable it is.²³

Regarding the schoolchildren's knowledge, the mean score remained above nine correct answers in 12 questions, i.e., the children have a good knowledge. However, other studies have reported that the schoolchildren's knowledge is very poor regarding nutritional aspects.^10,24

Some authors have studied the reproducibility and validity of a questionnaire aimed at U.S. high-school students, which contained questions about nutrition and physical activity. The productivity analysis results showed more consistent answers between test and re-test when the questions were related to the day before.²⁵

However, it is noteworthy that the CARDIOKID is a tool that aims to determine children's knowledge about healthy habits and risk factors for cardiovascular disease, which minimizes memory-related problems, but represents a more complex cognitive task, as it is related to the children's knowledge. In another study related to the reproducibility of a questionnaire on food consumption illustrated with pictures, researchers found that children in third and fourth grades, i.e., older than 7 years, provide better measures of reproducibility than did the original series.²⁶

Among the limitations of this study is the use of a convenience sample consisting of students from public schools and philanthropic institutions that attended different school shifts, all in the city of Porto Alegre, Brazil. Therefore, the generalization of this study may be limited, and it is necessary to apply this questionnaire again in other contexts. It is emphasized that the literature has shown, to date, no questionnaires assessing knowledge of healthy habits and risk factors for cardiovascular diseases among children, validated for Brazilian children. Therefore, it is believed that the present study is the first to obtain reproducible evidence for schoolchildren aged 7 to 11 years. The use of CARDIOKID in other studies will allow for its improvement and the assessment of its effectiveness in measuring changes in knowledge after educational interventions.

The questionnaire showed satisfactory validity and reliability (internal consistency and reproducibility), and was easy to apply and easy to understand, allowing its use in studies on healthy habits and risk factors for cardiovascular diseases with children of this age range.

Funding

FAPICC (Fundo de Apoio à Pesquisa do Instituto de Cardiologia do RS/FUC à Ciência e a Cultura) and FAPERGS (Fundação de Amparo à Pesquisa do Estado do Rio Grande do Sul)/CAPES (Coordenação de Aperfeiçoamento de Pessoal de Nível Superior).

Conflicts of interest

The authors declare no conflicts of interest.