VALIDATION OF AN EDUCATIONAL BOOKLET: EFFECT ON THE KNOWLEDGE ABOUT PREVENTION OF METABOLIC SYNDROME IN ADOLESCENTS

VALIDACIÓN DE UN CUADERNILLO EDUCATIVO: EFECTO SOBRE EL CONOCIMIENTO ACERCA DE LA PREVENCIÓN DEL SÍNDROME METABÓLICO ENTRE ADOLESCENTES

ABSTRACT

Objective:

to validate an educational booklet for the promotion of knowledge about the prevention of Metabolic Syndrome in adolescents.

Method:

a time series quasi-experiment conducted in 2020 with 37 adolescents from five public schools belonging to the teaching state network of Picos, Piauí, Brazil. It was conducted in two phases: 1) development of a script of questions about the prevention of MS and its content validation by 17 experts; 2) quasi-experiment with reading of the booklet called “Síndrome Metabólica: Como me prevenir?” (“Metabolic Syndrome: How can I protect myself?”, and assessment of the adolescents' knowledge through a script of questions before and after reading the booklet. To analyze content validity of the script of questions, the Content Validity Ratio was calculated (CVR>0.529). The McNemar test (p<0.05) was used to investigate the effect of the educational booklet on the adolescents' knowledge. The study was approved by the Research Ethics Committee of Universidade Federal do Piauí.

Results:

the script of question's overall CVR was 0.718, representing good content validation. Three assertions were removed (CVR<0.529). Regarding the adolescents' knowledge, 45.9% improved it, with no significant difference (p=0.067).

Conclusion:

the adolescents' knowledge was higher in almost half of the participants in the post-test, showing that reading the educational booklet favored knowledge about the prevention of Metabolic Syndrome.

DESCRIPTORS:
Metabolic X Syndrome; Adolescent; Education in health; Educational technology; Validation studies

RESUMEN

Objetivo:

validar el cuadernillo educativo para promover el conocimiento sobre la prevención del Síndrome Metabólico entre adolescentes.

Método:

cuasi experimento de series temporales realizado en 2020 con 37 adolescentes de cinco escuelas públicas pertenecientes a la red estatal de enseñanza de Picos, Piauí, Brasil. Se realizó en dos fases: 1) desarrollo del guion de preguntas sobre prevención del SM y su validación de contenido a cargo de 17 especialistas; 2) cuasiexperimento con lectura del cuadernillo “Síndrome Metabólica: Como me prevenir?” (“Síndrome Metabólico: ¿Cómo puedo protegerme?”), y evaluación del conocimiento de los adolescentes antes y después de la lectura a través del guion de preguntas. Para analizar la validez del contenido del guion de preguntas, se calculó la Relación de Validez de Contenido (Content Validity Ratio, CVR>0,529). Para investigar el efecto del cuadernillo educativo sobre el conocimiento de los adolescentes, se utilizó la prueba de McNemar (p<0,05). El estudio fue aprobado por el Comité de Ética e Investigación de la Universidade Federal do Piauí.

Resultados:

el CVR total del guion fue de 0,718, lo que representa buena validación del contenido del guion de preguntas. Se eliminaron tres afirmaciones (CVR<0,529). En relación con el conocimiento de los adolescentes, el 45,9% presentó mejoras, sin diferencias significativas (p=0,067).

Conclusión:

el conocimiento de los adolescentes fue superior en prácticamente la mitad de los participantes en el post-test, lo que demuestra que leer el cuadernillo educativo favoreció el conocimiento sobre la prevención del Síndrome Metabólico.

DESCRIPTORES:
Síndrome Metabólico X; Adolescente; Educación en salud; Tecnología educativa; Estudios de validación

RESUMO

Objetivo:

validar a cartilha educativa para promoção do conhecimento sobre prevenção da Síndrome Metabólica em adolescentes.

Método:

quase experimento de séries temporais realizado em 2020 com 37 adolescentes de cinco escolas públicas da rede estadual de ensino de Picos-Piauí-Brasil. Foi realizado em duas fases: 1) desenvolvimento de roteiro de questões sobre prevenção da SM e sua validação de conteúdo por 17 especialistas; 2) quase-experimento com implementação de leitura da cartilha “Síndrome Metabólica: Como me prevenir?”, e avaliação do conhecimento dos adolescentes pré e pós-leitura via roteiro de questões. Para análise da validade de conteúdo do roteiro de questões calculou-se a Content Validity Ratio (CVR>0,529). Para investigar o efeito da cartilha educativa no conhecimento dos adolescentes utilizou-se o teste de McNemar (p<0,05). O estudo foi aprovado pelo Comitê de Ética e Pesquisa da Universidade Federal do Piauí.

Resultados:

a CVR total do roteiro foi de 0,718, representando boa validação de conteúdo do roteiro de questões. Três assertivas foram eliminadas (CVR<0,529). Sobre o conhecimento dos adolescentes, 45,9% melhoraram seu conhecimento, sem diferença significativa (p=0,067).

Conclusão:

o conhecimento dos adolescentes foi maior em quase metade dos adolescentes no pós-teste, demonstrando que a leitura da cartilha educativa favoreceu o conhecimento sobre a prevenção da Síndrome Metabólica.

DESCRITORES:
Síndrome X Metabólica; Adolescente; Educação em saúde; Tecnologia educacional; Estudos de validação

INTRODUCTION

Metabolic Syndrome (MS) constitutes a Brazilian and worldwide public health problem, defined by the presence of a set of interrelated risk factors, which include the following: abdominal fat, fasting hyperglycemia, arterial hypertension, low high-density lipoprotein-cholesterol (HDL-c), and hypertriglyceridemia1. For years considered relevant only in adults and aged individuals, it is now frequently present among young people, children and adolescents2^-3.

Currently, the worldwide prevalence values of MS in children and adolescents are 3.3% and 29.2% when associated with overweight and obesity4^-6. In MS, preventive knowledge is essential7. In this context, there is a need for the development and applicability of planned educational interventions, easy to understand and appealing and that draw the population's interest. Thus, printed materials, such as educational booklets, have been an efficient pedagogical resource employed in education in health with adolescents8^-9.

Based on these aspects, Moura (2016)10 elaborated the educational booklet called “Metabolic Syndrome: How can I protect myself”, to instruct adolescents about concepts, risk factors and prevention of MS. This booklet had its content and appearance validated by a number of evaluators and its understanding analyzed with the target population. This study gives continuity to the validation process of the educational material, verifying its effect on the adolescents' knowledge about the prevention of MS. Therefore, the objective was to validate the educational booklet for the promotion of knowledge about the prevention of Metabolic Syndrome (MS) in adolescents.

METHOD

This is a time series quasi-experiment, of the before-and-after type and with a quantitative approach, for external validation of the educational booklet called “Metabolic Syndrome: How can I protect myself?”. The study was conducted in two phases: 1) elaboration and content validation of a script of questions about MS; 2) quasi-experiment, with an intervention using the educational booklet and evaluation of the adolescents' knowledge pre- and post-intervention.

It was carried out in five public schools of the teaching state network from the urban area of the city of Picos, Piauí, from August to November 2020. The teaching state network of the municipality has 17 schools; of these, 10 have 9^th grade Elementary level classes and groups from 1^st to 3^rd year of High School level. Thus, the five schools participating in the study were selected among those 10 institutions via a draw made in the “Gerador aleatório” (Random Generator) app for smartphones.

The population consisted in adolescents aged from 14 to 17 years old, enrolled in the five schools selected in the draw. Choice of this age group was due to being the one contemplated in the semantic analysis process of the educational booklet. To estimate sample size, a formula based on McNemar's Chi-square test, used to analyze frequencies (proportions) of two related samples, was employed11. From such calculation, a number of 38 adolescents was obtained to comprise the sample, which was distributed across the five schools. In each school, it was sought to attain an approximate sample of 10 adolescents, aiming to account for a total of 50 participants. The initial sample consisted in 41 adolescents. Four participants were considered losses, for not finishing the last research stage, which ended with 37 participants.

The following inclusion criteria were defined to participate in the research: adolescents attending school, of both genders, and aged from 14 to 17 years old; and having a cell phone with WhatsApp for contacts with the researcher. The exclusion criterion adopted was adolescents with cognitive and/or hearing deficits or any difficulty that would preclude communication and completion of the research stages. This information was obtained in the first contact with the adolescents' parents or guardians. The discontinuity criterion was non-participation in all the research stages.

The first phase of the study consisted in the elaboration and content validation of a script of questions, which was developed by the study researchers based on the content found in the educational booklet called “Metabolic Syndrome: How can I protect myself?”. It consisted of 20 assertions: 15 true and the rest false, aiming to compare the adolescents' level of knowledge about the topic. The number of assertions elaborated encompassed all the topics covered in the booklet, with one assertive for each topic.

After its elaboration, the script went through the process of content validation by a set of judges composed of different professionals who fit into some of these areas of interest: researchers/professors in the fields of MS, adolescents' health, development and/or validation of instruments, and learning assessment.

To establish sample size for the judges, the Psychometrics concepts were adopted, which recommend from six to 20 experts12. Seventeen (17) expert judges evaluated the script of questions, comprising the sample at this stage of the study. The experts were chosen from the database of the National Council for Scientific and Technological Development (Conselho Nacional de Desenvolvimento Científico e Tecnológico, CNPq); 54 judges were identified, of which only five replied. Thus, the “snowball” sampling technique was also adopted, with the first five evaluators as a starting point. They were asked to nominate other judges who fit the criteria for participation in the study13. Based on this method, 12 judges participated in the research, complementing the research sample.

Due to the need to ensure quality parameters for selection of the experts, the set of requirements for defining judges of teaching and assistance-related content was used, as recommended by Jasper (1994) 14, namely: having skill/knowledge acquired through experience; having specialized skill/knowledge; having special skills in a certain type of study; passing a specific test to identify experts; and having a high rating assigned by some official authority. The judges who met at least 2 of the requirements described were selected, and those who did not answer the invitation or did not return the documents received by the established deadline were excluded.

The evaluators were contacted via email. The email message included an invitation letter and explained the study objective and methodology by means of the specialists' evaluation instructions. In addition to that, the Informed Consent Form (ICF) was forwarded, as well as a copy of the educational booklet in Portable Document Format (PDF) and the link in online format to the content validation instrument for the script of questions, developed from the Google Docs web tool.

In the content validation process of the script of questions, the item rating model was added next to each question, a model that uses a four-point ordinal categorical scale: 1) Not indicative; 2) Not very indicative; 3) Considerably indicative; and 4) Very indicative, about understanding of the educational booklet content15. Options three and four are considered adequate; in addition to that, in each question a field was added for notes and recommendations for changes made by the experts.

Analysis of the elaboration and validation of the script of questions was performed by calculating the Content Validity Ratio (CVR), which is a linear statistic of the proportion of experts that evaluate an item as “essential” 16. The CVR values vary from -1 (total disagreement) to +1 (total agreement); CVR values above zero indicate that more than half of the members agree with an essential item17.

To assess content validity of the questions, the four-point ordinal scale was converted to a three-point scale varying between minus one (-1=not indicative); zero (0=very little indicative); and plus one (+1=considerably or very much indicative). Thus, the items' CVR (CVR-I) was calculated, as well as the total CVR of the instrument (CVR-S), which shows the validity of the entire script and is based on the mean CVR values found per item18. As the number of experts who participated in the study was 17 judges, the critical CVR value (cutoff point) used was 0.529, that is, items with CVR values equal to or higher than this threshold were considered as with good evidence of content validity. Items with lower CVR values were reevaluated or removed17. Changes were also made to the items in the script of questions according to the judges' notes and recommendations.

The second phase of the study consisted of the external validation of the educational booklet with implementation of a quasi-experimental intervention with a single group of adolescents, divided into four stages.

Initially, the researcher contacted the Principal's Office of the five schools drawn by sending email messages and making phone calls, explaining the objectives and procedures to be implemented during the study and requesting the list of students with the contact information of the parents or guardians of the adolescents in each class selected.

With the lists of students, a draw was carried out to define those who would comprise the sample from each school. Immediately after that, first contact with the parents or guardians was established, with the objective of requesting verbal consent for the adolescents to take part in the study. At this time, diverse information about the research, its objectives and the ethical and moral procedures was provided.

Subsequently, first contact with the adolescents was requested, in order to invite them to participate in the study. After their acceptance, the ICF and the Informed Assent Form (IAF) were read and explained and sent via WhatsApp to the participants, as well as the link to the research instruments, the sociodemographic characterization questionnaire and the questionnaire regarding knowledge about the prevention of MS (pre-test). The questionnaires were filled-out online.

Subsequently, the educational booklet was forwarded to each participant in PDF format. Along with the each adolescent, the researcher read the booklet through a video call via WhatsApp or in the Google Meet app. Any and all doubts were subsequently resolved. After that moment, the third stage of the research was explained to the adolescents, which consisted of the intervention through the reading of the booklet by the participants, for a month. In this stage, weekly messages were sent through a WhatsApp group created by the researcher for each class selected, with the objective of encouraging reading of the booklet, totaling four messages that contained the following content:

Good morning/Good afternoon!

My name is Mariana. I am passing by to remind you to read the educational booklet on Metabolic Syndrome. If you already read it or are reading it, confirm here in the group. And, if you have not read it yet, it is important that you do so, according to what has been agreed upon.

One month after the first meeting, the fourth stage of the study took place, consisting in sending the questionnaire regarding knowledge about the prevention of MS (post-test), aiming to assess the participants' knowledge on the subject matter after the educational intervention process with use of the booklet. The stages are illustrated in Figure 1.

Figure 1 -
Stages of the Educational Intervention. Picos, PI, Brazil, 2020.

The data obtained were typed and organized in Microsoft Office Excel. For the data statistical analyses, the R (version 4.0.2) and the Statistical Package for the Social Sciences (SPSS) (version 20.0) software programs were used.

To analyze the data referring to the educational intervention, the adolescents' sociodemographic variables were described. The qualitative variables were summarized by means of absolute (n) and relative (%) frequencies and the quantitative variables, by using measures of central tendency (mean) and dispersion (standard deviation). To observe the effect of the intervention with the use of the booklet on the adolescents' knowledge, the correct answers to each of the 17 questions were considered and, to investigate the effect on overall knowledge, the sum of the questions answered correctly by the adolescents before and after the intervention was considered. Considering that the same sample of adolescents was observed in the pre- and post-tests, characterizing sample pairing, the McNemar's test was used at a 0.05 significance level to investigate the effect of the educational booklet on the adolescents' level of knowledge about the prevention of MS, in percentage terms. The booklet will have a significant effect when the null hypothesis “the booklet exerts no effect at all” is rejected (p-value<0.05). The results obtained were expressed in tables and graphs and discussed against the pertinent literature.

The study was approved by the Research Ethics Committee of Universidade Federal do Piauí. The ethical precepts set forth in Resolution 466/12 of the National Health Council/Ministry of Health (Conselho Nacional de Saúde/Ministério da Saúde, CNS/MS) were respected.

RESULTS

The first version of the script of questions was elaborated with 20 true and false assertions about the concepts, risk factors and prevention measures for MS, contained in the booklet called “Metabolic Syndrome: How can I protect myself?”. After elaboration of the questions, the script went through the content validation stage by 17 experts, most of whom were female (82.4%), nurses (76.4%), and with academic training time from 10 to 20 years (58.8%). In relation to the schooling level, most of them were PhDs (76.4%). The places where the evaluators performed their professional activities were eight Brazilian states: Piauí, Ceará, Pernambuco, Amazonas, Pará, Minas Gerais, São Paulo and Paraná.

The specialists have experience in assistance (58.8%), teaching (82.4%) and research (94.1%) in the following areas of interest: MS, adolescents' health, elaboration and/or validation of instruments, and learning assessment. In addition to that, 94.1% of the specialists have authored scientific publications in one of these areas.

The specialists assessed pertinence of the items, indicating an option about adequacy of the definitions presented in the educational booklet. With this assessment as a starting point, calculation of the CVR for each questions and for the script of questions as a whole was performed (Table 1).

When analyzing the data, it is observed that the calculated CVR for most of the items reached a value higher than 0.529, which meets the minimum threshold recommended by the scientific literature. Five items obtained the minimum CVR of 0.529 (3, 6, 11, 18 and 20): of these, two were reassessed (3 and 6) according to the suggestions made by the evaluators and three (11, 18 and 20) were removed. Even with the assessment of some items as inadequate, the overall CVR value was 0.718, representing good content validation of the instrument as a whole (Table 1).

Table 1 -
Content Validity Ratio (CVR) corresponding to the script of questions about Metabolic Syndrome. Picos, PI, Brazil, 2020. (n=17)

Items		CVR*
1	Metabolic Syndrome is characterized by a set of factors that increase the chances of having heart or blood vessel problems.	1.0
2	A person with Metabolic Syndrome can present accumulation of fat in the abdominal region.	1.0
3	High blood sugar is not a component identified in people with Metabolic Syndrome.	0.412**
4	High blood pressure is a component identified in people with Metabolic Syndrome.	1.0
5	Excess of fat accumulated in the cells is a component identified in people with Metabolic Syndrome.	1.0
6	High good cholesterol (HDL) increases the risk of suffering Metabolic Syndrome.	0.176**
7	A person with a family history of Metabolic Syndrome is also at risk of developing this disease.	0.765
8	The risk of developing Metabolic Syndrome increases if the person leads a sedentary life, without practicing physical activity.	1.0
9	All the individuals who follow an inadequate diet can present more chances of developing Metabolic Syndrome.	0.647
10	Smoking is a risk factor for the development of Metabolic Syndrome.	0.765
11	Metabolic Syndrome is only found in adults and in aged individuals.	0.176†
12	Currently, the number of children and adolescents with Metabolic Syndrome is ever increasing.	0.882
13	The Metabolic Syndrome diagnosis must be made by a health professional, through the evaluation of physical and laboratory exams.	1.0
14	Metabolic Syndrome can be prevented!	0.882
15	Healthy eating prevents the development of Metabolic Syndrome; that is why vegetables, legumes and fruits must be included in the meals.	1.0
16	At all ages, it is fundamental to practice some physical exercise; in the case of adolescents, such practice must be periodic, at least 60 minutes of daily physical activity.	1.0
17	Regular physical activity helps prevent the development of risk factors for Metabolic Syndrome; in addition to that, it also increases self-esteem and helps improve well-being.	1.0
18	The smoking habit causes aesthetics problems, but does not favor the emergence of cardiovascular diseases (heart and blood vessels).	0.059†
19	Excessive alcohol consumption can cause physical and mental malaise, aggressive behavior and, in the long term, a large number of fatal diseases, Metabolic Syndrome among them.	0.647
20	Experiencing persistent stressful moments does not interfere in cardiac diseases (heart and blood vessels) or in the development of Metabolic Syndrome.	-0.059†
Overall CVR*		0.718

• ^*Content Validity Ratio; ^**Items reassessed; †Items removed.

In addition to assessing the pertinence of each question, the evaluators could include notes and recommendations for changes, in case they considered it necessary. Most of the suggestions made by the evaluators dealt with changes in the writing, for the questions to be clearer and more objective.

After content validation and analysis of the notes and recommendations made by the experts, of the 20 assertions included in the script of questions, eleven were reassessed and modified, six remained unchanged, and three were removed. The decision to remove assertions “11”, “18” and “20” was based on the following results: CVR value <0.529 and absence of suggestions for altering the writing of these assertions by the judges. Therefore, the final version of the script of questions consisted of 17 assertions: 15 true and two false.

A total of 37 adolescents finished the educational intervention. Table 2 presents the sociodemographic characterization of the adolescents participating in the study.

Table 2 -
Sociodemographic characterization of the adolescents participating in the research. Picos, PI, Brazil, 2020. (n=37)

Variables	N	%	Mean±SD*
Gender
Female	26	70.3
Male	11	29.7
Age group			15.9±1.0
14-15 years old	13	35.1
16-17 years old	24	64.9
Schooling (years of study)			12.4±1.5
9-10 years	2	5.4
11-12 years	17	46.0
13-14 years	15	40.5
15-16 years	3	8.1
Skin color (self-reported)
White	11	29.7
Black	6	16.2
Asian	3	8.1
Brown	17	45.9
Household arrangement
Parents	34	91.9
Others	3	8.1
Number of people living in the house			4.2±1.2
2-3	11	29.7
4-5	23	62.2
6+	3	8.1
Marital status
Single	35	94.6
Married	1	2.7
Stable union	1	2.7
Occupation in the last 12 months
Only studies	34	91.9
Paid work	3	8.1
Family income			1,791.09±1,702.65
≤ 1 minimum wage**	17	45.9
> 1 minimum wage	20	54.1
Economic class†
Class A	2	5.4
Class B1	0	0
Class B2	9	24.3
Class C1	8	21.6
Class C2	11	29.7
Class D-E	7	18.9

• ^*Standard Deviation; ^**Considering the value of the Brazilian minimum wage in 2020: R$ 1,045.00; ^†National Association of Research Companies (2019).

In relation to the assessment of the intervention effect through reading of the educational booklet, it was noticed that the percentage of correct answers in the post-test was identical or higher in most of the items, although there was no significant difference between the pre-test and the post-test, except in item “11 - Currently, the number of children and adolescents with Metabolic Syndrome is ever increasing”, in which 30 (81.1%) correct answers were observed in the pre-test and 36 (91.1%) in the post-test (Table 3).

Table 3 -
Comparison of the number of correct answers before and after the intervention with the educational booklet about MS. Picos, PI, Brazil, 2020. (n=37)

Item	Pre-test	Post-test	p-value*
1 - Metabolic Syndrome is characterized by a set of factors that increase the chances of having heart or blood vessel problems.	36 (97.3)	36 (97.3)	1.000
2 - A person with Metabolic Syndrome can present accumulation of fat in the abdominal region.	33 (89.2)	33 (89.2)	1.000
3- Low blood sugar is one of the components identified in people with Metabolic Syndrome.	9 (24.3)	10 (27.0)	1.000
4 - High blood pressure is one of the components identified in people with Metabolic Syndrome.	31 (83.8)	35 (94.6)	0.289
5 - Accumulation of fat in the cells is one of the components identified in people with Metabolic Syndrome.	28 (75.7)	29 (78.4)	1.000
6- When high, good cholesterol (HDL) increases the risk of suffering Metabolic Syndrome.	21 (56.8)	18 (48.6)	0.629
7 - Individuals who follow an inadequate diet can present more chances of developing Metabolic Syndrome.	33 (89.2)	37 (100)	0.125
8 - A person with a family history of Metabolic Syndrome is also at risk of developing this disease.	25 (67.6)	27 (73)	0.774
9 - The risk of developing Metabolic Syndrome increases if the person leads a sedentary life, without practicing physical activity.	34 (91.9)	37 (100)	0.250
10 - Not smoking is a preventive measure for Metabolic Syndrome.	31 (83.8)	34 (91.9)	0.453
11 - Currently, the number of children and adolescents with metabolic Syndrome is ever increasing.	30 (81.1)	36 (97.3)	0.031
12 - The Metabolic Syndrome diagnosis must be made by a health professional, through the evaluation of physical and laboratory exams.	36 (97.3)	33 (89.2)	0.375
13 - Metabolic Syndrome can be prevented.	36 (97.3)	37 (100)	1.000
14 - Healthy eating, by including vegetables, legumes and fruits in the diet, prevents the development of Metabolic Syndrome; that is why they must be included in the meals.	34 (91.9)	37 (100)	0.250
15 - In the case of adolescents, the daily practice of physical activity must be periodic, at least for 60 minutes.	34 (91.9)	36 (97.3)	0.625
16 - Periodic practice of physical activity helps to prevent the development of risk factors for Metabolic Syndrome.	35 (94.6)	33 (89.2)	0.625
17 - Excessive alcohol consumption can cause physical and mental malaise, aggressive behaviors and, in the long term, many fatal diseases.	35 (94.6)	36 (97.3)	1.000

• ^*McNemar's exact test, 0.05 significance level.

As for the adolescents' overall knowledge status, 45.9% improved their knowledge about the prevention of MS, 35.1% continued with the same scores, and 18.9% had their scores reduced. However, no statistically significant difference was observed in the improvement of the overall score (p=0.067>0.05) (Table 4).

Table 4 -
Effect of reading the booklet on the level of knowledge. Picos, PI, Brazil, 2020. (n=37)

Situation	N	%	p-value*
It was reduced	7	18.9	0.067
Indifferent	13	35.1
It increased	17	45.9
Total	37	100	-

• ^*McNemar's exact test, 0.05 significance level.

In Figure 2 below, the green dots represent the adolescents who improved their scores, 17 in total, and the red dots represent those who had their scores reduced, 7 adolescents observed. On the straight line are those who did not change their scores even after the intervention with the educational booklet; it is worth noting that there are overlapping points, as there are adolescents who obtained the same score on the pre- and post-tests.

Figure 2 -
Dispersion plot: score in the pre-test vs. score in the post-test. Picos, PI, Brazil, 2020. (n=37)

DISCUSSION

The script of questions was elaborated with diverse information from the educational booklet entitled “Metabolic Syndrome: How can I protect myself?”. Subsequently, its content was validated by 17 specialists from different areas related to the theme assessed. Content validation is an essential stage in the development of measuring instruments. For this process, it is important to have an evaluation by experts with experience and qualification in the instrument's field of knowledge to assess the degree to which each of the instrument's items is valid, reliable and representative19.

To analyze the assessment of the agreement level among the specialists, the CVR values for each question and for the script of questions (overall CVR) were calculated. Use and acceptance of this test was observed in validation studies of instruments for adolescents and in the health area20^-21. Thus, according to the experts, the script of questions showed good content validity, indicating that it contains relevant assertions, which can be used to identify the adolescents’ knowledge before and after reading the educational booklet.

In the content validation of some studies, it is common to perform qualitative procedures, which include leaving room for the judges to contribute with suggestions and/or comments8^,19^,22. For the most part, the assertions included in the script of questions were evaluated as adequate by the experts regarding suitability of the educational booklet content, even though their contributions and recommendations were recorded for further improvement of its applicability to the target population.

The final version of the script of questions constituted the questionnaire on knowledge about MS, used to assess the adolescents' level of knowledge before and after reading the educational booklet. Each of the script's assertions is worth one point; the overall score, obtained by adding up the correct answers, varies from 0 to 17. The higher the score, the higher the adolescents' level of knowledge about MS.

This study compared the adolescents' knowledge about MS before and after reading the educational booklet, aiming to externally validate this care-educational technology. Based on the results, the percentage of correct answers observed in the post-test was identical or higher in most of the assertions, with no significant difference, except in item 11.

Other quasi-experiments that evaluated adolescents' knowledge before and after the implementation of educational technologies, found a positive post-intervention effect, with statistically significant improvements in some items23^-24.

In this study, there were no statistically significant differences between the answers to each of the questions assessed in the pre- and post-test. This result can be explained by the characteristics of the educational booklet that, although developed for adolescents, its designer was inspired by a theory aimed at people with low literacy levels25. The content for people with low literacy levels made it easier to understand most of the assertions in the script of questions.

This aspect contributed to the high level of knowledge in the pre-test and to the scarce or nonexistent difference in the pre-test. These results suggest better planning in the elaboration phase of the educational technology that takes into account the designer, depth of the content, and the literacy level of the target population. Thus, it is inferred that, because of its playful designer characteristics and low literacy requirements, the booklet called “Metabolic Syndrome: How can I prevent myself?” may demonstrate a positive effect on the knowledge about MS in children with up to seven years of schooling.

A study that evaluated the effect of an educational intervention on children's knowledge about healthy habits and risk factors for cardiovascular diseases observed previous knowledge on the theme before the educational interventions, in the intervention and control groups, without presenting any statistical difference. The authors emphasized that these results can be explained because schools and the media have given importance to discussions about proper nutrition and physical activities in improving quality of life and preventing cardiovascular diseases and their risk factors26.

Questions “6”, “12” and “16” presented a reduction in the number of correct answers in the post-test. This result can be justified by the possibility that the participants answered the pre-test correctly at random, which does not express true knowledge on the subject matter. As the questions are dichotomous, they have a 50% chance of being correctly answered at random, which can be a factor that hinders showing the effect of the educational booklet on the adolescents' knowledge about MS.

In relation to the overall status of the adolescents' knowledge level, 45.9% of the adolescents improved their knowledge on aspects related to MS, but no significant difference was observed in the improvement of the total score.

In a study conducted over a longer period of time and in the face-to-face modality, a divergent result was observed in relation to the present study. After an intervention with five EiH meetings related to the risk factors for MS, there was an increase in the level of knowledge after educational interventions, that is, the mean of the late post-test (90 days later) was higher than in the pre-test, with statistical significance (p<0.0001)27. In addition to that, the attitudes of changes after the educational interventions were evaluated, and it was found that 48.8% reported having managed to change eating habits and 62.8% of the students reported having introduced daily physical activity27.

From this perspective, it is noteworthy that educational interventions increase the level of knowledge, contribute to the development of critical and reflective thinking, lead individuals to autonomy, capable of deciding and modifying attitudes with a view to self-care in health28. It is worth emphasizing the relevance of the continuity of the educational strategies, since changes in the life habits, such as adherence to healthy eating and practice of physical activity, occur in the medium- and long-term and result from individual dedication and professional assistance27.

From the current result, it is possible to visualize that, even without significant difference in the statistical analyses, almost half of the adolescents presented improvements in the scores of knowledge about MS after the intervention with the educational booklet. Consequently, development of educational interventions with the use of booklets contributes to improving knowledge of their target populations. In this sense, a number of scholars have developed and validated educational booklets to address issues of importance to adolescents' health, such as prevention of overweight, drug use and sexual violence, among other topics23^,29.

The use of the educational booklet called “Metabolic Syndrome: How can I protect myself?” in this educational intervention did not show a positive effect on increasing the level of knowledge about MS among the adolescents. These results can be related to the short extension in time of the educational intervention, to the reduced number of participants and to remote data survey, which allowed excluding adolescents without Internet access and the occurrence of measurement bias, since this type of research study makes it impossible to control the respondents and the lack of assurance that the participants had previously read and analyzed the questions investigated before sending their answers.

Thus, it is relevant to continue the implementation of the educational booklet and script of questions in new studies, seeking to reach a larger number of adolescents, with the possibility of carrying out educational interventions with face-to-face meetings in the school setting. So that it is possible to obtain more precise results about the effect of the booklet on the adolescents' knowledge, a longer follow-up period is needed, which allows evaluating changes in attitudes and lifestyle.

Considering that the educational booklet presents easy-to-understand language and that no significant difference was found in the adolescents' knowledge, it is also suggested to apply these instruments in people with low literacy in health, such as children and pre-adolescents, so that it is possible to test the effectiveness of the technology in individuals with lower schooling levels than adolescents.

CONCLUSION

It was verified that, in the post-test, knowledge was better in almost half of the adolescents, although without a statistically significant difference. Thus, the intervention with the use of the educational booklet showed no positive effect on increasing knowledge about MS among the adolescents, except for one item in the script of questions.

The script of questions was elaborated and validated by experts for its application specifically in this study, and should undergo other evaluation processes for application in different studies on adolescents' knowledge about MS.

Even with the limitations, this study allowed reaching adolescents in a difficult scenario for the conduction of research studies, which was the pandemic, and addressed important issues regarding health promotion and prevention of MS, increasing the participants' knowledge on the theme.

It is noted that it is possible to develop and validate a script of questions that allows evaluating health education actions with adolescent students about aspects related to MS and to guide the elaboration and validation of new educational technologies about prevention and adoption of healthy lifestyle habits among adolescents.

REFERENCES

• 1. Alberti KG, Eckel RH, Grundy SM, Zimmet PZ, Cleeman JI, Donato KA, et al. Harmonizing the metabolic syndrome: a joint interim statement of the International Diabetes Federation Task Force on Epidemiology and Prevention; National Heart, Lung, and Blood Institute; American Heart Association; World Heart Federation; International Atherosclerosis Society; and International Association for the Study of Obesity. Circulation [ Internet]. 2009 [cited 2021 Aug 12]; 120(16):1640-5. Available from: https://doi.org/10.1161/CIRCULATIONAHA.109.192644
  » https://doi.org/10.1161/CIRCULATIONAHA.109.192644
• 2. Carvalho CA de, Fonseca PCA, Barbosa JB, Machado SP, Santos AM dos, Silva AAM da. The association between cardiovascular risk factors and anthropometric obesity indicators in university students in São Luís in the State of Maranhão, Brazil. Ciênc Saúde Coletiva [Internet]. 2015 [cited 2020 Jun 16]; 20(2):479-90. Available from: https://doi.org/10.1590/1413-81232015202.02342014
  » https://doi.org/10.1590/1413-81232015202.02342014
• 3. Kuschnir MCC, Bloch KV, Szklo M, Klein CH, Barufaldi LA, Abreu GA, et al. ERICA: prevalence of metabolic syndrome in Brazilian adolescents. Rev Saúde Pública [Internet]. 2016 [cited 2019 Jul 17]; 50(1 Suppl 1):11s. Available from: https://doi.org/10.1590/s01518-8787.2016050006701
  » https://doi.org/10.1590/s01518-8787.2016050006701
• 4. Friend A, Craig L, Turner S. The prevalence of metabolic syndrome in children: a systematic review of the literature. Metab Syndr Relat Disord [Internet]. 2013 [cited 2020 May 30]; 11(2):71-80. Available from: https://doi.org/10.1089/met.2012.0122
  » https://doi.org/10.1089/met.2012.0122
• 5. Précoma DB, Oliveira GMM, Simão AF, Dutra OP, Coelho OR, Izar MCO, et al. Atualização da Diretriz de Prevenção Cardiovascular da Sociedade Brasileira de Cardiologia-2019. Arq Bras Cardiol [Internet]. 2019 [cited 2020 Dec 16]; 113(4):787-91. Available from: http://publicacoes.cardiol.br/portal/abc/ português /2019/ v11304/pdf/11304022.pdf
  » http://publicacoes.cardiol.br/portal/abc/ português /2019/ v11304/pdf/11304022.pdf
• 6. Bezerra MRE, Lyra MJ, dos Santos MAM, Colares V, de Menezes VA. Fatores de risco modificáveis para doenças crônicas não transmissíveis em adolescentes: Revisão Integrativa. Adolesc Saúde [Internet]. 2018 [cited 2020 Jul 19]; 15(2):113-20. Available from: http://adolescenciaesaude.com/detalhe_artigo.asp?id=726
  » http://adolescenciaesaude.com/detalhe_artigo.asp?id=726
• 7. Malta DC, Morais Neto OL, Silva Junior JB. Apresentação do plano de ações estratégicas para o enfrentamento das doenças crônicas não transmissíveis no Brasil, 2011 a 2022. Epidemiol Serv Saúde [Internet]. 2011 [cited 2019 Sep 7]; 20(4):425-38. Available from: http://scielo.iec.gov.br/pdf/ess/v20n4/v20n4a02.pdf
  » http://scielo.iec.gov.br/pdf/ess/v20n4/v20n4a02.pdf
• 8. Moura JRA, Silva KCB da, Rocha AESH, Santos SD dos, Amorim TRS, Silva ARV da. Construction and validation of a booklet to prevent overweight in adolescents. Acta Paul Enferm. [Internet]. 2019 [cited 2019 Sep 15]; 32(4):365-73. Available from: https://doi.org/10.1590/1982-0194201900051
  » https://doi.org/10.1590/1982-0194201900051
• 9. Soares GM, Zangerolamo L, Rosa LRO, Branco RCS, Carneiro EM, Barbosa-Sampaio HC. Impact of a playful booklet about diabetes and obesity on high school students in Campinas, Brazil. Adv Physiol Educ [Internet]. 2019 [cited 2021 Aug 12]; 43(3):266-9. Available from: https://doi.org/10.1152/advan.00160.2018
  » https://doi.org/10.1152/advan.00160.2018
• 10. Moura IH, Silva AFR, Rocha AESH, Lima LHO, Moreira TMM, Silva ARV. Construction and validation of educational materials for the prevention of metabolic syndrome in adolescents. Rev Latino-Am Enfermagem [Internet]. 2017 [cited 2021 Aug 13]; 25:e2934. Available from: https://doi.org/10.1590/1518-8345.2024.2934
  » https://doi.org/10.1590/1518-8345.2024.2934
• 11. Arango HG. Bioestatística: teórica e computacional. 3rd ed. Rio de Janeiro, RJ(BR): Guanabara Koogan; 2009.
• 12. Pasquali L. Psicometria: teoria e aplicações. Brasília, DF(BR): Universidade de Brasília; 1997.
• 13. Polit DF, Beck CT. Fundamentos da pesquisa em enfermagem: avaliação de evidências para a prática de enfermagem. 9th ed. Porto Alegre, RS(BR): Artmed; 2018.
• 14. Jasper MA. Expert: a discussion of the implications of the concept as used in nursing. J Adv Nurs [Internet]. 1994 [cited 2019 Jul 20]; 20(4):769-76. Available from: https://doi.org/10.1046/j.1365-2648.1994.20040769.x
  » https://doi.org/10.1046/j.1365-2648.1994.20040769.x
• 15. Pasquali L. Princípios de elaboração das escalas psicológicas. Rev Psiquiatr Clínica. 1998; 25(5):206-13.
• 16. Lawshe CH. A quantitative approach to content validity. Pers Psychol [Internet]. 1975 [cited 2021 Aug 20]; 28: 563-75. Available from: https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.460.9380&rep=rep1&type=pdf
  » https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.460.9380&rep=rep1&type=pdf
• 17. Ayre C, Scally AJ. Critical Values for Lawshe’s Content Validity Ratio: revisiting the original methods of calculation. Meas Eval Couns Dev [Internet]. 2014 [cited 2021 Aug 20]; 47(1):79-86. Available from: https://doi.org/10.1177%2F0748175613513808
  » https://doi.org/10.1177%2F0748175613513808
• 18. Ortiz La Banca R, Rebustini F, Alvarenga WA, de Carvalho EC, Lopes M, Milaszewski K, Nascimento LC. Checklists for assessing skills of children with type 1 diabetes on insulin injection technique. J Diabetes Sci Technol [Internet]. 2021 [cited 2021 Aug 20]; 9:1932296820984771. Available from: https://doi.org/10.1177%2F1932296820984771
  » https://doi.org/10.1177%2F1932296820984771
• 19. Alexandre NMC, Coluci MZO. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Cienc Saude Coletiva [Internet]. 2011 [cited 2019 Sep 19]; 16(7):3061-8. Available from: https://doi.org/10.1590/S1413-81232011000800006
  » https://doi.org/10.1590/S1413-81232011000800006
• 20. Mousazadeh S, Rakhshan M, Mohammadi F. Investigation of Content and Face Validity and Reliability of Sociocultural Attitude towards Appearance Questionnaire-3 (SATAQ-3) among Female Adolescents. Iran J Psychiatry [Internet]. 2017 [cited 2021 Aug 19];12(1):15-20. Available from: https://www.ncbi.nlm.nih.gov/pubmed/28496497
  » https://www.ncbi.nlm.nih.gov/pubmed/28496497
• 21. Zare E, Simbar M, Shahhosseini Z, Alavi MH. Design and Psychometric Properties of Male Adolescent Health Needs-Assessment Scale. Int J Community Based Nurs Midwifery [Internet]. 2016 [cited 2021 Aug 19]; 4(4):297-308. Available from: https://www.ncbi.nlm.nih.gov/pubmed/27713893
  » https://www.ncbi.nlm.nih.gov/pubmed/27713893
• 22. Martins FDP, Pontes CM, Javorski M, Gomes LF, Barros ACR de, Leal LP. Design and validation of an evaluation instrument on knowledge of schoolchildren about breastfeeding. Acta Paul Enferm [Internet]. 2017 [cited 2020 Jun 1];30(5):466-78. Available from: https://doi.org/10.1590/1982-0194201700068
  » https://doi.org/10.1590/1982-0194201700068
• 23. Praxedes MLS, Queiroz MVO, Vieira RP. Effectiveness of an educational game on contraception with adolescent students: a quasi-experimental study. Online Braz J Nurs [Internet]. 2019 [cited 2021 Jan 11]; 18(4). Available from: https://doi.org/10.17665/1676-4285.20196184
  » https://doi.org/10.17665/1676-4285.20196184
• 24. Uchôa MALF. Validação clínica de uma tecnologia em saúde: efeito no conhecimento sobre prevenção da obesidade em adolescentes escolares [dissertação]. Fortaleza (CE), Universidade Estadual do Ceará; 2019.
• 25. Doak CC, Doak LG, Root JH. Teaching patients with low literacy skills. 2th ed. Philadelphia, PA(US): J.B. Lippincott; 1996.
• 26. Cecchetto FH, Pena DB, Pellanda LC. Playful interventions increase knowledge about healthy habits and cardiovascular risk factors in children: The CARDIOKIDS Randomized Study. Arq Bras Cardiol [Internet]. 2017 [cited 2021 Feb 2];109(3):199-206. Available from: https://doi.org/10.5935/abc.20170107
  » https://doi.org/10.5935/abc.20170107
• 27. Cortez RMA, Batista AMO, Rocha MR da, Santos IMS, Moura JRA, Silva ARVda. Students’ knowledge of metabolic syndrome after educational intervention. Rev Bras Enferm [Internet]. 2018 [cited 2019 Sep 15]; 71( 4 Suppl):1493-9. Available from: https://doi.org/10.1590/0034-7167-2017-0342
  » https://doi.org/10.1590/0034-7167-2017-0342
• 28. Souza VB, Silva JS, Barros MC, Freitas PSP. Soft technologies in health to potentize the quality of care to pregnant women. Rev Enferm UFPE online [Internet]. 2014 [cited 2021 Jan 11]; 8(5):1388-93. Available from: https://doi.org/ 10.5205/1981-8963-v8i5a9824p1388-1393-2014
  » https://doi.org/ 10.5205/1981-8963-v8i5a9824p1388-1393-2014
• 29. Moura MIL de, Leal JB, Leal JB, Correia VGA, Leal JBP, Silva MG, et al. First reader on prevention of drug use for teens. J Nurs UFPE online [Internet]. 2019 [cited 2021 Jan 13]; 13(4):1106-14. Available from: https://doi.org/10.5205/1981-8963-v13i04a237875p1106-1105-2019
  » https://doi.org/10.5205/1981-8963-v13i04a237875p1106-1105-2019

NOTES

• ORIGIN OF THE ARTICLE

  Extracted from the dissertation - Clinical validation of an educational booklet about knowledge of the preventive actions for Metabolic Syndrome among adolescents, presented at the Graduate Program in Health and Community of Universidade Federal do Piauí, in 2021.

• APPROVAL OF ETHICS COMMITTEE IN RESEARCH

  Approved in the research Ethics Committee of Universidade Federal do Piauí, Senador Helvídio Nunes de Barros Campus, opinion No. 3,644,984/2019, Certificate of Presentation for Ethical Appreciation No. 21259119.0.0000.8057.

Edited by

EDITORS

Associated Editors: Natália Gonçalves, Monica Motta Lino. Editor-in-chief: Roberta Costa.

Publication Dates

• Publication in this collection
  07 Feb 2022
• Date of issue
  2022

History

• Received
  24 Mar 2021
• Accepted
  19 Oct 2021