Validation for the Portuguese language of the Educational Practices Questionnaire (Student Version)

Abstract

Objective

To translate and validate the Educational Practices Questionnaire (Student Version) for the Portuguese language.

Methods

Methodological instrument translation and validation study. For the validation process, the event: III Workshop Brazil – Portugal: Care for Critical Patients was set up.

Results

103 nurses participated in the research. The psychometric tests (scale validity and reliability, correlation pattern between the variables, goodness-of-fit test of the sample and sphericity) presented satisfactory results. The clusters found in the factor analysis were not in accordance with the literature. Therefore, the division the original authors had made was followed.

Conclusion

The scale was called: Questionário de Práticas Educativas. The findings demonstrated good psychometric properties and suitable potential use. Further research is needed to consolidate the questionnaire and expand its dimensionality.

Nursing education research; Education, nursing; Simulation; Validation studies; Questionnaires

Resumo

Objetivo

Traduzir e validar para língua portuguesa o Educational Practices Questionnaire (Student Version).

Métodos

Estudo do tipo metodológico, de tradução e validação de instrumento. Para o processo de validação criou-se o evento: III Workshop Brasil – Portugal: Atendimento ao Paciente Crítico.

Resultados

Participaram da pesquisa 103 enfermeiros. Os testes psicométricos (validade e fidelidade da escala, o padrão de correlação entre as variáveis, o teste de adequação amostral e o teste de esfericidade) apresentaram resultados satisfatórios. Os agrupamentos encontrados na análise fatorial não apresentaram coerência com a literatura estudada. Assim, optou-se em seguir a divisão estabelecida pelos autores originais.

Conclusão

A escala foi denominada: Questionário de Práticas Educativas. Os achados demonstraram boas propriedades psicométricas e um adequado potencial de uso, todavia futuras pesquisas se fazem necessárias para consolidação desse questionário, bem como a expansão de sua dimensionalidade.

Pesquisa em educação de enfermagem; Educação em enfermagem; Simulação; Estudos de validação; Questionários

Introduction

Educational practices permeate any and all teaching method and strategy. In 1987, a set of seven principles was defined(1) for good educational practices that considerably contribute to the learning process. This definition received support from the American Association of Higher Education (AAHE) and the American Association of Colleges of Universities (AACU) and consists of the following principles: 1) Encouragement of student-teacher contact: The relation established between student and tutor in and beyond the school environment should be considered as a motivational factor, which encourages the student to think about his own values and future plans; 2) Encouragement of cooperation among students: Good learning happens in a collaborative and social manner, not in a competitive and isolated manner. The involvement among students permits the sharing of ideas and improvements in knowledge construction; 3) Encouragement of active learning: The way the teacher teaches should be meaningful to the students and, therefore, the students’ past experiences and daily reality should be taken into account, so that they can relate theory and practice. The education process should go beyond the mere distribution of contents from teacher to students, expecting them to simply listen and memorize; 4) Supply of immediate feedback: the students need to be aware of their education process. The teacher should provide appropriate information on the students’ performance in due time to allow them to reflect on their actions and use the resources needed to construct their knowledge; 5) Emphasis on the time to study: the teacher should advise the students on how to manage the time to study and the importance of developing the study habit, among other daily activities; 6) Establishment of high expectations: both teacher and student should be motivated to teach and learn. Both should hold high expectations in their actions for the teaching-learning process to happen in an environment that favors knowledge exchange; 7) Respect for different talents and forms of learning: many ways exist to learn. Each individual has his/her own knowledge construction characteristics. Therefore, these particularities should be valued and encouraged in the school environment.(¹)

In the light of the theoretical framework on simulation in nursing education,(²,³) simulated teaching consists of five elements: Teacher, Student, Educational Practices, Characteristics of the Simulation and Results. All of these factors are interconnected in the learning process through the clinical simulation, which encourages the teacher-student relation, encourages teamwork, rescues theoretical contents and experiences, motivates learning, provides immediate feedback through the debriefing, respects and values the student’s learning. This feedback, provided in a positive manner to students in the early phase of their professional life, contributes to the valuation of individuals’ clinical training, entailing greater satisfaction with learning.(⁴)

Nursing teaching through clinical simulation allows individuals to experience a wide range of professional practice contexts, ranging from highly stressful contexts like death to very happy ones like life.(⁵)

In the nursing profession, care integrality is one of the guiding axes of education, and requires a pedagogical proposal that encourages the students to reflect on aspects of health practice, on their assessment process as a tool that helps to acknowledge their shortages, with a view to internalization and further correction. The understanding of the learning process should be perceived as something that involves both knowledge, skills and attitudes, as spaces for reflection on the subjects’ reality.(⁶) In this conception, clinical simulation demands clinical reasoning from the students, reflexive thinking that makes them take decisions, one of the core skills for health professionals and mainly among nurses. Fast and agile clinical reasoning contributes significantly to successful professional interventions, to the quality of care delivery, which can positive or negatively have a direct impact in the patient’s life. In that sense, simulation permits competency building for nursing care through the combination of knowledge, skills and wanting to act, knowing how to act and being able to act (attitudes).(⁷,⁸)

To understand how individuals who participate in high-fidelity simulation perceive the educational practices, the National League for Nursing (NLN) - an organization that strives for excellence in nursing education – developed the Educational Practices Questionnaire (Student Version), a 16-item tool with two subscales (one related to educational practices and the other to the importance attributed to the item). The tool is divided in four factors: 1) Active learning, 2) Collaboration, 3) Diverse ways of learning and 4) High expectations. Answers are provided on a five-point Likert scale, including the option “not applicable” when the declaration is not relevant for the simulation performed. The validation study of the questionnaire involved 395 students, including 350 women and 45 men, with an average age of 29 years. The reliability using Cronbach’s alpha corresponded to 0.86 for the scale of characteristics of the educational practices and 0.91 for the importance of the items.(⁹)

Recent studies have appointed the need for further research on the experience of individuals who use simulation as a teaching strategy with a view to better exploring its potential,(¹⁰) and optimizing its use in learning(¹¹)

Thus, the objective in this study was to translate and validate this American tool for the Portuguese language, which can measure characteristics of the educational practices in clinical simulation; and then publish it to contribute to the advancement of research involving simulation as a teaching strategy.

Methods

The first phase in this study was the translation of the tool. The criterion adopted followed the proposal by Ferrer et al.(¹²) Two teachers experts in both languages translated the tool to Portuguese, after which a first consensus version was obtained in Portuguese. This version was submitted to an expert committee invited to participate: seven nurse experts from the field of nursing fundamental, all of whom were knowledgeable on simulation as a teaching strategy. Only four of them attended the meeting to analyze the tool. After informing on the research objective, the experts formalized their consent by signing the free and informed consent form. The questionnaire items were classified as valid or non-valid through the calculation of the Content Validity Index (CVI);(¹³) for items with a CVI between 100% and 80%, the translation was maintained in the final version of the tool; the language of items with a CVI inferior to 80% was modified. At the end of this process, the tool was submitted to back translation by two teachers, one with expertise and the other native in the English language. After reaching a consensus on the back-translated version, it was compared with the original version, showing that the meaning of the items was not changed. Next, the semantic validation was undertaken, as well as a pretest with ten undergraduate students who had already engaged in simulation as a teaching strategy. The students were invited to manifest their doubts and considerations. This process happened regularly until reaching the final version of the questionnaire.

In the second phase, the tool was validated. For this phase, the III Workshop Brazil - Portugal: care delivery to critical patients was set up, promoted by a Brazilian educational institution in partnership with an educational institution from Portugal. To participate in this event, nurses were invited, professionally active or not, holding any post-graduation degree or not, with of without past experience with simulation as a teaching strategy. The event was offered free of charge and disseminated in the print and electronic media, with 180 places for registration on the website of the institution. The candidates could choose among three days to participate in the event; that is, each day of the workshop was reserved for 60 participants. All places were occupied in advance. The candidates received material for background study by e-mail. Among the 180 registrations, 103 actually attended the event. The workshop included a lecture on care delivery to critical patients and simulation, taught by faculty members from Brazil and Portugal with expertise in the area. The participants in this event were invited to take part in the research and manifested their acceptance by signing the free and informed consent form. To characterize the participants, a tool was developed with the following variables: age, sex, year of graduation from undergraduate program, years of experience, data on education, employment and experience with simulated teaching. After the theoretical content, still in the morning period, the participants were divided in three groups, each of which attended three skills training workshops. Low, medium, high-fidelity simulation and role play were used as teaching strategies throughout the event. In the afternoon, each group went through three different clinical simulations, in which each scenario involved care delivery to critical patients in a certain situation and covered the background study material, the theoretical content and skills training. After going through all phases, the participants again met in an auditorium and completed the Questionário de Práticas Educativas. Then, the data were coded in Excel worksheets and analyzed using Statistical Package for the Social Sciences-SPSS (version 22 for Windows). For all tests, statistical significance was set as p<0.05.

The study was registered on Plataforma Brasil under Certificado de Apresentação para Apreciação Ética (CAAE): 10551512.1.0000.5393.

Results

In the first phase of the study, the CVI for some items was inferior to 80%, so that their language was modified. In items 4 and 13, the word “didactic” was added to specify the material the tool refers to. In items 8 and 16, the word “instructor” was replaced by “teacher”, as the roles in the simulated environment in the United States differ from the roles in simulated environments in Brazil and Portugal. In these countries, in general, a single person, i.e. the teacher, serves as a teacher, instructor and facilitator. After this adaptation, the rest of the process took place regularly.

The final version of the scale has been described in chart 1.

Chart 1
Description of the items in the Questionário de Práticas Educacionais

Item
Factor 1) Active learning
1. During the simulation activity, I was able to discuss the ideas and concepts taught in the course with the teacher and other students.
2. I participated actively in the debriefing session after the simulation.
3. I was able to reflect further on my comments during the debriefing session.
4. There was sufficient opportunity during the simulation to discover if I clearly understood the didactic material.
5. I learned from the teacher’s comments before, during or after the simulation.
6. I got clues during the simulation in due time.
7. I had the chance to discuss the objectives of the simulation with my teacher.
8. I was able to discuss ideas and concepts taught during the simulation with my teacher.
9. The teacher was able to respond to the students’ individual needs during the simulation.
10. The use of simulation activities made my learning time more productive.
Factor 2) Collaboration
11. I was able to work with my colleagues during the simulation.
12. During the simulation, my colleagues and I had to work together in clinical practice.
Factor 3) Diverse ways of learning
13. The simulation offered different ways to learn the didactic material.
14. This simulation offered various forms to assess my learning.
Factor 4) High expectations
15. The objectives of the simulated experience were clear and easy to understand.
16. My teacher informed on the objectives and expectations to be achieved during the simulation.

In the second phase, 103 (100%) subjects participated in the workshop who agreed to take part in the research and were included in the study sample. Most participants, 90 (87.4%), were female, with an average age of 32.1 years. As regards education, on average, the participants had graduated in 2005. Twenty (19.4%) participants had not taken any type of specialization course and the remainder was either taking a course or had concluded the following course modalities: 64 (62.1%) Lato Sensu specialization, 47 (45.7%) Master’s and 20 (19.4%) Ph.D. What employment is concerned, 77 (74.8%) had a fixed job, being 48 (46.6%) in clinical nursing, 23 (22.3%) in teaching and six (5.8%) in service management. Concerning the experience in simulated teaching, 52 (50.5%) reported that they were not familiar with simulation as a teaching strategy, while 51 (49.5%) indicated familiarity with the tool.

With regard to the validity and reliability of the scale, the correlation pattern between the variables showed 36.3% (96) of correlations superior to 0.30. The goodness-of-fit of the sample, verified using the Kaiser-Meyer-Olkin test, corresponded to 0.81; Bartlett’s sphericity test <0.001 and anti-image matrix coefficients between 0.67 and 0.91.

In the factor extraction analysis, the total explained variance showed five eigenvalues superior to 1.00, with a cumulative percentage that explains more than 72% of the total variance, suggesting that the scale could be divided in five factors.

In the factor rotation, the commonality test demonstrated coefficients superior to 0.54 for all factors, strengthening the possible extraction of the factors; these procedures followed the analysis model the original authors had used: exploratory factor analysis with varimax rotation. The results of the analysis in this study were very different from the findings of the original version, without a link between the clusters. Therefore, the division established in the original version was followed.

To verify the internal consistency, Cronbach’s alpha was used, with the following coefficients: active learning 0.86, collaboration 0.87, diverse ways of learning 0.77, high expectations 0.70 and, for the general scale 0.90.

Table 1 presents descriptive statistical coefficients for the Questionário de Práticas Educativas and the Scale of Importance of the Item.

Table 1
Descriptive statistics of the factors of the Questionário de Práticas Educacionais and the Scale of Importance of the Item (n = 103)

Questionário de Práticas Educacionais
Variables	M*	M**	Mean	SD
Active learning	3.60	5.00	4.56	0.41
Collaboration	2.50	5.00	4.66	0.52
Diverse ways of learning	3.50	5.00	4.64	0.47
High expectations	2.00	5.00	4.58	0.59
General	3.56	5.00	4.59	0.39
Scale of Importance of the Item
Variables	M*	M**	Mean	SD
Active learning	3.80	5.00	4.80	0.31
Collaboration	3.50	5.00	4.86	0.35
Diverse ways of learning	3.50	5.00	4.85	0.33
High expectations	3.00	5.00	4.84	0.36
General	3.75	5.00	4.82	0.29

• *Minimum; **Maximum; SD- Standard deviation

Pearson’s correlation coefficient of the Questionário de Práticas Educacionais and the Scale of Importance of the Item have been described in table 2.

Table 2
Pearson’s correlation coefficient of Questionário de Práticas Educacionais and Scale of Importance of the Item (n = 103)

Questionário de Práticas Educacionais
Variables	Factor 1	Factor 2	Factor 3	Factor 4	General Scale
Active learning		0.48	0.59	0.64	0.96
Collaboration	0.48		0.47	0.42	0.65
Diverse ways of learning	0.59	0.47		0.42	0.71
High expectations	0.64	0.42	0.42		0.75
General scale	0.96	0.65	0.71	0.75
Scale of Importance of the Item
Variables	Factor 1	Factor 2	Factor 3	Factor 4	General Scale
Active learning		0.67	0.73	0.73	0.97
Collaboration	0.67		0.72	0.53	0.78
Diverse ways of learning	0.73	0.72		0.56	0.82
High expectations	0.73	0.53	0.56		0.79
General scale	0.97	0.78	0.82	0.79

• Significant correlation at 0.01

Discussion

Among the educative practices, high-fidelity simulation enriches the teaching-learning process, being a relevant strategy that can be implemented in the curriculum of educational institutions, as it consolidates this process and enhances the students’ competences and skills.(¹⁴) The potential use of the scale in health institutions should also be highlighted in the continuing education process of their human resources, as professional qualification is fundamental for effective and high-quality healthcare.

The Portuguese version of the Educational Practices Questionnaire (Student Version) was called: Questionário de Práticas Educativas. The name Student Version was removed as the tool can be applied to any and all individuals who engage in a learning process.

The results of the psychometric tests appointed a high correlation between the variables and the fitness of the sample to develop the study. The total explained variance signaled that the scale could be divided in five factors, differently from the original scale. In the exploratory factorial analysis with varimax rotation, the items were grouped in a rather unexpected manner; as there was no logical explanation, the findings of the original study were followed. One justification for this unexpected grouping can be related to the characteristics of the sample studied. The validation study of the original version was developed with undergraduate nursing students, while the present study involved nursing professionals. The composition of the group studied influences the factor analysis: the more heterogeneous a sample is, the higher the correlations between the test scores.(¹⁵) In view of the great heterogeneity of the study sample and the high correlations in the test, the scale factors should be better assessed in subsequent studies, either involving students or professionals.

Another source of influence this study may have undergone is related to the sample size: in the literature,(¹⁶) it is suggested that, for the sake of factor analysis, the sample should contain at least five participants per variable and, in total, at least 200 subjects. Others(¹⁷) recommend using ten subjects per variable and at least 100 subjects in total, or argue(¹⁸) that the desired sample size depends on the size of factor loadings obtained, around 0.80. On the other hand, some sources(¹⁹) classify samples of 50 individuals as very small, of 100 as small, of 200 as reasonable, of 300 as good, of 500 as very good and of 1,000 or more as excellent. Also, as a general role, having at least 100 subjects per factor measured is recommended.(²⁰) Without a consensus on the sample size, in the future, other study using the Questionário de Práticas Educativas can better clarify the factor division of the scale, whether involving undergraduate students or nursing professionals.

What the internal consistency is concerned, the results found are higher than the findings for the original scale, for the Questionário de Práticas Educativas as well as for the Scale of Importance of the Item, reasserting the coherence of the scales. The internal consistency of the four factors was also good in both scales, except for factor four, with a lower consistency in both scales.

In terms of descriptive statistics, the participants obtained higher averages in factor 2 of the Questionário de Práticas Educativas, followed by factors 3, 4 and 1. Identical results were obtained when the degree of importance of the item was assessed. These results should be further explored in other specific studies though. Pearson’s correlation test demonstrated a strong positive correlation between most factors in the Questionário de Práticas Educativas, appointing convergence between the factors and the general scale, and a strong positive correlation between the factors and the Scale of Importance of the Item. Although the sample size and specificity are considered as limitations in this study, we expect that future studies can offer support to consolidate the validity of the questionnaire and strengthen its potential use.

Conclusion

The educational practices involve important principles for the learning process in the different teaching strategies, including clinical simulation. Trying to see simulated teaching from this perspective allows us to understand better how the individuals who use simulation in their education have experienced it. In this study, the Educational Practices Questionnaire (Student Version) was translated and validated for the Portuguese language. In Portuguese, it is called Questionário de Práticas Educativas and can be applied to anyone using high-fidelity simulation in his/her education. Good psychometric results were found in this sample, except in the factor analysis, which requires further investigation in subsequent studies.

Acknowledgements

To the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), process nº: 2298-14-7 and Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq), process nº: 477954/2012-0.

Referências

• ¹
  Chickering AW, Gamson ZF. Seven principles for good practice in undergraduate education. AAHE Bulletin. 1987; 39(7):3-7.
• ²
  Jeffries PR. Simulation in nursing education: From conceptualization to evaluation. New York (EUA): National League for Nursing; 2007.
• ³
  Jeffries PR. A framework for designing, implementing, and evaluating simulations used as teaching strategies in nursing. Nurs Educ Perspect. 2005; 26(2):96-103.
• ⁴
  Parikh PP, White M, Markert RJ, Eustace R, Tchorz K. Simulation-based end-of-life care training during surgical clerkship: assessment of skills and perceptions. J Surg Res. 2015; 196(2):258-63.
• ⁵
  Gillan PC, Arora S, Sanderson H, Turner L. Palliative Care Simulation: Nurturing Interprofessional Collegiality. Health Interprof Pract. 2013; 2(1):eP1051.
• ⁶
  Kloh D, Reibnitz KS, Boehs AE, Wosny AM, Lima MM. The principle of integrality of care in the political-pedagogical projects of nursing programs. Rev Lat Am Enfermagem. 2014; 22(4):693-700.
• ⁷
  Thidemann IJ, Söderhamn O. High-fidelity simulation among bachelor students in simulation groups and use of different roles. Nurse Educ Today. 2013; 33(12):1599-604.
• ⁸
  Moura EC, Caliri MH. Simulação para desenvolvimento da competência clínica de avaliação de risco para úlcera por pressão. Acta Paul Enferm. 2013; 26(4):369-75.
• ⁹
  Jeffries PR, Rizzolo MA. National League for Nursing/Leardal project summary report: Designing and implementing models for the innovative use of simulation to teach nursing care of ill adults and children: A national, multi-site, multi-method study. New York: National League for Nursing; 2006.
• ¹⁰
  Ramm D, Thomson A, Jackson A. Learning clinical skills in the simulation suite: the lived experiences of student nurses involved in peer teaching and peer assessment. Nurse Educ Today. 2015; 35(6):823-7.
• ¹¹
  Najjar RH, Lyman B, Miehl N. Nursing students’ experiences with high-fidelity simulation. Int J Nurs Educ Scholarsh. 2015;12. pii: /j/ijnes.2015.12.issue-1/ijnes-2015-0010/ijnes-2015-0010.xml. doi: 10.1515/ijnes-2015-0010.
  » https://doi.org/10.1515/ijnes-2015-0010.
• ¹²
  Ferrer M, Alonso J, Prieto L, Plaza V, Monsó E, Marrades R, et al. Validity and reliability of the St George´s Respiratory Questionnaire after adaptation to a different language and culture: the Spanish example. Eur Respir J. 1996; 9(6):1160-6.
• ¹³
  Polit DF, Beck CT. The content validity index: are you sure you know what’s being reported? critique and recommendations. Res Nurs Health. 2006; 29(5):489-97.
• ¹⁴
  Valadares AFM, Magro MCS. Opinião dos estudantes de enfermagem sobre a simulação realística e o estágio curricular em cenário hospitalar. Acta Paul Enferm. 2014; 27(2):138-43.
• ¹⁵
  Laros JA. O uso de análise fatorial: algumas diretrizes para pesquisadores. In: Pasquali L, organizador. Análise fatorial para pesquisadores. Brasília (DF): LabPAM; 2005. p.163-84.
• ¹⁶
  Gorsuch RL. Factor analysis. 2a ed. Hillsdale (EUA): Lawrence Erlbaum Associates; 1983.
• ¹⁷
  Crocker L, Algina J. Introduction to classical and modern test theory. New York (EUA): Holt, Rinehartand Winston; 1986.
• ¹⁸
  Guadagnoli E, Velicer WF. Relation of sample size to the stability of component patterns. Psychol Bull. 1988; 103(1):265-75.
• ¹⁹
  Comrey AL, Lee HB. A first course in factor analysis. Hillsdale (EUA): Erlbaum; 1992.
• ²⁰
  Pasquali L. Instrumentos psicológicos: Manual prático de elaboração. Brasília (DF): LabPAM; 1999.

Publication Dates

• Publication in this collection
  Jul-Aug 2016

History

• Received
  10 Feb 2016
• Accepted
  22 Aug 2016