Cultural adaptation to Brazil and psychometric performance of the “Evidence-Based Practice Questionnaire”

Abstracts

Objectives

To culturally adapt the instrument “Evidence-Based Practice Questionnaire” (EBPQ) to the Portuguese language and assess its psychometric qualities.

Methods

The steps of cultural adaptation of measurement instruments were followed. Reliability was verified through internal consistency, stability by test-retest, and construct validity by the contrasted groups approach.

Results

High Cronbach’s alpha (0.91 to 0.68) and satisfactory Intraclass Correlation Coefficient (0.90) were obtained in all domains. In assessing construct validity, significant differences were found between groups of nurses with different backgrounds.

Conclusion

The steps of cultural adaptation of measurement instruments have been successfully completed. The Brazilian version obtained presents reliable psychometric properties for its use in this population.

Nursing research; Clinical nursing research; Evidence-based nursing; Psychometric

Objetivo

Adaptar culturalmente o instrumento EBPQ para a língua portuguesa e avaliar suas qualidades psicométricas.

Métodos

Foram seguidos os passos de adaptação cultural de instrumentos de medida. Foi verificada a confiabilidade por meio da avaliação da consistência interna e da estabilidade pelo teste-reteste e a validade de constructo com abordagem de grupos contrastados.

Resultados

Obteve-se Coeficiente Alfa de Cronbach elevado em todos os domínios (0,91 – 0,68) e Coeficiente de Correlação Intraclasse satisfatório (0,90). Na avaliação da validade de constructo, houve diferença significativa entre os grupos de enfermeiros com diferentes formações.

Conclusão

As etapas de adaptação cultural de instrumentos de medida foram concluídas com sucesso. A versão brasileira obtida apresenta propriedades psicométricas confiáveis para a sua utilização nessa população.

Pesquisa em enfermagem; Pesquisa em enfermagem clínica; Enfermagem baseada em evidências; Psicometria

Introduction

Evidence-based practice (EBP) is a technology that has been gaining popularity with the purpose of improving clinical effectiveness. Its application involves using the best available clinical evidence on individual patient care and implies to improve clinical professional knowledge with the most consistent and reliable scientific findings, resulting from the advancement of clinical research.⁽¹⁾

Currently, the process of EBP is implemented in five steps: 1 formulate a searchable, answerable question; 2 find the best evidence to answer the clinical question; 3 appraise the evidence according to its validity, applicability and impact; 4. integrate the evidence with clinical expertise, customer values and circumstances and information from the practical context; and 5 evaluate the effectiveness and efficiency of the information found in the application of steps 1-4 and think about ways to improve job performance.⁽²⁾

In the context of nursing, EBP apparently emerged with the Cochrane Group, journals such as the Evidence-Based Nursing and centers such as the Joanna Briggs Institute for Evidence-Based Nursing. ⁽³⁾ Evidence-based nursing is defined as “a problem-solving approach to clinical care that incorporates use of current best evidence from well-designed studies, a clinician’s expertise, and patient values and preferences.⁽⁴⁾

However, in nursing, clinical care seems not to have been benefited with the production of knowledge. Personal difficulties continue to be observed, such as motivation and dissemination of scientific findings, as well as situational difficulties, such as limited resources and inadequate organization of time.^(1,5-7) In order to be successful, EBP requires individual and organizational strategies that address factors that interfere in its utilization.⁽⁸⁾

Regarding the production of nursing knowledge on EBP and its application, various measurement instruments have been developed to evaluate the use and the barriers to adoption of EBP.⁽⁹⁾

An evaluation of EBP was proposed in 1998 by means of the “Evidence-Based Practice Questionnaire” (EBPQ). This instrument was developed in the United Kingdom in order to appraise attitudes, knowledge and implementation of EBP for physicians and other health care providers.⁽¹⁰⁾ The analysis of the psychometric properties of EBPQ was subsequently performed in a sample of nurses of various levels of training and was proved to be a valid and reliable tool.⁽¹¹⁾ It is a brief self-administered questionnaire, of easy understanding, which explores the use of EBP by health professionals in everyday practice.

This instrument has been used in international research, with the purpose to evaluate practice, knowledge and attitudes of students and attending nurses. The results show important information about adopting EBP among these professionals and suggest strategies for its dissemination.^(12-15)

The EBPQ was recently adapted and validated for the Spanish language, obtaining a reduced version, however suitable for use in that culture.⁽¹³⁾ This version was used in a study in Spain to diagnose the factors that nurses perceive as facilitators to EBP.⁽¹⁶⁾

Considering the lack of instruments in Brazil for assessing EBP among nurses and the importance of this information to investigate its application in clinical care as a tool that provides quality assistance, the objective of this study is to provide a version of the Evidence-based Practice Questionnaire (EBPQ) for the Brazilian population by means of the process of cultural adaptation, as well as through the evaluation of its measurement properties.

Methods

The Evidence-Based Practice Questionnaire (EBPQ) consists of 24 items rated on a scale of one to seven (Likert scale). The score of the instrument is calculated by adding the response values of each question, totaling 168 points, with higher scores indicating more positive attitudes toward EBP. The scores can be also evaluated by fields, calculating the arithmetic mean. The items are categorized into three dimensions:

1. Practice of Evidence-Based Nursing: six questions or 42 points;

2. Attitudes related to Evidence-Based Practice: four questions or 28 points;

3. Knowledge and skills associated with Evidence-Based Practice: 14 questions or 98 points.

In the end, the instrument presents questions related to the characterization of the research subjects regarding sociodemographic and data related to occupational training, work experience and field of practice.

In the original study, the instrument had a Cronbach’s alpha of 0.87 and satisfactory convergent validity (p <0.001).⁽¹¹⁾

After the author’s consent, the essential steps of cultural adaptation were followed, as recommended by specialized publications, in order to ensure its quality.⁽¹⁷⁾

First, an independent translation was performed by two translators separately. One of the translators was aware of the goals and concepts involving the instrument to be translated and the other translator had no prior knowledge of these concepts and goals. The two translated versions of the instrument were confronted by the advisor, researcher and a mediator. After identifying the discrepancies, a single synthesized version of the instrument was obtained.

Subsequently, the obtained synthesized version was back-translated into English by two translators native-speakers of English who had not participated in the first stage, thus obtaining back-translation 1 and back-translation 2. These translators did not receive information on the concepts and purposes of the instrument.

After completing the back-translation phase, a committee composed of seven bilingual participants and nursing research experts consolidated all versions produced into one single version that was used in the pre-test. A specific instrument was constructed for the purposes of this assessment containing the versions: original, synthesis of translations and back-translations. The committee evaluated the semantic, idiomatic, cultural and conceptual equivalence of each item or question of the EBPQ.

To calculate the level of agreement between the committee judges the Content Validity Index (CVI) was used.⁽¹⁸⁾ Was considered satisfactory an index of agreement equal to or above 90%.⁽¹⁹⁾ Thus, through the evaluation of the judges committee it is possible to verify the content validity of the questionnaire.

The Pre-test involved a sample of 30 nurses from a public hospital. At this stage, the understanding of the instrument was assessed, identifying questions or concepts considered difficult to understand.

Nurses that took part in this study were from a public hospital of a public university located in upstate São Paulo, Brazil. For data analysis, subjects were divided into two groups. Group 1 included nurses, students and faculty with a master’s or doctoral degree, or were doctoral candidates. Group 2 included nurses who had only completed their graduation and who were not enrolled in any postgraduate course at the time of data collection.

Nurses who had a completed or ongoing latu sensu specialization course were excluded from the study. Subjects who were on vacation or leave during the period of data collection were also excluded.

Convenience sampling was performed, thus the number of subjects in each group were equivalent. The sample size was obtained by calculating the sample size for the Cronbach’s alpha, totaling approximately 160 individuals.⁽²⁰⁾

Reliability was assessed by internal consistency and stability. For internal consistency, Cronbach’s alpha coefficient was used. For stability, assessed by test-retest, in which the questionnaire was administered on two separate occasions, with an interval of 10 to 15 days, the Intraclass Correlation Coefficient was used.⁽²¹⁾

The validity of the construct was verified with the known groups approach to determine the degree to which the instrument demonstrated different scores for groups of each group. It was expected to find higher scores in the Group 1, consisting of nurses with a Master’s degree or PhDs, in relation to nurses who held completed the undergraduate studies. The validity was evaluated by the non-parametric Mann-Whitney test.

The development of the study met national and international standards of ethics in research involving human beings.

Results

Following the judges’ evaluation of semantic-idiomatic, conceptual and cultural equivalences of the EBPQ, minimal changes related to the translated terms were suggested, such as replacing the word “persevero” (Portuguese word for persevere) with “mantenho” (Portuguese word for maintain).

In the equivalence assessment, agreement for items 1 and 14 was 57%; item 18, 71% and 85% for items 3, 4, 7, 9, 12, 16, 19, 20, 23, 28 and 29. The suggestions were analyzed in a discussion meeting with the researcher, the advisor, and the members of the research group on cultural adaptation of measurement instruments. The agreement rates for all the other items were above 90%.

The participants of the pre-test were 30 nurses of a public hospital, who performed direct care activities and/or nursing management. It was found that the average time to complete the questionnaire was nine minutes, the minimum time was six and the maximum time was 21 minutes.

Seven nurses found it difficult to understand question 01, in relation to the word “lacuna” (Portuguese word for gap). This difficulty was revised, and it was decided to add the word “falta” (Portuguese word for lack) as a synonym of the first word, for better interpretation of the question.

The study included 158 nurses, 81 in Group 1 (nurses with master’s or doctoral degrees) and 77 in Group 2 (public hospital nurses without postgraduate degrees). The subjects were 148 (84%) women and 10 men (16%), of ages between 23 and 66 years. The time since graduation ranged from 01 to 43 years. In Group 1, most subjects were from the College of Nursing, 64.2% held a master’s degree, 29.6% a doctoral degree and 6.2% a postdoctoral degree. In Group 2, all nurses worked in the public hospital, of which 70 (90.9%) performed direct care activities and 7 (9.0%) were unit managers (Table 1).

Table 1
Socio-demographic characterization, according to the division into groups (n = 158)

Variable	Group 1 (n=81)		Group 2 (n=77)		p-value
	Mean (SD*)	Observed variation	Mean (SD*)	Observed variation
Age	41.4(10.7)	25.0 – 66.0	39.1(10.2)	23.0 – 61.0	0.1368
Time since graduation	18.2(10.5)	3.0 – 43.0	14.1(10.0)	1.0 – 32.0	0.0076
Time in practice	17.5 (10.6)	1.0 – 43.0	12.6(9.9)	1.0 – 30.0	0.0021
Variable	n(%)		n(%)
Gender
Female	75(92.6)		73(94.8)
Male	6(7.4)		4(5.2)
Place of work
Public Hospital	31(38.3)		77(100.0)
College of Nursing	50(61.7)		0(0)
Position
Direct care	15(18.7)		70(90.9)
Management	14(17.5)		7(9.9)
University professor	24(30.0)		0(0)
Student	27(33.7)		0(0)
Education
Undergraduate degree	0(0)		77(100.0)
Master’s degree	52(64.2)		-(-)
Doctoral degree	24(29.6)		-(-)
Postdoctoral degree	5(6.2)		-(-)

• ^*
  SD - Standard Deviation

Data description of the Evidence-Based Practice questionnaire:

The final score for the Evidence-Based Practice and Clinical Effectiveness scale for nurses was 129.15 for Group 1 (nurses with master’s or doctoral degrees) and 111.24 for Group 2 (public hospital nurses without postgraduate degrees).

In both groups, the domain with the highest average score per item was Domain 2 - Attitudes related to EBP (5.92 and 5.35 for Group 1 and 2, respectively), followed by Domain 1 - Evidence-Based Nursing Practice (5.38 and 4.54) and finally, Domain 3 - Knowledge and Skills (5.22 and 4.46).

Reliability

Internal consistency

Regarding internal consistency, the reliability of each question in relation to the domain of the instrument was evaluated, first for all subjects (n = 156) and then for each group of nurses.

The instrument presented satisfactory internal consistency values. In the overall sample, Domain 3 had the highest Cronbach’s alpha (0.92), which assesses the Knowledge and Skills associated with EBP. Domain 1, which evaluates EBP among nurses presented Cronbach’s alpha of 0.86. Questions inquiring attitudes related to EBP (Domain 2) obtained the lowest value (0.68).

Stability (test-retest)

To evaluate the reliability with respect to temporal stability, a sample of 50 nurses was used.The questionnaire showed high stability for all domains and for the instrument as a whole (ICC = 0.90) (Table 2).

Table 2
Intraclass correlation coefficients (ICC) and confidence intervals (95% CI) (n = 50)

Domains		Number of items	ICC*	IC 95%**
1	Evidence-Based Practice	6	0.84	0.74 - 0.91
2	Attitudes toward EBP	4	0.85	0.75 - 0.91
3	Knowledge and skills associated to EBP	14	0.86	0.77 - 0.92
Total EBPQ		24	0.90	0.83 - 0.94

• ^*
  ICC – Intraclass correlation coefficients;
• ^**
  CI – Confidence interval of 95%

The assessment of construct validity by means of the known groups approach showed that the subjects in Group 1 (master’s or doctoral degrees) had significantly higher means compared to those in Group 2 (undergraduate degree). Considering the total score of the instrument, Group 1 also showed higher values compared to Group 2. Thus, it is observed that the instrument was able to demonstrate differences in scores between known groups (Table 3).

Table 3
Comparison between the mean scores of the items and the score of the domains

Domains		Group 1 (n=81)		Group 2 (n=77)		p-value
		Mean (SD*)	Total score	Mean (SD*)	Total score
1	Evidence-Based Practice	5.38(1.05)	32.31	4.57(1.19)	27.29	<0.0001
2	Attitudes toward EBP	5.92(0.82)	23.68	5.34(1.07)	21.42	0.0004
3	Knowledge and skills associated to EBP	5.23(0.71)	73.16	4.46(0.84)	62.53	<0.0001
Total EBPQ		5.38(0.71)	129.15	4.64(0.83)	111.24	<0.0001

Discussion

Like other validation studies, the study has limitations with regard to the validity of self-reported measures, because of the impossibility of adding direct measures of EBP, such as the observation of professional practice in accordance with their research findings. Furthermore, because it is the first study in Brazil using a measuring instrument for evaluation of EBP among nurses, other studies about the EBPQ must be produced in the national reality, to ensure greater validity through other methodological approaches, the expansion of the locations, and the expansion of quantity and characteristics of nurses.

However, the analysis of individual and organizational barriers in the incorporation of decision-making in individual care for the patient through the use of the EBPQ can facilitate the understanding and the directing of formation of technological skills necessary for a scientifically adequate healthcare choice. The study provided a culturally adapted questionnaire into the Portuguese language, with satisfactory assessment of validity and reliability, in order to create a tool for the development and evaluation of the implementation of EBP in educational programs or initiatives, and for institutions interested in acknowledging its use or the knowledge of the professionals.

The cultural adaptation was performed following the steps recommended by international protocols.⁽²²⁾ In the evaluation made by the committee of judges, a percentage of agreement (CVI) above 90% was obtained for most items.

The mean scores for domains found in the Brazilian version were similar to the scores of other studies that used the EBPQ.⁽²³⁾ The study obtained its highest average score per item in the field related to opinions and attitudes towards EBP, which shows good acceptability and positive opinions on the subject among Brazilian nurses. A study conducted with nurses, of various levels of training, in California also found the highest average score in this area.⁽¹²⁾

The lowest mean scores were found in the domain related to knowledge and skills associated to EBP. The items questioned the frequency of literature review and the elaboration of research questions. This corroborates with other studies using the EBPQ to assess the perceptions of nurses on the topic.⁽¹³⁾

Regarding the evaluation of the psychometric properties of EBPQ, this showed high values for internal consistency in the analysis of the instrument as a whole, with greater accuracy for the domain of knowledge and skills (0.92), followed by the domain related to the application of EBP (0.86) and, finally, attitudes (0.68). The Cronbach’s alpha score for the domain on attitudes is justified by its low number of items. Similar results were found in international studies and in the original study.^(9,12,24)

In the analysis of the reliability through stability, a ICC of 0.90 was obtained for the questionnaire as a whole, which shows the temporal stability of the instrument.

The assessment of construct validity indicated significant differences between groups for the domains of practice, attitudes and knowledge of EBP (p <0.0001). This result represents that the use of EBP remains an academic and scientific reality and that the application of research in the practical field remains incipient.

A study found higher EBPQ scores among nurses with a master’s or doctoral degree, nurse managers and educators, and lower scores between the domains of the instrument for nurses that held only an undergraduate degree. Higher educational and training levels, such as a master’s degree, tend to show more satisfactory results in relation to knowledge and use of EBP.^{(7,13,14,24,25)}

After completing the steps necessary to deliver the questionnaire for the Brazilian context, it is suggested that it can be a useful tool for evaluating educational strategies and for evaluating health institutions concerned with health care quality. In addition, the instrument can be used to measure the personal evaluation of professionals regarding their practice, awakening critical thinking about the quality of their practice.

Conclusion

It is concluded that the process of cultural adaptation was performed successfully, and that the Questionnaire of Evidence-Based Practice and Clinical Effectiveness presents satisfactory validity and reliability.

References

• ¹
  Evidence-Based Medicine Working Group. Evidence-based medicine: a new approach to teaching the practice of medicine. JAMA. 1992; 268(17):2420-5.
• ²
  Hoffmann T, Bennett S, Del Mar C. Evidence-based practice across the health professions. Australia: Elsevier; 2010. Introduction to evidence-based practice.p.16-37.
• ³
  Estabrooks CA. Will Evidence-based nursing practice make practice perfect? Can J Nurs Res. 1998; 30(1):15-36.
• ⁴
  Mulhall A. Nursing, research, and the evidence. Evid Based Nurs. 1998;1:4-6.
• ⁵
  Ribas CR, Zanetti ML, Caliri MH. A arte da comunicação do conhecimento científico. Rev Eletron Enf. 2009;11(3):712-6.
• ⁶
  Polit DF, Beck CT. Nursing Research: Generating and assessing evidence for nursing practice. Lippincott Williams & Wilkins: Philadelphia; 2008.
• ⁷
  Dalhein A, Harthug S, Nilsen RM, Nortvedt MW. Factors influencing the development of evidence-based practice among nurses: a self report survey. BMC. 2012;12:367.
• ⁸
  Bostrom AM, Rudman A, Ehrenberg A, Gustavsson JP, Wallin L. Factors associated with evidence-based practice among registered nurses in Sweden: a national cross-sectional study. BMC. 2013; 13:165.
• ⁹
  Thorsteinsson HS. Translation and validation of two evidence-based nursing practice instruments. Int Nurs Rev. 2012; 59(2): 259-65.
• ¹⁰
  Upton D, Lewis B. Clinical effectiveness and EBP: design of a questionnaire. Int J Ther Rehab. 1998; 5(12):647-50.
• ¹¹
  Upton D, Upton P. Development of an evidence-based practice questionnaire for nurses. J Adv Nurs. 2006; 54(4): 454-8.
• ¹²
  Brown CE, Wickline MA, Ecoff L, Glaser D. Nursing practice, knowledge, attitudes and perceived barriers to evidence-based practice at an academic medical center. J Adv Nurs. 2008;65(2):371-81.
• ¹³
  Koehn Ml, Lehman K. Nurse´s perceptions of evidence-based nursing practice. J Adv Nurs. 2008; 62(2): 209-15.
• ¹⁴
  Prior PM, Wilkinson J, Neville S. Practice nurse use of evidence in clinical practice: a descriptive survey. Nurs Prax N Z. 2010;26(2):14-25.
• ¹⁵
  Pedro-Gómez JP, Morales-Asencio JM, Abad AS, Veny MB, Roman MJ, Ronda FM. Validación de la Española Del cuestionario sobre la Práctica Basada em la Evidencia em enfermería. Rev Esp Salud Pública. 2008; 83(4):577-86.
• ¹⁶
  Pedro-Gómez JP, Morales-Asencio JM, Abad AS, Veny MB, Vives GA, Campaner CP. Entorno de prática de los profesionales de enfermería y competência para la oncorporación de la evidencia a lãs decisiones: situación em las Islas Baleares. Gac Sanit. 2011;25(3):191-7.
• ¹⁷
  Beaton D, Bombardier C, Guillemin F, Ferraz MB. Recommendations for the Cross-Cultural Adaptation of the DASH & quick DASH Outcome Measures. Institute for Work & Health; 2007. p. 3-45.
• ¹⁸
  Alexandre NM, Coluci MZ. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Cien Saúde Colet. 2011;16(7):3061-8.
• ¹⁹
  Polit DF, Beck CT. The content validity index: are you sure you know what’s being reported? Critique and recomendations. Res Nurs Health. 2006;29(5):489-49.
• ²⁰
  Bonett D. Sample Size Requirements for Testing and Estimating Coefficient Alpha. J Educ Behav Stat. 2002;27(4):335-340.
• ²¹
  Alexandre NM, Gallasch CH, Lima MH, Rodrigues RC. A confiabilidade no desenvolvimento e avaliação de instrumentos de medida na área da saúde. Rev Eletr Enferm. 2013;15(3):802-9.
• ²²
  Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural a daptation of self-report measures. Spine. 2000; 25(24):3186-91.
• ²³
  González-Torrente S, Pericas-Beltrán J, Bennasar-Veny M, Adrover-Barceló R, Morales-Acensio J, Pedro-Goméz J. Perception of evidence-based practice and the Professional environment of Primary Health Care nurses in the Spanish context: a cross-sectional study. BMC. 2012;12:227.
• ²⁴
  Brown CE, Ecoff L, Kim SC, Wickline MA, Rose B, Klimpel K, et al. Multi-institutional study of barriers to research utilization and evidence-based practice among hospital nurses. J Clin Nurs. 2010;19(13-14): 1944-51.
• ²⁵
  Eizenberg MM. Implementation of evidence-based nursing practice: nurse´s personal anr professional factors? J Adv Nurs. 2011; 67(1):33-42.

Publication Dates

• Publication in this collection
  Sep-Oct 2014

History

• Received
  12 Mar 2014
• Accepted
  29 July 2014