Psychometric validation of the Creighton Competency Evaluation Instrument in simulation

Vilarinho, Jéssica de Oliveira Veloso; Felix, Jorge Vinícius Cestari; Kalinke, Luciana Puchalski; Mazzo, Alessandra; Lopes Neto, Francisco Diego Negrão; Boostel, Radamés; Silva, Nilton Orlando da; Fontoura, Amanda Carolina de Oliveira Bialetzki

Acessibilidade / Reportar erro

Brasil

Acta Paulista de Enfermagem

Español English

Brasil

Español English

sumário « anterior atual seguinte »

Sumário

Original Article • Acta Paul Enferm. 33 • 2020 • https://doi.org/10.37689/acta-ape/2020AO03146 copy

Psychometric validation of the Creighton Competency Evaluation Instrument in simulation

Authorship SCIMAGO INSTITUTIONS RANKINGS

Abstract

Objective:

To validate and assess the Creighton Competency Evaluation Instrument reliability, Brazilian Portuguese version.

Methods:

This is a methodological study conducted with 30 professors/nurses who used the simulation methodology. Two scenarios were elaborated with equal themes and divergent performances of students in order to test the ability of the instrument to differentiate them. The scenarios were recorded and made available to the experts by a website that enabled the collection. For construct validation, the experimentation and phi correlation techniques were used. For reliability, equivalence with the Fleiss Kappa coefficient and internal consistency with Kuder-Richardson coefficient (KR20) were analyzed.

Results:

When analyzing validity, 20 of the 23 items showed significant results. In analysis, the domains presented correlations, classified from slight to substantial. When the scenarios were compared, the correlations were low, poor or negative. Regarding reliability, the Fleiss Kappa coefficient found was fair (0.282) for scenario 1 and moderate for scenario 2 (0.408). The internal consistency measured with KR20 was 0.717 for scenario 1 and 0.805 for scenario 2, classified as moderate and substantial.

Conclusion:

The Creighton Competency Evaluation Instrument, Brazilian Portuguese version, was considered valid and reliable for use nationally.

Keywords
Clinical competence; Psychometrics; Simulation; Validation studies

Scenario 1 versus Scenario 2 item2	Phi correlation coefficient	Scenario 1 versus Scenario 2 item	Phi correlation coefficient
1	0.062	13	N/A
2	N/A	14	-0.141
3	0.062	15	0.060
4	0.034	16	-0.449
5	0.064	17	0.208
6	0.204	18	-0.141
7	0.073	19	-0.256
8	N/A	20	-0.254
9	-0.068	21	0.090
10	0.152	22	-0.099
11	-0.082	23	0.156
12	N/A	-	-

		Total of items	Fleiss kappa	P value	% agreement	Agreement degree
SCENARIO 1	GENERAL	23	0.282	<0.001*	81	Fair
	Evaluation	3	0.022	0.392	89	-
	Communication	5	0.356	<0.001*	88	Fair
	Clinical judgment	9	0.173	<0.001*	86	Mild
	Patient safety	6	0.281	<0.001*	63	Fair
SCENARIO 2	GENERAL	23	0.408	<0.001*	63	Moderate
	Evaluation	3	0.008	0.783	96	-
	Communication	5	0.546	<0.001*	78	Moderate
	Clinical judgment	9	0.225	<0.001*	61	Fair
	Patient safety	6	0.289	<0.001*	36	Fair

		Scenario 1	Scenario 2
	Confidence interval (95%)	KR20	KR20
	GENERAL	0.717	0.805
EVALUATION	ITEMS	0.633	0.494
	1 – Obtains pertinent data	0.434	0.000
	2 – Performs follow-up assessments as needed	0.658	0.659
	3 – Assesses the environment in an orderly manner	0.541	0.000
COMMUNICATION	ITEMS	0.201	0.404
	4 – Communicates effectively with intra/interprofissional team (TeamSTEPPS, SBAR, Written Read Back Over)	0.215	0.340
	5 – Communicates effectively with patient and significant other (verbal, nonverbal, teaching)	0.137	0.187
	6 – Documents clearly, concisely, & accurately	-0.058	0.369
	7 – Responds to abnormal findings appropriately	0.137	0.381
	8 – Promotes professionalism	0.215	0.431
CLINICAL JUDGEMENT	ITEMS	0.601	0.749
	9 – Interprets vital signs (T, PA, FR, FC, Pain)	0.583	0.734
	10 – Interprets lab results	0.609	0.725
	11 – Interprets subjective/objective (recognizes relevant from irrelevant data)	0.510	0.729
	12 – Prioritizes appropriately	0.528	0.761
	13 – Performs evidence based interventions	0.610	0.661
	14 – Performs evidence based rationale for interventions	0.610	0.691
	15 – Evalutes evidence based interventions and outcomes	0.512	0.742
	16 – Reflects on clinical experience	0.497	0.751
	17 – Delegates appropriately	0.627	0.718
PATIENT SAFETY	ITEMS	0.275	0.625
	18 – Uses patient identifiers	0.364	0.636
	19 – Utilizes standardized practices and precautions including hand washing	0.127	0.586
	20 – Administers medications safely	-0.104	0.496
	21 – Manages technology and equipment	0.253	0.598
	22 – Performs procedures correctly	0.475	0.602
	23 – Refletcs on potential hazards and errors	0.069	0.552

Escola Paulista de Enfermagem, Universidade Federal de São Paulo R. Napoleão de Barros, 754, 04024-002 São Paulo - SP/Brasil, Tel./Fax: (55 11) 5576 4430 - São Paulo - SP - Brazil
E-mail: actapaulista@unifesp.br

Acompanhe os números deste periódico no seu leitor de RSS

[1] Corresponding author Jessica De Oliveira Veloso Vilarinho, Email: jessica.o.veloso@gmail.com