Psychometric assessment of the Instructional Materials Motivation Survey (IMMS) instrument in a remote learning environment

Cardoso-Júnior, Aloísio; Faria, Rosa Malena Delbone de

doi:10.1590/1981-5271v45.4-20210066.ING

Acessibilidade / Reportar erro

Brasil

Revista Brasileira de Educação Médica

Español English

Brasil

Español English

sumário « anterior atual seguinte »

Sumário

ORIGINAL ARTICLE • Rev. bras. educ. med. 45 (04) • 2021 • https://doi.org/10.1590/1981-5271v45.4-20210066.ING copy

Psychometric assessment of the Instructional Materials Motivation Survey (IMMS) instrument in a remote learning environment

Authorship SCIMAGO INSTITUTIONS RANKINGS

Abstract:

Introduction:

Motivation contributes greatly to learning, being a predictor of student performance. Thus, instruments that assess motivation after exposure to different teaching strategies and materials can contribute to the analysis and decision on its effectiveness. In this sense, the Instructional Materials Motivation Survey (IMMS) instrument measures students’ motivation after instructional activities.

Objective:

This study aimed to evaluate the evidence of validity of the IMMS, previously translated and cross-culturally adapted to Brazilian Portuguese.

Method:

Cross-sectional study used to evaluate the psychometric properties of the IMMS questionnaire, applied to 211 students from the first, third and fourth semesters of the medical course at José do Rosário Vellano University (Unifenas - Campus Belo Horizonte). The principal component analysis (PCA) with Varimax rotation and Cronbach’s alpha coefficient were used to assess the validity and reliability of the instrument.

Results:

The PCA reduced the instrument’s items from 36 to 25 items, distributed in four dimensions. The saturation of the items in the dimensions ranged from 0.529 to 0.790 and the total explained variance was 63.12%. The reliability of the modified IMMS (IMMS-BRV), measured by Cronbach’s alpha, ranged from 0.76 (Attention dimension) to 0.93 (Interest dimension).

Conclusion:

The IMMS application in the remote education scenario, through asynchronous video lectures of human anatomy, resulted in an alternative instrument (IMMS-BRV), with fewer items (more parsimonious) and good internal consistency, demonstrating preliminary evidence of its validity and reliability adequacy.

Keywords:
Medical Education; Motivation; Educational Measurement; Distance Education; Psychometrics

Question	Answers					Total	Mean	SD
Question	1	2	3	4	5	Total	Mean	SD
1. When I watched these video lessons for the first time, I had the impression that it would be easy for me.	11.4	19.9	25.6	29.4	13.7	100.0	3.1	1.2
2. There was something interesting at the beginning of these activities that caught my attention	9.5	14.7	28.0	32.2	15.6	100.0	3.3	1.2
3. These video lessons were harder to understand than I would have liked.	20.4	28.4	14.2	26.6	10.4	100.0	2.8	1.3
4. After listening to the initial information, I felt confident that I knew what was expected to be learned from this activity.	12.8	19.4	25.6	27.5	14.7	100.0	3.1	1.2
5. Completing the exercises in this activity gave me a satisfying sense of accomplishment.	10.4	15.2	18.5	34.1	21.8	100.0	3.4	1.3
6. It is clear to me how the content of this material is related to knowledge I already have.	7.1	15.2	32.7	29.8	15.2	100.0	3.3	1.1
7. Many slides had so much information that it was difficult to choose and remember the important points.	27.0	25.1	18.0	18.5	11.4	100.0	2.6	1.4
8. These video lessons are interesting.	10.0	11.4	18.5	32.2	27.9	100.0	3.6	1.3
9. There were stories, pictures or examples that showed me how these video lessons could be important to some people.	5.2	8.5	22.3	33.2	30.8	100.0	3.8	1.1
10. Successfully completing this activity was important to me.	4.3	4.3	21.8	27.0	42.6	100.0	4.0	1.1
11. Video quality helped to keep my attention.	7.1	13.3	16.1	31.7	31.8	100.0	3.7	1.2
12. These video lessons are so abstract that it was hard to keep my attention on them.	25.1	28.4	15.7	18.5	12.3	100.0	2.6	1.4
13. As I worked on this activity, I was confident that I could learn the content.	8.0	16.6	26.1	32.2	17.1	100.0	3.3	1.2
14. I enjoyed these video lessons so much that I would like to know more about this subject.	15.2	22.7	27.5	25.1	9.5	100.0	2.9	1.2
15. The slides in this material look uninteresting.	30.8	31.8	16.1	12.8	8.5	100.0	2.4	1.3
16. The content of these video lessons is relevant to my interests.	1.4	3.3	10.0	27.5	57.8	100.0	4.4	0.9
17. The way the information is organized on the slides helped keep my attention.	8.5	12.8	19.9	34.1	24.7	100.0	3.5	1.2
18. There are explanations or examples of how people use the knowledge of these video lessons	6.2	13.7	30.8	21.8	27.5	100.0	3.5	1.2
19. The exercises in these video lessons were very difficult.	18.5	33.6	38.9	7.6	1.4	100.0	2.4	0.9
20. These video lessons have elements that stimulated my curiosity.	8.0	17.5	20.4	37.0	17.1	100.0	3.4	1.2
21. I really enjoyed studying this activity.	11.4	18.9	25.6	28.9	15.2	100.0	3.2	1.2
22. The amount of repetition in these video lessons made me eventually bored.	27.0	20.4	23.7	19.4	9.5	100.0	2.6	1.3
23. The content and style of the slides in this material give the impression that it is worth getting to know them.	6.6	15.2	20.4	37.4	20.4	100.0	3.5	1.2
24. I learned some things that were surprising or unexpected	6.2	11.8	17.1	38.4	26.5	100.0	3.7	1.2
25. After working on this activity for some time, I was confident that I would be able to pass a test on it.	13.8	18.5	23.2	28.4	16.1	100.0	3.1	1.3
26. These video lessons were not relevant to my needs because I already knew most of them.	63.0	29.4	4.7	1.9	1.0	100.0	1.5	0.8
27. The way feedback was given after the exercises or other comments on the activity helped me feel rewarded for my effort.	19.9	18.5	30.8	19.9	10.9	100.0	2.8	1.3
28. The variety of reading excerpts, exercises, illustrations, etc., helped keep my attention on the activity.	10.5	16.1	21.3	32.2	19.9	100.0	3.4	1.3
29. The writing style is boring.	27.9	24.2	23.2	17.1	7.6	100.0	2.5	1.3
30. I was able to correlate the content of these video lessons to things I have seen, done or thought about in my own life.	8.0	16.1	27.5	31.8	16.6	100.0	3.3	1.2
31. There are so many words on each slide that it is annoying.	47.4	28.0	13.7	8.5	2.4	100.0	1.9	1.1
32. It was good to successfully complete this activity.	4.2	3.8	22.3	30.8	38.9	100.0	4.0	1.1
33. The content of these video lessons will be useful to me.	0.5	3.3	7.6	30.3	58.3	100.0	4.4	0.8
34. I really did not understand much of the material in these video lessons.	23.2	29.4	16.6	20.8	10.0	100.0	2.6	1.3
35. Good content organization helped me make sure I learned this material.	5.7	16.1	29.4	35.1	13.7	100.0	3.4	1.1
36. It was a pleasure working on such a well-planned activity.	12.8	16.6	25.6	24.2	20.8	100.0	3.2	1.3

FACTORS	Sum of squares of factorial loads WITHOUT Rotation			Sum of squares of factorial loads WITH Rotation
FACTORS	Eigenvalue	% of variance	% accumulated	Eigenvalue	% of variance	% accumulated
A	14,500	40,279	40,279	5,767	16,021	16,021
B	1,852	5,145	45,424	5,379	14,942	30,963
C	1,698	4,717	50,141	2,954	8,206	39,169
D	1,325	3,681	53,822	2,700	7,501	46,669
E	1,194	3,317	57,139	2,626	7,295	53,965
F	1,081	3,003	60,142	1,943	5,398	59,363
G	1,015	2,818	62,960	1,295	3,598	62,960
Total	22,665	62,961	--------	22,664	62,961	---------

IMMS Dimension	Excluded Item	Cause of exclusion
Attention	02 - There was something interesting at the beginning of these activities that got my attention.	IC
Relevance	06 - It is clear to me how the content of this material is related to things I already know.	MOC < 0.5
	10 - Successfully completing this lesson was important to me.	SL
	16 - The content of these video lessons is relevant to my interests.	MOC < 0.5
	18 - There are explanations or examples of how people use the knowledge of these video lessons.	MOC < 0.5
	26 - These video lessons were not relevant to my needs because I already knew most of them.	IC
	30 - I was able to correlate the content of these video lessons with things I have seen, done or thought about my own life.	MOC < 0.5
	33 - The content of these video lessons will be useful to me.	MOC < 0.5
Confidence	01 - When I first watched these video lessons, I had the impression that it would be easy for me.	IC
Confidence	19 - The exercises in these video lesson were too difficult.	SIC
Satisfaction	32 - It felt good to successfully complete this lesson.	SL

Items^* * Items numbered as in the IMMS - version translated into Brazilian Portuguese (Cardoso-Júnior et al.11). /dimensions	Loads (saturation of items in the dimensions)				MOC	MSA	Mean
	A	B	C	D	MOC	MSA	Mean
3				0.753	0.678	0.894	2.8
4		0.661			0.608	0.959	3.1
5		0.529			0.503	0.954	3.4
7			0.717		0.645	0.871	2.6
8	0.779				0.728	0.960	3.6
9	0.589				0.503	0.958	3.8
11	0.663				0.625	0.939	3.7
12				0.569	0.621	0.939	2.6
13		0.734			0.691	0.958	3.3
14	0.625				0.614	0.955	2.9
15	0.599				0.581	0.948	2.4
17	0.551				0.621	0.963	3.5
20	0.586				0.704	0.958	3.4
21		0.555			0.686	0.961	3.2
22			0.631		0.560	0.939	2.6
23	0.671				0.630	0.973	3.5
24	0.627				0.579	0.940	3.7
25		0.790			0.700	0.930	3.1
27	0.556				0.510	0.965	2.8
28	0.709				0.625	0.945	3.4
29			0.693		0.626	0.918	2.5
31			0.776		0.661	0.910	1.9
34				0.595	0.678	0.946	2.6
35		0.533			0.642	0.938	3.4
36	0.622				0.764	0.950	3.2
Cronbach’s Alpha	0.93	0.87	0.76	0.78		global MSA 0.947
% of explained variance	24.299	17.416	12.149	9.256
Eigenvalue	6.075	4.354	3.037	2.314
Total explained variance	63.12%

Dimension/Pearson		Dimension
Dimension/Pearson		A	B	C	D
A	r	1.00	0.81	0.55	0.65
A	p	< 0.001	< 0.001	< 0.001	< 0.001
B	r	---------	1.00	0.50	0.64
B	p	---------	< 0.001	< 0.001	< 0.001
C	r	---------	---------	1.00	0.50
C	p	---------	---------	< 0.001	< 0.001
D	r	---------	---------	---------	1.00
D	p	---------	---------	---------	< 0.001

Formed dimensions / Items	Cronbach’s alpha if item were removed
Interest (Cronbach’s alpha = 0.93)
8	0.92
9	0.93
11	0.93
14	0.92
15	0.93
17	0.92
20	0.92
23	0.92
24	0.93
27	0.93
28	0.92
36	0.92
Confidence (Cronbach’s Alpha = 0.87)
4	0.86
5	0.86
13	0.84
21	0.85
25	0.85
35	0.85
Attention (Cronbach’s Alpha = 0.76)
7	0.75
22	0.71
29	0.67
31	0.68
Expectancy (Cronbach’s Alpha = 0.78)
3	0.71
12	0.74
34	0.66

Validated Brazilian Version of the Instructional Materials Motivation Survey (IMMS-BRV)					Discordo totalmente		Discordo parcialmente	Não concordo, nem discordo		Concordo parcialmente	Concordo totalmente
1. Há 25 afirmações neste questionário. Por favor, pense sobre cada sentença em relação a cada material didático que você acabou de estudar e indique o quão verdadeira é a sentença. Dê a resposta que verdadeiramente se aplica a você, e não aquela que você gostaria que fosse verdadeira, ou aquilo que você pensa que outros querem ouvir
2. Pense em cada afirmação isoladamente e indique quão verdadeira ela é. Não se deixe influenciar por suas respostas a outras afirmações.
3. Registre suas respostas na folha fornecida e siga as instruções adicionais que podem ser fornecidas em relação à folha de resposta que está sendo usada nesta pesquisa. Obrigado.
1. Este material foi mais difícil de entender do que eu gostaria.					①		②	③		④	⑤
2. Depois de ler as informações iniciais, eu me senti confiante de que sabia o que se esperava que deveria ser aprendido com essa atividade.					①		②	③		④	⑤
3. Completar os exercícios dessa atividade me deu um sentimento satisfatório de realização.					①		②	③		④	⑤
4. Muitas das páginas tinham tanta informação que foi difícil escolher e lembrar os pontos importantes.					①		②	③		④	⑤
5. Esses materiais são interessantes.					①		②	③		④	⑤
6. Havia histórias, figuras ou exemplos que me mostraram como esse material poderia ser importante para algumas pessoas.					①		②	③		④	⑤
7. A qualidade da escrita ajudou a manter minha atenção.					①		②	③		④	⑤
8. Essa atividade é tão abstrata que foi difícil manter minha atenção nela.					①		②	③		④	⑤
9. Enquanto eu trabalhava nessa atividade, estava confiante de que poderia aprender o conteúdo.					①		②	③		④	⑤
10. Gostei tanto dessa atividade que gostaria de saber mais sobre este assunto.					①		②	③		④	⑤
11. As páginas desse material parecem pouco interessantes.					①		②	③		④	⑤
12. A forma como a informação está organizada nas páginas ajudou a manter minha atenção.					①		②	③		④	⑤
13. Esta atividade tem elementos que estimularam minha curiosidade.					①		②	③		④	⑤
14. Gostei muito de estudar essa atividade.					①		②	③		④	⑤
15. A quantidade de repetição nesta atividade me fez ficar, eventualmente, entediado.					①		②	③		④	⑤
16. O conteúdo e o estilo de escrita neste material dão a impressão de que vale a pena conhecê-lo.					①		②	③		④	⑤
17. Aprendi algumas coisas que foram surpreendentes ou inesperadas.					①		②	③		④	⑤
18. Depois de trabalhar nessa atividade por algum tempo, eu estava confiante de que seria capaz de passar em um teste sobre ela.					①		②	③		④	⑤
19. A maneira como foi dado o feedback após os exercícios, ou outros comentários na atividade, ajudou-me a sentir recompensado pelo meu esforço.					①		②	③		④	⑤
20. A variedade de trechos de leitura, exercícios, ilustrações etc., ajudou a manter minha atenção na atividade.					①		②	③		④	⑤
21. O estilo de escrever é entediante.					①		②	③		④	⑤
22. Há tantas palavras em cada página que é irritante.					①		②	③		④	⑤
23. Eu realmente não consegui entender muito o material desta atividade.					①		②	③		④	⑤
24. A boa organização do conteúdo me ajudou a ter certeza de que eu aprenderia este material.					①		②	③		④	⑤
25. Foi um prazer trabalhar em uma atividade tão bem planejada.					①		②	③		④		⑤
Itens distribuídos por dimensão
Dimensão	Itens
Interesse	5	6	7	10	11^* * Inverted items: the score of the items must be inverted.	12	13	16	17	19	20	25
Confiança	2	3	9	14	18	24
Atenção	4^* * Inverted items: the score of the items must be inverted.	15^* * Inverted items: the score of the items must be inverted.	21^* * Inverted items: the score of the items must be inverted.	22^* * Inverted items: the score of the items must be inverted.
Expectativa	1^* * Inverted items: the score of the items must be inverted.	8^* * Inverted items: the score of the items must be inverted.	23^* * Inverted items: the score of the items must be inverted.

Associação Brasileira de Educação Médica SCN - QD 02 - BL D - Torre A - Salas 1021 e 1023 | Asa Norte, Brasília | DF | CEP: 70712-903, Tel: (61) 3024-9978 / 3024-8013, Fax: +55 21 2260-6662 - Brasília - DF - Brazil
E-mail: rbem.abem@gmail.com

Acompanhe os números deste periódico no seu leitor de RSS

[1] aloisiocardosojr@gmail.com rmdfmedicina@gmail.com