Cooperative learning scale: validation among Portuguese and Brazilian university students

ABSTRACT

The aim of this study was to adapt and validate the Spanish version of the Cooperative Learning Scale (CAT) for Portuguese and Brazilian higher education students. The study included 493 higher education students aged 17-45 years from a university in northern Portugal and 414 Brazilian university students aged 14-52 years from different universities in Brazil. To this adaptation, the original instrument, designed and validated for Spanish contexts and for students from the 3rd cycle of basic education and secondary education, went through several phases: a) evaluation by experts; b) pilot study; c) confirmatory factorial analysis; and d) determination of convergent and divergent validity. Both the Portuguese and the Brazilian versions of the CAT present a good adjustment model and include five factors with three items each: positive interdependence, interpersonal skills, promotive interaction, individual accountability, and group processing.

Keywords:
Cooperative Learning; Cooperative Learning Scale; Cooperative Groups; Higher Education

RESUMO

O objetivo deste estudo foi adaptar e validar a versão espanhola da Escala de Aprendizagem Cooperativa (EAC) para estudantes portugueses e brasileiros do ensino superior. Participaram no estudo 493 estudantes do Ensino Superior com idades entre 17 e 45 anos de uma universidade do norte de Portugal e 414 estudantes universitários brasileiros com idades entre 14 e 52 anos de universidades do Brasil. Para a adaptação, o instrumento original, desenhado e validado para contextos espanhóis para alunos do 3.º ciclo dos Ensinos Básico e Secundário, passou por várias fases: a) avaliação por especialistas; b) estudo piloto; c) análise fatorial confirmatória; e d) determinação da validade convergente e divergente. A versão portuguesa e a brasileira da EAC apresentam um bom modelo de ajustamento e incluem cinco fatores com três itens cada: interdependência positiva, competências interpessoais, interação estimuladora, responsabilidade individual e de grupo e processo de grupo.

Palavras-chave:
Aprendizagem Cooperativa; Ensino Superior; Escala de Aprendizagem Cooperativa; Grupos Cooperativos

RESUMEN

El objetivo de este estudio fue adaptar y validar la versión en español de la Escala de Aprendizaje Cooperativo (EAC) para estudiantes de educación superior portugueses y brasileños. Participaron en el estudio 493 estudiantes de educación superior entre 17 y 45 años de una universidad del norte de Portugal y 414 brasileños de entre 14 y 52 años de universidades de Brasil. Para la adaptación, el instrumento original, diseñado y validado para contextos españoles y para estudiantes del 3er ciclo de educación básica y de educación secundaria, pasó por varias fases: a) evaluación de expertos; b) estudio piloto; c) análisis factorial confirmatorio; y d) determinación de la validez convergente y divergente. Tanto la versión portuguesa como la brasileña del EAC presentan un buen modelo de ajuste e incluyen cinco factores con tres ítems cada uno: interdependencia positiva, habilidades sociales, interacción promotora, responsabilidad individual, y procesamiento grupal.

Palabras clave:
Aprendizaje Cooperativo; Educación Superior; Escala de Aprendizaje Cooperativo; Grupos Cooperativos

INTRODUCTION

There is a growing consensus in pedagogy about the need to encourage the use of student-centered teaching methods, in which students are more actively involved in the learning process, as opposed to the use of traditional, teacher-centered methods (Hannafin, 2012; Carr, Palmer and Hagel, 2015; Schweisfurt, 2015; EC, 2016; Hynes, 2017; Crisol-Moya, Romero-López and Caurcel-Cara, 2020; Evans, 2020). The importance of valuing these methods in teaching and learning process stems from the guidelines of various international organizations and researchers, as well as from constructivist and socio-constructivist theories of learning. Both have focused on the importance of the skills needed for students’ success in the 21^st century, and, although their conceptualizations vary slightly, the majority emphasize critical thinking, communication, collaboration, and creativity as fundamental skills (NRC, 2011; Lai and Viering, 2012; Soland, Hamilton and Stecher, 2013; Lench, Fukuda and Anderson, 2015; Care et al., 2018; Rios et al., 2020), as they prepare students for the job market and for active, conscious, and responsible civic participation (Conley, 2007; Martins et al., 2017; OECD, 2017a, 2017b; OECD 2018a, 2018b, 2019; Mehta and Fine, 2019).

According to constructivist and socio-constructivist theories of learning, students construct meaning and learn in social contexts (Piaget, 1964; Vygotsky, 1978). From a cognitive perspective, research on conceptual change (Mason, 2001) demonstrates that the confrontation of divergent viewpoints promotes learning, contributing to the restructuring of concepts through socio-cognitive conflict. The sociocultural approach reinforces the idea that the effectiveness of peer learning can be explained by the possibility for students to take on the role of mediators in their peers’ learning process (Vygotsky, 1978).

This approach includes cooperative learning, which is grounded on the assumption that learning is inherently social, with participants engaging in dialogue, thus facilitating learning. Johnson, Johnson and Holubec (2009) refer to cooperative learning as a teaching method that involves the use of small, heterogeneous groups, where students work together to maximize their own learning and that of their peers (Cohen, 1994; Johnson and Johnson, 1994; Kagan, 1994; Veenman et al., 2002). It also contributes to the development of social skills such as mutual respect, solidarity, reciprocal feelings of obligation, mutual assistance, and the ability to adopt common perspectives (Johnson and Johnson, 1994). Valuing heterogeneity in group composition allows teachers to explore the diversity within the different groups and enhance interpersonal (or peer) learning (Balkcom, 1992). For Slavin (1995), cooperative learning is an educational approach that involves groups of students working together to solve a problem, complete a task, or create a product. Johnson, Johnson and Holubec (1994) and Pateşan, Balagiu and Zechia (2016) assert that cooperative learning brings about a new attitude in students, placing them at the center of their learning experiences instead of adopting a passive attitude as mere recipients of knowledge (Johnson, Johnson and Smith, 2014).

Over the past few decades, cooperative learning has gained prominence and is one of the most recommended methods of instruction at all levels of education (Slavin, 1995; Rogoff, Matusov and White, 1996; Marzano and Mid-continent Regional Educational Laboratory, 1998; Hattie, 2009; Topping, 2017; Shonfeld and Gibson, 2018). This is because it actively engages students in realistic, adult-like tasks, enhancing their critical thinking, reasoning, and problem-solving skills (Wincel, 2013), which are essential to address the challenges of the 21^st century: global interdependence, the increasing number of democracies, the need for creative entrepreneurs, and the changes in interpersonal relationships (Johnson and Johnson, 2014).

Cooperative learning only occurs when five basic elements are present in group work: positive interdependence, individual and group accountability, promotive interaction (preferably face-to-face), social skills, and group processing (Johnson, Johnson and Holubec, 2009; Johnson and Johnson, 2014).

1. Positive interdependence is considered the heart of cooperative learning and assumes that a student cannot succeed unless other group members do as well, and vice versa. To achieve this, common learning objectives are pursued and can be strengthened by assigning roles within the group;

2. Individual and group accountability is ensured if each member fulfills their part of the work, if each one’s performance is evaluated, and if the results of this evaluation are understood through metacognitive analysis, providing feedback for future improvement;

3. Promotive interaction, preferably face-to-face, is promoted by dialogue among all group members to understand how to solve problems and share their knowledge with each other;

4. The social skills of each group member must be taken into account to ensure the quality and level of cooperation. Decision-making, communication, conflict resolution, leadership, and building trust are skills that teachers should promote within groups;

5. Group processing or reflection on the work done by the group exists if its members have the opportunity to discuss the quality of the results obtained and whether the group’s objective was achieved and how.

The metacognitive skills of group members are a prerequisite that allows them to analyze the learning achieved and the individual and group work processes, both by individuals and by the groupI, improving not only the work process but also interpersonal relationships (Lopes and Silva, 2009).

Studies conducted over the recent two decades have shown the benefits of cooperative learning compared to more traditional teaching methodologies (Johnson and Johnson, 1994; Slavin, 1995; Sharan, 1999). Yager, Johnson and Johnson (1985), Johnson and Johnson (1989, 2000), Panitz (1996), Veenman et al. (2002), Johnson, Johnson and Smith (2007), Sharan (2010), Kyndt et al. (2013) and Gillies (2016) present the benefits of cooperative learning in four major categories: social (e.g., encourages understanding of diversity; creates a stronger social support system; promotes positive social responses to problems; psychological (e.g., increased self-esteem; student satisfaction with learning experiences); academic (e.g. critical thinking, oral communication skills, increased metacognitive skills); and assessment (e.g. alternative assessment methods, immediate feedback to students and teachers, easier supervision of students in groups).

From the above, the potential of cooperative learning in developing students’ skills at various levels, particularly those essential for the 21^st century, stands out. However, a limitation at this level is that teachers do not have a suitable tool to assess the quality of cooperative work carried out by students. An instrument that allows them to assess, when students work in groups, whether the five elements that ensure that the work conducted is genuinely cooperative are present. The existing instruments are designed to assess: (i) overall cooperative learning, positive interdependence, evaluation, teacher academic support or heterogeneity, as is the case with the Classroom Life Management Questionnaire (Johnson and Johnson, 1983); (ii) the reasons why teachers implement or not cooperative learning, such as the Cooperative Learning Implementation Questionnaire (Centre for the Study of Learning and Performance, 1998); (iii) the Conditions for Cooperative Learning (Hijzen, Boekaerts and Vedder, 2006), which assesses teaching behavior and academic support tasks; and (iv) the benefits of cooperative learning (Lopes, Silva and Rocha, 2014). The instruments referenced in the literature that seek to assess the five basic elements of cooperative learning, such as the Cooperative Learning Observational Schedule (Veenman et al., 2002), are not intended for students but for external evaluators; and the Quality of Cooperative Learning (Hijzen, Boekaerts and Vedder, 2006) which assesses only positive interdependence and social skills. There are questionnaires that allow for the assessment of the five basic elements of cooperative groups (Bay and Çetin, 2012; Fernandez-Rio et al., 2017), both validated for non-Portuguese contexts. This brief literature review indicates a gap regarding instruments that assess the five basic elements of cooperative learning in Portuguese language and in higher education.

Based on the above, the aim of this study was to adapt and validate the Cooperative Learning Questionnaire (Fernandez-Rio et al., 2017) for Portuguese and Brazilian higher education students.

METHOD

PARTICIPANTS

Two samples of students were selected, namely one Portuguese and one Brazilian. Students responded to the questionnaire during a class period. The Portuguese sample includes 493 university students from different cycles and higher education courses (Communication Sciences, Communication and Multimedia, Nutrition Sciences, Nursing, Humanities, and Psychology). Most participants attended the first year of their undergraduate studies (n = 286; 58.0%), followed by participants in their third year (n = 116; 23.5%), second year (n = 78; 15.8%) and, finally, their master’s degrees (n = 13; 2.6%). The average age was 20.01 years (SD = 4.20; Min = 17; Max = 58) and the majority of the sample was female (n = 400; 81.1%). The Brazilian sample included 414 university students from different cycles and higher education courses (almost the majority - 40.8% - attended Medicine, and 3.4% attended Biomedicine; the rest are distributed across the courses of Administration, Learning, Quality Control, Sports, Life Sciences, and Technology). Most of the participants were in the first year of their degree (n = 267; 64.5%), followed by those in the second year of their degree (n = 103; 24.9%), the third year (n = 30; 7.2%), the fourth year (n = 11; 2.7%) and, finally, the fifth year (n = 3; 0.7%). Average age was 21.72 years (SD = 5.44; Min = 17; Max = 52) and the majority of the sample was female (n = 267; 64.5%).

INSTRUMENTS

Sociodemographic questionnaire

The sociodemographic questionnaire included the variables gender, age, academic year degree, and course.

Cooperative Learning Scale

The version of the Cooperative Learning Scale (Fernandez-Rio et al., 2017) has 20 items and five subscales (positive interdependence, social skills, face-to-face promotive interaction, individual and group accountability, and group processing). However, in a conference of experts, it was considered useful to break down some items (for example, the original item consisted of We work on dialogue, listening skills and/or debate; the split items consisted of: a) we debate each other’s ideas; b) we listen attentively to each other’s opinions and points of view). It was then decided to include five new items resulting from this development. The original version plus the five items (25 items) were translated and back-translated according to the guidelines of Brislin’s method (1970) and the International Test Commission guidelines for test adaptation: A criterion checklist (Hernández et al., 2020). A five-point Likert scale response format was used (from 1 = strongly disagree, to 5 = strongly agree), being considered the best option under a statistical point of view: it can reduce the level of frustration of impatient respondents and increase the percentage and quality of responses (Allen and Seaman, 2007).

The authors of the original scale found Cronbach’s alpha values between 0.72 and 0.89. The English version (Fernandez-Rio et al., 2021), which includes 15 items and the same five subscales, also reports good Cronbach’s alpha values (0.72-0.90) (Table 1).

Table 1 -
Cronbach’s alpha values for the Cooperative Learning Scale

Sub-scales	Validated version for Spanish	Validated English version
Positive interdependence	0.72	0.78
Individual and group accountability	0.79	0.72
Face-to-face promotive interaction	0.76	0.75
Social skills	0.74	0.83
Group processing	0.75	0.84
Cooperative Learning Scale	0.89	0.90

• Source: prepared by the authors.

PROCEDURES

The project was submitted to and approved by the Research Ethics Committee of Universidade de Trás-os-Montes e Alto Douro (UTAD; Doc76-CE-UTAD-2021). The research protocol for this study was built on Google Forms, whose link was made available using the snowball technique for Portuguese and Brazilian university students of both genders. This protocol included the aim of the study, the voluntary nature of participation, as well as the informed consent guaranteeing anonymity of participants and data confidentiality.

Specifically, in the first phase, two authors of this study translated the original version from Spanish into Portuguese. Then, in a second phase, two native Spanish-speaking experts back-translated the instrument, concluding that the two versions did not differ significantly in terms of content and meaning. CLS was initially applied in a pilot study, both in Portugal and Brazil, using a convenience sample of 65 participants (30 men and 35 women) aged between 18 and 40 years old. The results of this study showed that the version tested was easy to understand.

RESULTS

DATA ANALYSIS

Descriptive analysis indicators were used to characterize the sample and the items (mean, standard deviation, minimum, maximum). Kurtosis and asymmetry values assessed the normal distribution of the variables.

Confirmatory factor analysis, with maximum likelihood estimation, was carried out using the corrected ꭓ² of Satorra and Bentler (2010) (χ² < 2) using AMOS 27 (Arbuckle, 2020). The comparative fit index (CFI), the Tucker-Lewis index (TLI), and the root mean square error of approximation (RMSEA) were used to assess the overall fit of the global model. Higher CFI and TLI values and lower RMSEA values indicate better fit. CFI and TLI ≥ 0.90 and RMSEA ≤ 0.08 were criteria for adequate model fit, while CFI and TLI ≥ 0.95 and RMSEA ≤ 0.06 were criteria for very well-fitting models (Hu and Bentler, 1999). Browne and Cudeck (1992) used the concept of “close fit”, and PCLOSE provides a test of this fit (≥ 0.05). Standardized root mean square (SRMR) makes it possible to assess the average magnitude of the discrepancies between the observed and expected correlations as an absolute measure of the model’s fit criterion and must have a value < 0.08 (Kline, 2015). Significance was set at p < 0.050.

Reliability was assessed by the value of Cronbach’s alpha (α > 0.700) (Salkind and Frey, 2019). Convergent validity was calculated by the values of composite reliability (CR > 0.700) (Netemeyer, Bearden and Sharma, 2003) and average variance extracted (AVE > 0.500) (Cheung and Wang, 2017). Discriminant validity was assessed by the square root of AVE values, which should be greater than the correlation of the specific construct with any of the other constructs (Cheung and Wang, 2017).

Differences were calculated using Student’s t-test for independent samples, with significance set at p < 0.05.

PORTUGUESE VERSION

Preliminary analyses

Table 2 shows the descriptive statistics of CLS items for the total sample. The asymmetry and kurtosis values guarantee the normal distribution of the items. Table 3 shows the mean of the scale if the item is excluded, the variance of the scale if the item is excluded, the total corrected correlation of the item, and Cronbach’s alpha if the item is excluded. Table 4 shows the correlations between the 25 items and the total CLS.

Table 2 -
Cooperative Learning Scale: frequencies

Items	Portuguese version				Brazilian Version
	M (1-5)	SD	Sk	Kt	M (1-5)	SD	Sk	Kt
1. In class, we discuss each other’s ideas.	3.61	0.93	-0.81	0.21	2.29	1.62	0.77	-1.04
2. In class, we analyze our group’s strengths and weaknesses.	3.06	0.99	-0.34	-0.62	2.26	1.33	0.87	-0.25
3. In class, everyone in the group helps each other to carry out the activity.	3.53	1.09	-0.54	-0.64	2.42	1.56	0.71	-1.01
4. In class, we care about the success of everyone in the group.	3.77	1.02	-0.67	-0.17	2.53	1.64	0.53	-1.33
5. In class, each member of the group is responsible for doing their part for the common work.	4.02	0.85	-1.10	1.59	2.49	1.74	0.56	-1.45
6. In class, we ask each other for help.	3.94	0.88	-0.89	0.79	2.45	1.71	0.60	-1.38
7. In class, we evaluate both our own performance and that of our group mates.	3.45	0.98	-0.47	-0.30	2.28	1.53	0.78	-0.86
8. In class, we can’t finish an activity without input from our colleagues,	3.13	0.99	-0.07	-0.66	2.25	1.19	0.80	0.08
9. In class, we motivate each other during group activities.	3.60	0.97	-0.64	0.05	2.39	1.59	0.69	-1.08
10. In class, group work helps everyone learn better.	3.81	1.01	-0.89	0.39	2.57	1.69	0.46	-1.46
11. In class, we listen carefully to each other’s opinions and points of view.	3.96	0.80	-1.03	1.68	2.29	1.64	0.80	-1.05
12. In class, we find ways to improve the group’s performance.	3.70	0.90	-0.87	0.71	2.39	1.64	0.67	-1.19
13. In class, we share materials, information and knowledge when we carry out group activities.	4.07	0.78	-1.01	1.68	2.54	1.85	0.47	-1.67
14. In class, we show that we trust each other.	3.34	0.89	-0.36	0.15	2.17	1.31	0.85	-0.28
15. In class, each member of the group contributes even if they don’t like the activity.	3.43	1.00	-0.48	-0.42	2.14	1.39	1.01	-0.22
16. In class, we accept differences of opinion.	4.08	0.79	-1.17	2.45	2.46	1.72	0.58	-1.40
17. In class, at the end of an activity or task, we reflect on whether we have achieved our objectives.	3.50	0.99	-0.66	-0.16	2.31	1.42	0.75	-0.65
18. In class, everyone’s ideas are fundamental to the success of the group’s work.	4.17	0.76	-1.19	2.48	2.59	1.79	0.44	-1.62
19. In class, we sit opposite each other to communicate better.	3.06	1.18	-0.22	-0.88	2.24	1.21	0.79	-0.01
20. In class, each member takes responsibility for the success of the group.	3.65	0.95	-0.76	0.37	2.20	1.41	0.91	-0.43
21. In class, we listen carefully to each other’s ideas.	3.96	0.80	-1.01	1.52	2.54	1.62	0.49	-1.31
22. In class, we resolve conflicts in the group.	3.46	0.94	-0.50	0.01	2.25	1.33	0.81	-0.36
23. In class, we help each other to achieve the objectives of the work.	3.92	0.82	-0.86	0.99	2.35	1.68	0.69	-1.24
24. In class, we encourage each other to overcome difficulties.	3.74	0.88	-0.74	0.58	2.39	1.61	0.65	-1.16
25. In class, we make decisions by consensus within the group.	3.92	0.83	-0.89	1.15	2.30	1.66	0.77	-1.11
Total	3.68	0.55	0.59	0.59	2.36	0.92	1.03	0.51
α	0.922				0.921

• Source: prepared by the authors.
• Min: minimum; Max: maximum; M: mean; SD: standard deviation; Sk: skewness; Kt: kurtosis; α: Cronbach’s alpha.

Table 3 -
Cooperative Learning Scale: psychometric properties

	Portuguese version				Brazilian Version
	Average scale if item excluded	Scale variance if item excluded	Correlation total corrected item	Cronbach’s alpha if item excluded	Average scale if item excluded	Scale variance if item excluded	Correlation total corrected item	Cronbach’s alpha if item excluded
1. In class, we discuss each other’s ideas.	88.28	174.22	0.457	0.920	56.79	498.78	0.399	0.921
2. In class, we analyze our group’s strengths and weaknesses.	88.83	172.28	0.505	0.919	56.82	500.55	0.471	0.920
3. In class, everyone in the group helps each other to carry out the activity.	88.37	168.14	0.602	0.918	56.66	493.09	0.502	0.919
4. In class, we care about the success of everyone in the group.	88.13	168.26	0.642	0.917	56.55	486.15	0.573	0.918
5. In class, each member of the group is responsible for doing their part for the common work.	87.87	175.60	0.444	0.920	56.59	487.05	0.523	0.919
6. In class, we ask each other for help.	87.96	173.92	0.498	0.919	56.62	485.52	0.554	0.918
7. In class, we evaluate both our own performance and that of our group mates.	88.45	173.06	0.477	0.920	56.79	491.83	0.533	0.919
8. In class, we can’t finish an activity without input from our colleagues.	88.76	174.14	0.428	0.921	56.82	513.38	0.288	0.922
9. In class, we motivate each other during group activities.	88.30	170.98	0.569	0.918	56.69	487.30	0.577	0.918
10. In class, group work helps everyone learn better.	88.09	170.54	0.561	0.918	56.51	482.08	0.609	0.917
11. In class, we listen carefully to each other’s opinions and points of view.	87.94	172.94	0.602	0.918	56.79	484.19	0.602	0.917
12. In class, we find ways to improve the group’s performance.	88.20	169.49	0.684	0.916	56.69	479.85	0.666	0.916
13. In class, we share materials, information, and knowledge when we carry out group activities.	87.83	175.76	0.483	0.920	56.53	476.35	0.625	0.917
14. In class, we show that we trust each other.	88.55	172.07	0.573	0.918	56.91	502.59	0.441	0.920
15. In class, each member of the group contributes even if they don’t like the activity.	88.46	171.99	0.508	0.919	56.94	494.71	0.543	0.918
16. In class, we accept differences of opinion.	87.81	174.15	0.555	0.919	56.62	482.70	0.591	0.918
17. In class, at the end of an activity or task, we reflect on whether we have achieved our objectives.	88.39	170.92	0.556	0.918	56.77	497.06	0.493	0.919
18. In class, everyone’s ideas are fundamental to the success of the group’s work.	87.72	175.32	0.515	0.919	56.48	481.47	0.580	0.918
19. In class, we sit opposite each other to communicate better.	88.84	175.39	0.306	0.924	56.83	508.43	0.375	0.921
20. In class, each member takes responsibility for the success of the group.	88.25	170.83	0.584	0.918	56.88	495.23	0.525	0.919
21. In class, we listen carefully to each other’s ideas.	87.94	171.19	0.689	0.917	56.53	485.22	0.595	0.917
22. In class, we resolve conflicts in the group.	88.44	173.04	0.504	0.919	56.82	500.03	0.477	0.919
23. In class, we help each other achieve the work’s objectives.	87.98	170.17	0.720	0.916	56.72	476.86	0.688	0.916
24. In class, we encourage each other to overcome difficulties.	88.15	170.10	0.674	0.917	56.69	481.13	0.660	0.916
25. In class, we make decisions by consensus within the group.	87.98	172.26	0.616	0.918	56.78	485.55	0.575	0.918

• Source: prepared by the authors.

Table 4 -
Cooperative Learning Scale: total item correlations

Items	Portuguese Version	Brazilian Version
	CLS Total	CLS Total
1. In class, we discuss each other’s ideas.	0.510**	0.457**
2. In class, we analyze our group’s strengths and weaknesses.	0.558**	0.515**
3. In class, everyone in the group helps each other to carry out the activity.	0.652**	0.552**
4. In class, we care about the success of everyone in the group.	0.685**	0.620**
5. In class, each member of the group is responsible for doing their part for the common work.	0.493**	0.577**
6. In class, we ask each other for help.	0.546**	0.605**
7. In class, we evaluate both our own performance and that of our group mates.	0.532**	0.579**
8. In class, we can’t finish an activity without the input of our colleagues.	0.487**	0.335**
9. In class, we motivate each other during group activities.	0.616**	0.622**
10. In class, group work helps everyone learn better.	0.610**	0.654**
11. In class, we listen carefully to each other’s opinions and points of view.	0.639**	0.646**
12. In class, we find ways to improve the group’s performance.	0.719**	0.704**
13. In class, we share materials, information, and knowledge when we carry out group activities.	0.526**	0.673**
14. In class, we show that we trust each other.	0.617**	0.486**
15. In class, each member of the group contributes even if they don’t like the activity.	0.561**	0.585**
16. In class, we accept differences of opinion.	0.594**	0.639**
17. In class, at the end of an activity or task, we reflect on whether we have achieved our objectives.	0.605**	0.539**
18. In class, everyone’s ideas are fundamental to the success of the group’s work.	0.555**	0.630**
19. In class, we sit opposite each other to communicate better.	0.383**	0.419**
20. In class, each member takes responsibility for the success of the group.	0.629**	0.569**
21. In class, we listen carefully to each other’s ideas.	0.720**	0.640**
22. In class, we resolve conflicts in the group.	0.554**	0.521**
23. In class, we help each other achieve the work’s objectives.	0.748**	0.725**
24. In class, we encourage each other to overcome difficulties.	0.708**	0.698**
25. In class, we make decisions by consensus within the group.	0.653**	0.622**

• Source: prepared by the authors.
• **p < 0.001.

Confirmatory factor analysis

A confirmatory factor analysis (CFA) of the 25 items and five factors was carried out to confirm the model proposed by the authors. CFA showed an unacceptable model fit [χ² (270) = 3.326; p <0.001; CFI = 0.864; TLI = 0.849; RMSEA = 0.069; PCLOSE = 0.000; SRMR = 0.043]; although the modification indices suggested correlations between some errors, it was not possible to find an acceptable model (Figure 1).

Figure 1 -
Cooperative Learning Scale Portuguese Version (I)

As a good fit model was not found, it was decided to exclude the item whose Cronbach’s alpha value would increase if the item was excluded (item 19) (Table 2); it was also decided to exclude the item that correlated with the total scale below 0.500 (item 8) (Table 3t); and to exclude the items that were least saturated in their respective factors (items 1, 6, 21, 7, 8, and 19) (Figure 1). Finally, items 15, 23, and 25 were excluded in order to keep only 3 items per factor, since the model would present a worse fit if these three items remained. A good fit was found [χ² (80) = 2.654; p <0.001; CFI = 0.953; TLI = 0.926; RMSEA = 0.058; PCLOSE = 0.080; SRMR = 0.041], although five correlations were established between the errors of the items, all within the same factor, except for the correlation between items 3 and 20, which belong to different factors (Figure 2).

Figure 2 -
Cooperative Learning Scale Portuguese Version (II)

Convergent and divergent validity

Reliability was assessed by Cronbach’s alpha, with values above those recommended. Convergent validity for CLS was assessed by CR values (whose values were above the reference values) and AVE values (whose values were above 0.500) (Cheung and Wang, 2017). Discriminant validity was assessed by the square root of the AVE values (Table 5). All these values were higher than the correlation values for each construct, except for total CLS.

Table 5 -
Mean, standard deviation, Cronbach’s alpha, correlations, composite reliability, average variance extracted and square root of average variance extracted of the Cooperative Learning Scale and its subscales - Portuguese version

	M	DP	α	CR	AVE	1	2	3	4	5	6
1 CLS Total	3.71	0.56	0.883	0.956	0.595	0.771
2 CLS Social Skills	3.83	0.65	0.650	0.816	0.600	0.786**	0.903
3 CLS Group processing	3.42	0.75	0.688	0.828	0.617	0.812**	0.584**	0.785
4 CLS Positive Interdependence	3.92	0.65	0.739	0.777	0.537	0.806**	0.549**	0.537**	0.733
5 CLS Face-to-face promotive interaction	3.56	0.74	0.735	0.850	0.655	0.820**	0.570**	0.583**	0.565**	0.809
6 CLS Individual and group accountability	3.82	0.71	0.610	0.796	0.566	0.791**	0.485**	0.522**	0.621**	0.549**	0.752

• Source: prepared by the authors.
• **p < 0.001; M: mean; SD: standard deviation; α: Cronbach’s alpha; CR: composite reliability; r: Pearson’s correlation; AVE: average variance extracted; bold: square root of average variance extracted.

BRAZILIAN VERSION

Preliminary analyses

Table 2 shows the descriptive statistics of the CLS items for the total sample. The asymmetry and kurtosis values guarantee the normal distribution of the items. Table 3 shows the mean of the scale if the item is excluded, the variance of the scale if the item is excluded, the total corrected correlation of the item, and Cronbach’s alpha if the item is excluded. Table 4 shows the correlations between the 25 items and the total CLS.

Confirmatory factor analysis

A CFA of the 25 items and five factors was carried out to confirm the model proposed by the authors. CFA showed an unacceptable model fit [χ² (265) = 2.408; p < 0.001; CFI = 0.892; TLI = 0.878; RMSEA = 0.058; PCLOSE = 0.009; SRMR = 0.047]; although modification indices suggested correlations between some errors, it was not possible to find an acceptable model (Figure 3).

Figure 3 -
Cooperative Learning Scale (CLS) Brazilian Version (I)

As a good fit model was not found, it was decided to exclude the item whose Cronbach’s alpha value would increase if the item was excluded (item 8) (Table 2); it was also decided to exclude the items that correlated with the total scale below 0.500 (items 1, 14, and 19) (Table 3); and to exclude the items that were least saturated in their respective factors (items 2, 6, 7, 20, 21, and 23) (Figure 3). This kept 3 items per factor; the model would present a worse fit if these items remained. A good fit was found [χ² (79) = 2.227; p <0.001; CFI = 0.951; TLI = 0.934; RMSEA = 0.054; PCLOSE = 0.236; SRMR = 0.038], although a correlation was established between two errors of two items within the same factor (Figure 4).

Figure 4 -
Cooperative Learning Scale Brazilian Version (II)

Convergent and divergent validity

Reliability was assessed by Cronbach’s alpha, with values below those recommended, especially for the social skills subscale. Convergent validity for CLS was assessed by CR values (whose values were above the reference values) and AVE values (whose values were above 0.500, except for the total) (Cheung and Wang, 2017). Discriminant validity was assessed by the square root of AVE values (Table 4). All these values were higher than the correlation values for each construct, except for total CLS.

Differences between the portuguese and brazilian versions

Looking at Table 1, we can see that the mean values of the Portuguese items are higher than the mean values of the Brazilian items. The final versions of the total CLS as well as its subscales differ in terms of the items included in them, except for the “social skills” and “positive interdependence” subscales. Therefore, only these two can be compared; the differences between them are statistically significant, with the Brazilian sample showing significantly lower values than the Portuguese one. With regard to the “social skills” subscale, the mean value of the Portuguese version is 3.83 (SD = 0.65) and that of the Brazilian version is 2.33 (SD = 1.16) [t(905) = 24.56; p < 0.001; d = 0.92]. With regard to the “positive interdependence” subscale, the mean value of the Portuguese version is 3.92 (SD = 0.65) and that of the Brazilian version is 2.52 (SD = 1.31) [t(905) = 20.93; p < 0.001; d = 1.01] (Table 6).

Table 6 -
Mean, standard deviation, Cronbach’s alpha, correlations, composite reliability , average variance extracted and square root of average variance extracted of the Cooperative Learning Scale and its subscales - Brazilian version

	M	SD	α	CR	AVE	1	2	3	4	5	6
1 CLS Total	2.40	1.03	0.893	0.919	0.403	0.709
2 CLS Social Skills	2.43	1.27	0.581	0.780	0.544	0.828**	0.738
3 CLS Group processing	2.33	1.18	0.610	0.796	0.566	0.844**	0.678**	0.752
4 CLS Positive Interdependence	2.52	1.31	0.626	0.799	0.572	0.840**	0.669**	0.634**	0.756
5 CLS Face-to-face promotive interaction	2.43	1.26	0.678	0.824	0.609	0.844**	0.639**	0.628**	0.640**	0.780
6 CLS Individual and group accountability	2.40	1.21	0.614	0.797	0.567	0.830**	0.598**	0.605**	0.604**	0.655**	0.753

• Source: prepared by the authors.
• **p < 0.001; M: mean; SD: standard deviation; α: Cronbach’s alpha; CR: composite reliability; r: Pearson’s correlation; AVE: average variance extracted; bold: square root of average variance extracted.

When comparing the averages of the total and subscales of CLS in relation to gender within each country, no statistically significant differences are found in any of the dimensions. However, by joining the two samples together and comparing gender in relation to the subscales that retain the same items (“social skills” and “positive interdependence”), statistically significant differences were found between the genders, with the female gender showing significantly higher values. With regard to the “social skills” subscale, the average value for the female sample is 3.22 (SD = 1.16) and for the male sample, it is 2.30 (SD = 1.21) [t(902) = 3.07; p = 0.002; d = 1.18]. With regard to the positive interdependence subscale, the average value for the female sample was 3.35 (SD = 1.20), and for the male sample 3.09 (SD = 1.29) [t(902) = 2.79; p = 0.006; d = 1.22].

DISCUSSION

The aim of this study was to adapt and validate the original version of CLS (Fernandez-Rio et al., 2017) for Portuguese and Brazilian higher education students. To this end, after obtaining the necessary authorizations, the instrument was translated and back-translated from Spanish to Portuguese and from Portuguese to Spanish, respectively, and the first and last versions were compared.

Since it was not possible to find a good fit model similar to that proposed by the authors of the original version with the 25 items (Fernandez-Rio et al., 2017), a CFA was carried out, and a structure similar to that of the English version of the same scale was found (Fernandez-Rio et al., 2021). Specifically, 15 items were distributed across five factors (each with three items), all subordinated to a second-order factor. The exclusion of two items per factor was carried out on statistical criteria, i.e., the items that allowed for a better fit remained in the model, in light of what the authors of the English version did (Fernandez-Rio et al., 2021). Therefore, CLS, in the context of higher education for the Portuguese and Brazilian populations, is a valid instrument. Results showed well-defined factors corresponding to a priori expectations, with all fit indices and information criteria indicating that CLS is a valid instrument for the Portuguese-language context (Portugal and Brazil).

The different versions (Spanish/original, English, Portuguese, and Brazilian) show that the items are equivalent and that the factors have the same number of items in all versions, with only the original version having one additional item per factor (Table 7). This scale includes the five basic elements of cooperative learning (social skills; group processing; positive interdependence; face-to-face promotive interaction; and individual and group accountability), unlike previous instruments (Johnson and Johnson, 1983; Center for the Study of Learning and Performance, 1998; Veenman et al., 2002; Hijzen, Boekaerts and Vedder, 2006).

Table 7 -
Comparison of the four versions of the Cooperative Learning Scale

Validated version for Spanish	Validated English version	Validated version for Portugal	Validated version for Brazil
In class: 1. We work on dialog, listening skills, and/or debate. 2. We put them together so that the whole group knows what is being done. 3. The help of my colleagues to complete the tasks is important. 4. Group members relate and interact during tasks. 5. Each member of the group must participate in the group’s tasks. 6. Exposing and defending ideas, knowledge, and points of view to colleagues. 7. We make decisions by consensus among the group members. 8. We can’t finish an activity without the contributions of our colleagues. 9. Interaction between group members is necessary to complete the task. 10. Each component of the group must make an effort in the group’s activities. 11. We listen to the opinions and points of view of our colleagues. 12. We discuss ideas among group members. 13. It is important to share materials, information... to do the tasks. 14. We relate to each other to do the activities. 15. Every member of the group should try to take part, even if they don’t like the task. 16. We reach agreements in the face of different opinions or conflicts. 17. We reflect individually and together within the group. 18. The better each member of the group does their job, the better result the group gets. 19. We work directly with each other. 20. Each member of the group must do their part of the group’s work to complete the task.	In the classroom: 1. We work on discussing, debating, and listening to others. 2. We talk to each other to make sure that everyone in the group knows what is being done. 3. We cannot finish the tasks without the groupmates’ contributions. 4. Groupmates relate with each other and interact during the tasks. 5. Every group member has to participate in the group’s tasks. 6. We present and defend ideas and individual points of view in front of the groupmates. 7. We reach agreements within the group to make decisions. 8. It is important to share resources, and information to complete the tasks. 9. Interaction among groupmates is necessary to complete the tasks. 10. Every group member must strive to try hard in the group’s activities. 11. We listen to each other’s ideas, opinions, and points of view. 12. Groupmates debate ideas and opinions. 13. The better each group member completes his/her task, the better it is for the group. 14. We work face-to-face with our groupmates. 15. It is important for every group member to try to participate, even if he/she does not like the task.	In class: 1. We listen carefully to each other’s opinions and points of view. 2. We analyze our group’s strengths and weaknesses. 3. Everyone in the group helps each other to carry out the activity. 4. We motivate each other during group activities. 5. Each member of the group is responsible for doing their part in the common work. 6. We accept differences of opinion. 7. We find ways to improve the group’s performance. 8. We share materials, information, and knowledge when we carry out group activities. 9. We show that we have confidence in each other. 10. Group work helps everyone learn better. 11. We resolve conflicts in the group. 12. At the end of an activity or a task, we reflect on whether we have met the intended objectives. 13. Everyone’s ideas are fundamental to the success of the group’s work. 14. We encourage each other to overcome difficulties. 15. Each member takes responsibility for the success of the group.	In class: 1. Everyone in the group helps each other to carry out the activity. 2. We care about the success of everyone in the group. 3. Each member of the group is responsible for doing their part in the common work. 4. We motivate each other during group activities. 5. Group work helps everyone learn better. 6. In class, we listen carefully to each other’s opinions and points of view. 7. We find ways to improve the group’s performance. 8. We share materials, information, and knowledge when we carry out group activities. 9. Each member of the group contributes even if they don’t like the activity. 10. We accept differences of opinion. 11. At the end of an activity or a task, we reflect on whether we have met the intended objectives. 12. Everyone’s ideas are fundamental to the success of the group’s work. 13. We resolve conflicts in the group. 14. We encourage each other to overcome difficulties. 15. We make decisions by consensus in the group.
Social skills: 1, 6, 11, 16; Group processing: 2, 7, 12, 17; Positive interdependence: 3, 8, 13, 18; Face-to-face promotive interaction: 4, 9, 14, 19; Individual and group accountability: 5, 10, 15, 20	Social skills = items 1, 6, 11; group processing = items 2, 7, 12; positive interdependence = items 3, 8, 13; Face-to-face promotive interaction = items 4, 9, 14; individual and group accountability = items 5, 10, 15.	Social skills: items 1, 6, 11; Group processing: items 2, 7, 12; Positive interdependence: items 3, 8, 13; Face-to-face promotive interaction: items: 4; 9; 14; Individual and group accountability: Items: 5, 10; 15	Social skills: items 1,b, 11; Group processing: items 2, 7, 12; Positive interdependence: items 3, 8, 13; Face-to-face promotive interaction: items: 4; 9; 14; Individual and group accountability: Items: 5, 10; 15

• Source: prepared by the authors.

The Portuguese and Brazilian versions (with 15 items) share 12 common items and three different ones, although consistent in content (Table 8).

Table 8 -
Cooperative Learning Scale : Portuguese and Brazilian versions

	Portuguese version	Brazilian version
1. In class, we debate each other’s ideas.
2. In class, we analyze our group’s strengths and weaknesses.	X
3. In class, everyone in the group helps each other to carry out the activity.	X	X
4. In class, we care about the success of everyone in the group.		X
5. In class, each member of the group is responsible for doing their part for the common work.	X	X
6. In class, we ask each other for help.
7. In class, we evaluate both our own work and that of our group mates.
8. In class, we can’t finish an activity without input from our colleagues.
9. In class, we motivate each other during group activities.	X	X
10. In class, group work helps everyone learn better.	X	X
11. In class, we listen carefully to each other’s opinions and points of view.	X	X
12. In class, we find ways to improve the group’s performance.	X	X
13. In class, we share materials, information, and knowledge when we carry out group activities.	X	X
14. In class, we show that we trust each other.	X
15. In class, each member of the group contributes even if they don’t like the activity.		X
16. In class, we accept differences of opinion.	X	X
17. In class, at the end of an activity or task, we reflect on whether we have achieved our objectives.	X	X
18. In class, everyone’s ideas are fundamental to the success of the group’s work.	X	X
19. In class, we sit opposite each other to communicate better.
20. In class, each member takes responsibility for the success of the group.	X
21. In class, we celebrate our successes.
22. In class, we resolve conflicts in the group.	X	X
23. In class, we help each other achieve the work’s objectives.
24. In class, we encourage each other to overcome difficulties.	X	X
25. In class, we make decisions by consensus within the group.		X

• Source: prepared by the authors.

It is observed that Portuguese students, when compared to Brazilian ones, have higher averages for all items in CLS and in the two subscales that can be compared. Furthermore, females have higher perceptions of cooperative work than males in both countries combined. These results suggest that Portuguese students perceive themselves as working more cooperatively than Brazilian students. An analysis of the results from PISA 2015 (OECD, 2017b) shows that Brazilian students ranked second to last out of 72 participating countries in collaborative problem-solving (CPS). In addition, girls in all the countries participating in PISA 2015 performed significantly better in CPS than males. Higher results regarding Portuguese students’ perceptions of cooperative work were also found in a study in which they were compared with Spanish students. Portuguese students perceived themselves as much more convinced that they work cooperatively in class than Spanish students (Fernandez-Rio et al., 2021). However, students who participated in PISA were not higher education students; nevertheless, these results may have some relationship with the scores obtained in CLS by Portuguese and Brazilian students in terms of overall scores, scores in the “social skills” and “positive interdependence” subscales, as well as those obtained by female participants from both countries on these subscales.

The results obtained by Portuguese students regarding cooperative group work may indicate an illusory perception of what it entails to work cooperatively, as they may lack true awareness of their performance in group work (Pazicni and Bauer, 2014). Therefore, the Dunning-Kruger effect (Kruger and Dunning, 1999) may partly explain these results. In fact, students may not be aware of what it takes to work cooperatively (Pazicni and Bauer, 2014), or, cumulatively, the social desirability bias may also be present (Kwak, Ma and Kim, 2021).

The significantly lower results obtained by Brazilian students compared to Portuguese ones can be explained by the fact that, despite technological advancements that provide easy access to all types of information worldwide, and the progressive change in the profile of higher education students, the traditional teaching system, with lectures as the primary pedagogical model, is still prevalent in Brazilian higher education (Troncarelli and Faria, 2014).

FINAL CONSIDERATIONS

The results have shown that CLS, in the context of higher education in both Portugal and Brazil, is a valid instrument. Well-defined factors were found that align with a priori expectations, with all fit indices and information criteria indicating that CLS is valid for assessing the five basic elements of cooperative learning in Portuguese language contexts in Portugal and Brazil in higher education. It could potentially serve as a useful tool for higher education teachers to understand their students’ perceptions of their learning, whether it is collaborative or not.

We suggest the development of future studies using CLS, with new samples drawn from broader and more diverse participant pools, augmented by the use of different measures to evaluate the implementation of cooperative learning.

REFERÊNCIAS

• ALLEN, Elaine; SEAMAN, Christopher. Likert scales and data analyses. Quality Progress, v. 40 n.7, p. 64-65. 2007. Disponível em: Disponível em: https://rube.asq.org/quality-progress/2007/07/statistics/likert-scales-and-data-analyses.html Acesso em: 01 jun. 2021.
  » https://rube.asq.org/quality-progress/2007/07/statistics/likert-scales-and-data-analyses.html
• ARBUCKLE, James. Amos (Version 27.0). Computer Program. Chicago, IL: SPSS/IBM, 2020.
• BALKCOM, Stephen. Cooperative learning. Washington, DC: Office of Research, 1992.
• BAY, Erdal; ÇETIN, Bayram. Development of cooperative learning process scale (CLPS). Journal of Human Sciences, [s. l.], v. 9, n. 1, p. 1063-1075, 2012. Disponível em: Disponível em: https://www.j-humansciences.com/ojs/index.php/IJHS/article/view/2057 Acesso em: 05 jan. 2021.
  » https://www.j-humansciences.com/ojs/index.php/IJHS/article/view/2057
• BRISLIN, Richard. Back-Translation for Cross-Cultural Research. Journal of Cross-Cultural Psychology, v. 1 n. 3, p. 185-216, 1970. Disponível em: Disponível em: https://journals.sagepub.com/doi/abs/10.1177/135910457000100301 Acesso em: 05 jan. 2021.
  » https://journals.sagepub.com/doi/abs/10.1177/135910457000100301
• BROWNE, Michael; CUDECK, Robert. Alternative ways of assessing model fit. Sociological Methods & Research, v. 21, n. 2, p. 230-258, 1992. Disponível em: Disponível em: https://journals.sagepub.com/doi/pdf/10.1177/0049124192021002005?casa_token=kIsweJjfi7kAAAAA:NWs_nB0QnAa0sKTTZtmxyrwNGawMtAtD-9HONbNyYTpge6HXEQz404M3ymDUXEQl7xCXl3zIHdA Acesso em: 07 set. 2021.
  » https://journals.sagepub.com/doi/pdf/10.1177/0049124192021002005?casa_token=kIsweJjfi7kAAAAA:NWs_nB0QnAa0sKTTZtmxyrwNGawMtAtD-9HONbNyYTpge6HXEQz404M3ymDUXEQl7xCXl3zIHdA
• CARE, Esther; KIM, Helyn; VISTA, Alvin; ANDERSON, Kate. Education system alignment for 21st century skills: focus on assessment. Washington, DC: Brookings Institute, 2018.
• CARR, Rodney; PALMER, Stuart; HAGEL, Pauline. Active learning: the importance of developing a comprehensive measure. Active Learning in Higher Education, v. 16, p. 173-186, 2015. Disponível em: Disponível em: https://journals.sagepub.com/doi/10.1177/1469787415589529 Acesso em: 01 jun. 2021.
  » https://journals.sagepub.com/doi/10.1177/1469787415589529
• CENTRE FOR THE STUDY OF LEARNING AND PERFORMANCE. The Cooperative Learning Implementation Questionnaire. Montreal, QC: Concordia University, 1998. Disponível em: Disponível em: https://www.concordia.ca/content/dam/artsci/research/cslp/docs/cliq.pdf Acesso em: 20 abr. 2021.
  » https://www.concordia.ca/content/dam/artsci/research/cslp/docs/cliq.pdf
• CHEUNG, Gordon; WANG, Chang. Current approaches for assessing convergent and discriminant validity with SEM: Issues and solutions. Academy of Management Proceedings, v. 2017, n. 1, p. 12706, 2017. https://doi.org/10.5465/ambpp.2017.12706abstract
  » https://doi.org/10.5465/ambpp.2017.12706abstract
• COHEN, Elizabeth. Restructuring the classroom: Conditions for productive small groups. Review of Educational Research, v. 1, n. 64, p. 1-35. 1994. Disponível em: https://journals.sagepub.com/doi/Disponível em: 10.3102/00346543064001001 Acesso em: 15 fev. 2021.
  » https://journals.sagepub.com/doi/10.3102/00346543064001001
• CONLEY, David. Redefining college readiness. Eugene, OR: Educational Policy Improvement Center, 2007.
• CRISOL-MOYA, Emilio; ROMERO-LÓPEZ, María Asunción; CAURCEL-CARA, María Jesús. Active methodologies in higher education: perception and opinion as evaluated by professors and their students in the teaching-learning process. Frontiers in Psychology, v. 11, p. 1-10, 2020. Disponível em: Disponível em: https://www.frontiersin.org/articles/10.3389/fpsyg.2020.01703/full Acesso em: 15 fev. 2021.
  » https://www.frontiersin.org/articles/10.3389/fpsyg.2020.01703/full
• European Commission (EC). A new skills agenda. 2016. Disponível em: Disponível em: https://www.researchgate.net/publication/313251567_A_New_Skills_Agenda_for_Europe Acesso em: 09 mar. 2021.
  » https://www.researchgate.net/publication/313251567_A_New_Skills_Agenda_for_Europe
• EVANS, Carla. Measuring student success skills: a review of the literature on collaboration. Dover, NH: National Center for the Improvement of Educational Assessment, 2020.
• FERNANDEZ-RIO, Javier; CECCHINI, Jose Antonio; MENDEZ-GIMENEZ, Antonio; MENDEZ-ALONSO, David; PRIETO, Jose. Design and validation of a questionnaire to assess cooperative learning in educational contexts. Anales de Psicologia, v. 33, n. 3, p. 680-688, 2017. Disponível em: Disponível em: https://revistas.um.es/analesps/article/view/analesps.33.3.251321 Acesso em: 05 jan. 2021.
  » https://revistas.um.es/analesps/article/view/analesps.33.3.251321
• FERNANDEZ-RIO, Javier; CECCHINI, José Antonio; MORGAN, Kevin; MENDEZ-GIMENEZ, Antonio; LLOYD, Rhys. Validation of the cooperative learning scale and cooperation global factor using bifactor structural equation modelling. Psicología Educativa, v. 20, n. 10, p. 1-7, 2021. Disponível em: Disponível em: https://journals.copmadrid.org/psed/art/psed2021a2 Acesso em: 07 set. 2021.
  » https://journals.copmadrid.org/psed/art/psed2021a2
• GILLIES, Robyn. Cooperative Learning: Review of Research and Practice. Australian Journal of Teacher Education, v. 41, n. 3, p. 39-54, 2016. Disponível em: Disponível em: https://ro.ecu.edu.au/ajte/vol41/iss3/3/ . Acesso em: 20 abr. 2021.
  » https://ro.ecu.edu.au/ajte/vol41/iss3/3/
• HANNAFIN, Michael. Student-Centered Learning. In: SEEL, Norbert. Encyclopedia of the Sciences of Learning. Boston, MA: Springer, 2012. p. 3211-3214.
• HATTIE, John. Visible learning: a synthesis of over 800 meta-analyses relating to achievement. London: Routledge, 2009.
• HERNÁNDEZ, Ana; HIDALGO, María Dolores; HAMBLETON, Ronald; GÓMEZ-BENITO, Juana. International Test Commission guidelines for test adaptation: A criterion checklist. Psicothema, v. 32, n. 3, p. 390-398, 2020. Disponível em: Disponível em: https://www.psicothema.com/pi?pii=4617 Acesso em: 20 abr. 2021.
  » https://www.psicothema.com/pi?pii=4617
• HIJZEN, Daphne; BOEKAERTS, Monique; VEDDER, Paul. The relationship between the quality of cooperative learning, students’ goal preferences, and perceptions of contextual factors in the classroom. Scandinavian Journal of Psychology, v. 47, n. 1, p. 9-21, 2006. Disponível em: Disponível em: https://onlinelibrary.wiley.com/doi/10.1111/j.1467-9450.2006.00488.x Acesso em: 20 abr. 2021
  » https://onlinelibrary.wiley.com/doi/10.1111/j.1467-9450.2006.00488.x
• HU, Li-tze; BENTLER, Peter M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, v. 6, n. 1, p. 1-55, 1999. Disponível em: Disponível em: https://www.tandfonline.com/doi/abs/10.1080/10705519909540118 Acesso em: 25 nov. 2021.
  » https://www.tandfonline.com/doi/abs/10.1080/10705519909540118
• HYNES, Mike. Students-as-producers: Developing valuable student-centered research and learning opportunities. International Journal of Research Studies in Education, v. 7, n. 4, p. 1-13, 2017. Disponível em: Disponível em: https://consortiacademia.org/10-5861ijrse-2017-1858/ Acesso em: 20 abr. 2021.
  » https://consortiacademia.org/10-5861ijrse-2017-1858/
• JOHNSON, David; JOHNSON, Roger. Social interdependence and perceived academic and personal support in the classroom. The Journal of Social Psychology, v. 120, n. 1, p. 77-82, 1983. Disponível em: Disponível em: https://www.tandfonline.com/doi/abs/10.1080/00224545.1983.9712012 Acesso em: 20 abr. 2021.
  » https://www.tandfonline.com/doi/abs/10.1080/00224545.1983.9712012
• JOHNSON, David; JOHNSON, Roger. Cooperation and competition: theory and research. Edina, Minnesota: Interaction Book Company, 1989.
• JOHNSON, David; JOHNSON, Roger. Learning together and alone: cooperative, competitive, and individualistic learning. 4. ed. Boston, MA: Allyn and Bacon, 1994.
• JOHNSON, David; JOHNSON, Roger. Cooperative Learning Methods: A Metaanalysis. Minneapolis: Cooperative Learning Center at the University of Minnesota, 2000.
• JOHNSON, David; JOHNSON, Roger. Cooperative Learning in 21st Century. [Aprendizaje cooperativo en el siglo XXI]. Anales de Psicología, v. 30, n. 3, p. 841-851, 2014. Disponível em: Disponível em: https://revistas.um.es/analesps/article/view/analesps.30.3.201241 Acesso em: 20 abr. 2021.
  » https://revistas.um.es/analesps/article/view/analesps.30.3.201241
• JOHNSON, David; JOHNSON, Roger; HOLUBEC, Edythe. Cooperative learning in the classroom. Alexandria: Association for Supervision and Curriculum Development, 1994.
• JOHNSON, David; JOHNSON, Roger; HOLUBEC, Edythe. Circles of learning: cooperation in the classroom. Edina, Minn.: Interaction Book Co., 2009.
• JOHNSON, David; JOHNSON, Roger; SMITH, Karl. Cooperative learning: Improving university instruction by basing practice on validated theory. Journal on Excellence in College Teaching, v. 25, n. 3&4, p. 85-118, 2014. Disponível em: Disponível em: https://static.pseupdate.mior.ca.s3.amazonaws.com/media/links/Cooperative_learn_validated_theory.pdf Acesso em: 20 abr. 2021.
  » https://static.pseupdate.mior.ca.s3.amazonaws.com/media/links/Cooperative_learn_validated_theory.pdf
• JOHNSON, David; JOHNSON, Roger; SMITH, Karl. The State of Cooperative Learning in Postsecondary and Professional Settings. Educational Psychology Review, v. 19, n. 1, p. 15-29, 2007. Disponível em: Disponível em: https://link.springer.com/article/10.1007/s10648-006-9038-8 Acesso em: 20 abr. 2021.
  » https://link.springer.com/article/10.1007/s10648-006-9038-8
• KAGAN, Spencer. Cooperative learning. San Clemente, CA: Kagan Publishing, 1994.
• KLINE, Rex. Principles and practice of structural equation modeling. Nova Iorque: Guilford Publications, 2015.
• KRUGER, Justin; DUNNING, David. Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, v. 77, n. 6, p. 1121-1134, 1999. Disponível em: Disponível em: https://psycnet.apa.org/doiLanding?doi=10.1037%2F0022-3514.77.6.1121 Acesso em: 20 abr: 2021.
  » https://psycnet.apa.org/doiLanding?doi=10.1037%2F0022-3514.77.6.1121
• KWAK, Dong-Heon (Austin); MA, Xiao; KIM, Sumin. When does social desirability become a problem? Detection and reduction of social desirability bias in information systems research. Information & Management, v. 58, n. 7, p. 1-130, 2021. Disponível em: Disponível em: https://www.sciencedirect.com/science/article/abs/pii/S0378720621000744?via%3Dihub Acesso em: 25 nov. 2021
  » https://www.sciencedirect.com/science/article/abs/pii/S0378720621000744?via%3Dihub
• KYNDT, Eva; RAES, Elisabeth; LISMONT, Bart; TIMMERS, Fran; CASCALLAR, Eduardo; DOCHY, Filip. A meta-analysis of the effects of face-to-face cooperative learning. Do recent studies falsify or verify earlier findings? Educational Research Review, v. 10, p. 133-149, 2013. Disponível em: Disponível em: https://www.sciencedirect.com/science/article/abs/pii/S1747938X13000122?via%3Dihub Acesso em: 20 abr. 2021.
  » https://www.sciencedirect.com/science/article/abs/pii/S1747938X13000122?via%3Dihub
• LAI, Emily; VIERING, Michaela. Assessing 21st century skills: integrating research findings. National Council for Measurement in Education. Vancouver, B.C., 2012.
• LENCH, Sarah; FUKUDA, Erin; ANDERSON, Ross. Essential skills and dispositions: Developmental frameworks for collaboration, communication, creativity, and self-direction. Lexington, KY: Center for Innovation in Education at the University of Kentucky, 2015.
• LOPES, José; SILVA, Helena Santos. A aprendizagem cooperativa na sala de aula. Lisboa: LIDEL, Edições Técnicas, 2009.
• LOPES, José; SILVA, Helena Santos; ROCHA, Magda. Escala de benefícios da aprendizagem cooperativa: construção e estudos de validação. Revista de Estudios e Investigación en Psicología y Educación, v. 1, n. 1, p. 15-25, 2014. https://doi.org/10.17979/reipe.2014.1.1.18
  » https://doi.org/10.17979/reipe.2014.1.1.18
• MARTINS, Guilherme d’Oliveira; GOMES, Carlos Alberto Sousa; BROCADO, Joana; PEDROSO, José Vitor; CAMILO, José León Acosta; SILVA, Luísa Maria Ucha; ENCARNAÇÃO, Marta Manuela Guerreiro Alves da; HORTA, Maria João do Vale Costa; CALÇADA, Maria Teresa Carmo Soares; NERY, Rui Fernando Vieira; RODRIGUES, Sónia Maria Cordeiro Valente. Perfil dos alunos à saída da escolaridade obrigatória. Lisboa: Ministério da Educação/Direção Geral da Educação, 2017.
• MARZANO, Robert; MID-CONTINENT REGIONAL EDUCATIONAL LABORATORY. A theory-based meta-analysis of research on instruction. Aurora, Colo: Mid-Continent Regional Educational Laboratory, 1998.
• MASON, Lucia. Responses to anomalous data on controversial topics and theory change. Learning and Instruction, v. 11, n. 6, p. 453-483, 2001. Disponível em: Disponível em: https://www.sciencedirect.com/science/article/abs/pii/S0959475200000426?via%3Dihub Acesso em: 05 maio 2021.
  » https://www.sciencedirect.com/science/article/abs/pii/S0959475200000426?via%3Dihub
• MEHTA, Jal; FINE, Sarah. In search of deeper learning: the quest to remake the American high school. Cambridge, MA: Harvard University Press, 2019.
• NATIONAL RESEARCH COUNCIL (NRC). Assessing 21st century skills: summary of a Workshop. Washington, DC: The National Academies Press, 2011.
• NETEMEYER, Richard; BEARDEN, William; SHARMA, Subhash. Scaling procedures: Issues and applications. Thousand Oaks: Sage Publications, 2003.
• ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT (OECD). PISA 2015 Assessment and analytical framework: science, reading, mathematic, financial literacy and collaborative problem solving. Paris: OECD Publishing, 2017a.
• ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT (OECD). PISA 2015 Collaborative Problem-Solving Framework. Paris: OECD Publishing, 2017b. Disponível em: Disponível em: https://www.oecd.org/pisa/pisaproducts/ Acesso em: 20 abr. 2021.
  » https://www.oecd.org/pisa/pisaproducts/
• ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT (OECD). PISA: Preparing our youth for an inclusive and sustainable world: The OECD PISA global competence framework. Paris: OECD Publishing, 2018a. Disponível em: Disponível em: https://www.oecd.org/education/Global-competency-for-an-inclusive-world.pdf Acesso em: 20 abr. 2021.
  » https://www.oecd.org/education/Global-competency-for-an-inclusive-world.pdf
• ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT (OECD). The future of education and skills: Education 2030. Paris: OECD Publishing, 2018b. Disponível em: Disponível em: https://www.oecd.org/education/2030/E2030%20Position%20Paper%20(05.04.2018).pdf Acesso em: 01 jun. 20213.
  » https://www.oecd.org/education/2030/E2030%20Position%20Paper%20(05.04.2018).pdf
• ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT (OECD). Envisioning the future of education and jobs: trends, data and drawings. Paris: OECD Publishing, 2019.
• PANITZ, Theodore. Getting students ready for cooperative learning. Cooperative Learning and College Teaching, v. 6, n. 2, p. 7-10, 1996. Disponível em: Disponível em: https://files.eric.ed.gov/fulltext/ED448443.pdf Acesso em: 21 abr. 2021.
  » https://files.eric.ed.gov/fulltext/ED448443.pdf
• PATEŞAN, Marioara; BALAGIU, Alina; ZECHIA, Dana. The benefits of cooperative learning. international conference. Knowledge-Based Organization, v. 22, n. 2, p. 478-483, 2016. Disponível em: Disponível em: https://sciendo.com/article/10.1515/kbo-2016-0082 Acesso em: 01 jun. 2021.
  » https://sciendo.com/article/10.1515/kbo-2016-0082
• PAZICNI, Samuel; BAUER, Christopher. Characterizing illusions of competence in introductory chemistry students. Chemistry Education Research and Practice, v. 15, n. 1, p. 24-34, 2014. Disponível em: Disponível em: https://pubs.rsc.org/en/content/articlelanding/2014/rp/c3rp00106g Acesso em: 05 maio 2021.
  » https://pubs.rsc.org/en/content/articlelanding/2014/rp/c3rp00106g
• PIAGET, Jean. Development and learning. In: RIPPLE, Richard; ROCKCASTLE, Verne (eds.). Piaget rediscovered. Cornel University, Ithaca, 1964, p. 7-20.
• RIOS, Joseph; LING, Guangming; PUGH, Robert; BECKER, Dovid; BACALL, Adam. Identifying critical 21st-century skills for workplace success: A content analysis of job advertisements. Educational Researcher, v. 49, n. 2, 80-89, 2020. Disponível em: Disponível em: https://journals.sagepub.com/doi/10.3102/0013189X19890600 Acesso em: 07 set. 2021.
  » https://journals.sagepub.com/doi/10.3102/0013189X19890600
• ROGOFF, Barbara; MATUSOV, Eugene; WHITE, Cynthia. Models of teaching and learning: Participation in a community of learners. In: OLSON, David; TORRANCE, Nancy (eds.), Handbook of education and human development: new models of learning, teaching, and schooling. Oxford: Blackwell, 1996. p. 388-414.
• SALKIND, Neil; FREY, Bruce. Statistics for people who (think they) hate statistics using R. Thousand Oaks, Calif.: Sage Publications Inc., 2019.
• SATORRA, Albert; BENTLER, Peter. Ensuring positiveness of the scaled difference chi-square test statistic. Psychometrika, v. 75, n. 2, p. 243-248, 2010. https://doi.org/10.1007/s11336-009-9135-y
  » https://doi.org/10.1007/s11336-009-9135-y
• SCHWEISFURTH, Michele. Learner-centred pedagogy: Towards a post-2015 agenda for teaching and learning. International Journal of Educational Development, v. 40, n. 2, p. 259-266, 2015. https://doi.org/10.1016/j.ijedudev.2014.10.011
  » https://doi.org/10.1016/j.ijedudev.2014.10.011
• SHARAN, Sholmo. (Ed.). Cooperative learning methods. 2. ed. Westport, CT: Praeger, 1999.
• SHARAN, Yael. Cooperative Learning for Academic and Social Gains: valued pedagogy, problematic practice. European Journal of Education, v. 45, n. 2, p. 300-313, 2010. https://doi.org/10.1111/j.1465-3435.2010.01430.x
  » https://doi.org/10.1111/j.1465-3435.2010.01430.x
• SHONFELD, Miri; GIBSON, David. Collaborative learning in a global world. Charlotte: Information Age Publishing, Inc., 2018.
• SLAVIN, Robert. Cooperative learning: theory, research, and practice. 2. ed. Boston: Allyn & Bacon, 1995.
• SOLAND, James; HAMILTON, Laura; STECHER, Brian. Measuring 21st century competencies: guidance for educators. Nova Iorque: Asia Society, 2013.
• TOPPING, K. J. Effective peer learning: from principles to practical implementation. London; New York: Routledge, Taylor & Francis Group, 2017.
• TRONCARELLI, Marcella Zampoli; FARIA, Adriano Antonio. A aprendizagem colaborativa para a interdependência positiva no processo ensino-aprendizagem em cursos universitários. Educação (UFSM), v. 39, n. 2, p. 427-444, 18 jun. 2014. https://doi.org/10.5902/198464447770
  » https://doi.org/10.5902/198464447770
• VEENMAN, Simon; VAN BENTHUM, Niek; BOOTSMA, Dolly; VAN DIEREN, Jildau; VAN DER KEMP, Nicole. Cooperative learning and teacher education. Teaching and Teacher Education, v. 18, n. 1, p. 87-103, 2002. https://doi.org/10.1016/S0742-051X(01)00052-x
  » https://doi.org/10.1016/S0742-051X(01)00052-x
• VYGOTSKY, Lev Semionovitch. Mind in society: the development of higher psychological processes. [s.l.] Harvard University Press, 1978.
• WINCEL, Melissa. Cooperative learning for primary. San Clemente, CA: Kagan, 2013.
• YAGER, Stuart; JOHNSON, David; JOHNSON, Roger. Oral discussion, group-to-individual transfer, and achievement in cooperative learning groups. Journal of Educational Psychology, v. 77, n. 1, p. 60-66, 1985. https://doi.org/10.1037/0022-0663.77.1.60
  » https://doi.org/10.1037/0022-0663.77.1.60

• How to cite this article:

  SILVA, Helena; LOPES, José; ESTRADA, Jose Antonio CecchinI; FERNÁNDEZ-RIO, Javier; LEITE, Ângela. Cooperative learning scale: validation among Portuguese and Brazilian university students. Revista Brasileira de Educação, v. 29, e290013, 2024. https://doi.org/10.1590/S1413-24782024290014

• Funding:

  The study didn’t receive funding.

Publication Dates

• Publication in this collection
  22 Mar 2024
• Date of issue
  2024

History

• Received
  23 Mar 2022
• Reviewed
  13 Feb 2023
• Accepted
  13 Feb 2023