Simulation of the New Proposed Method by CAPES for the Qualis 2017-2020 Classification of the Brazilian Archives of Biology and Technology - Part II

Abstract

This study aims to analyze the scientific production of the journal Brazilian Archives of Biology and Technology (BABT) from 2017 to 2020, in order to investigate whether the stratification assigned by the Coordination for the Improvement of Higher Education Personnel (CAPES) is consistent with the proposed method for classifying journals on Qualis 2017-2020. To this end, a mapping analysis of articles published in the years 2017, 2018, 2019 and 2020, located through the BABT journal website on the SciELO platform, available in Open Access format, was carried out. The results show that 526 articles were published in the period, adding up to 2525 authorships, with 711 authors being professors of Stricto Sensu Graduate Programs recognized by CAPES, of which, excluding redundancies, 577 linkages were obtained, in the three colleges, in eight of the nine major areas (88.88%) and in 33 of the 49 CAPES areas (67.34%). It is concluded that the journal comprises a very broad target audience and the A4 stratum in which the BABT was classified on Qualis 2017-2020 is consistent with the method proposed by the documents published by CAPES.

Keywords:
Stricto Sensu Graduate Programs; Qualis 2017-2020; Brazilian Archives of Biology and Technology

HIGHLIGHTS

• CAPES’ proposed method for the Qualis 2017-2020 classification of BABT was simulated.

• 526 articles were published in the period, adding up to 2525 authorships.

• Agrarian Sciences I is the area with the highest number of publications on BABT in the period.

• The criteria established by CAPES were apparently followed properly.

INTRODUCTION

Qualis was implemented by the Coordination for the Improvement of Higher Education Personnel (CAPES) in 1977, with the attribution of evaluating the performance of Stricto Sensu Graduate Programs (SSGP) in Brazil and qualitatively stratifying the country's scientific production [¹]. SSGP are the main promoters of scientific and technological development in the country, and CAPES, a foundation linked to the Ministry of Education, is responsible for evaluating, certifying and recognizing these SSGP [², ³].

Regarding the classification of scientific journals in Brazil, it uses Qualis as a classification factor [⁴], based on the publications of articles of authors associated with SSGP [⁵].

CAPES, in turn, classifies SSGP into 49 areas, which are grouped into nine major areas of knowledge: (i) Agrarian Sciences; (ii) Biological Sciences; (iii) Health Sciences; (iv) Exact and Earth Sciences; (v) Engineering; (vi) Multidisciplinary; (vii) Human Sciences; (viii) Applied Social Studies and; (ix) Linguistics, Letters and Arts. These areas are grouped into three fields: (i) Life Sciences; (ii) Exact, Technological and Multidisciplinary Sciences and; (iii) Humanities [⁶].

In 2007, Qualis underwent a reformulation, in which eight classification strata were created, named A1, A2, B1, B2, B3, B4, B5 and C, used until the evaluation period that comprised the years 2013 to 2016. By means of this classification, the journals with the best and worst evaluation are ranked, with A1 corresponding to the best evaluation, while C corresponds to the worst evaluation, this being a stratum that does not generate scores in the evaluation of the SSGP in Brazil [⁷].

However, in 2019, CAPES proposed a new evaluation method for Qualis 2017-2020, justifying that this new method would allow more balanced comparisons between the areas. Thus, based on Official Letter 6/2019-CGAP/DAV/CAPES and taking into account publications from 2017 and 2018, the proposal was based on four principles, as can be seen in Figure 1.

Figure 1
Principles of the new journal evaluation method.

The CiteScore Percentile and/or Impact Factor must be considered as the first stratification criterion, when the newspaper has percentage values in more than one base and in more than one category, the highest value among all will be taken into account.

This method resulted in an unofficial classification, named by CAPES as the circulating version of Qualis, baptized by the academic community as Qualis 2019 or “leaked-Qualis”, which was sectioned into eight strata calculated at equal intervals of 12.5 percentile points. Journals with percentiles above the median made up strata "A" (A1, A2, A3 and A4), and journals with percentiles below the median made up strata "B" (B1, B2, B3 and B4) [⁸, ⁹].

According to Schifini and Rodrigues [¹], scientific publications are the bridge between researchers around the world, becoming synonymous of quality, both for those who write them and for those who publish them. In this context, the journal Brazilian Archives of Biology and Technology (BABT) was classified as B3, which did not match the criteria set out in the aforementioned official letter, motivating a study comprising the journal's productions in the period 2017 and 2018, which its conclusion demonstrated that the assigned rating was inconsistent [⁹].

After numerous circular letters and ordinances, the Qualis 2017-2020 proposal was remodeled, and even though there is no official release of the new classification, changes were observed in relation to the partial version circulating in 2019, including in relation to the BABT. In this context, the objective of this study is to analyze the scientific production of the BABT from 2017 to 2020, in order to investigate whether the stratification assigned by CAPES is consistent with the proposed method for classifying the updated version of Qualis.

MATERIAL AND METHODS

The articles that make up this study were published in the years 2017, 2018, 2019 and 2020 and were located through the BABT website on the SciELO platform. The journal's website makes all published articles available in Open Access format. The data extracted in each document were the following: (i) Year of publication; (ii) Title of articles; (iii) Authors and; (iv) Association of each author's SSGP.

It should be noted that the method proposed by CAPES for Qualis classification only includes publications produced by professors of SSGP recognized by CAPES, and that the data consist of filling out the module Coleta, which is part of CAPES' Sucupira Platform, which is fed annually through the export of data from the researchers' Lattes Curriculum by SSGP coordinators.

The Lattes Curriculum of each author was accessed individually, in order to establish the association with SSGP in Brazil during the year of publication in the BABT. After this verification, the area of each SSGP and its Institution was identified, by consulting the Evaluated and Recognized Courses section of the CAPES website. Authors who were not associated with SSGP as professors, or did not have a Lattes Curriculum, were not considered for this study.

RESULTS

It can be highlighted that in the period researched in this study, BABT presented seven published editions: volumes 60, 61, 62 and 63 and a special number for each of the last three volumes listed, all used as a portfolio in this research. In Table 1, it is presented the number of publications, organized by colleges, major area and area, considering only publications with the authorship of professors of SSGP recognized by CAPES.

Table 1
Distribution of SSGP publications according to CAPES classification.

College	Great area	Area
College of Life Sciences (297)	Agrarian Sciences (158)	Food Science (48)
		Agrarian Sciences I (81)
		Veterinary Medicine (22)
		Animal Science / Fishing Resources (7)
	Biological Sciences (67)	Biodiversity (30)
		Biological Sciences I (11)
		Biological Sciences II (18)
		Biological Sciences III (8)
	Health Sciences (72)	Physical Education (5)
		Nursing (1)
		Pharmacy (30)
		Medicine I (6)
		Medicine II (10)
		Medicine III (3)
		Nutrition (5)
		Dentistry (10)
		Collective Health (2)
College of Exact, Technological and Multidisciplinary Sciences (269)	Exact and Earth Sciences (36)	Astronomy / Physics (2)
		Computer Science (8)
		Chemistry (26)
	Engineering (115)	Engineering I (41)
		Engineering II (27)
		Engineering III (14)
		Engineering IV (33)
	Multidisciplinary (118)	Biotechnology (47)
		Environmental Sciences (16)
		Teaching (7)
		Interdisciplinary (46)
		Materials (2)
College of Humanities (11)	Human Sciences (5)	Education (4)
		Geography (1)
	Applied Social Sciences (6)	Public and Business Administration, Accounting and Tourism (5)
		Urban and Regional Planning / Demographics (1)

In total, 526 articles were published in this period, adding up to 2525 authorships, with 711 authors being professors of SSGP recognized by CAPES. Excluding redundancies, there were 577 associations to SSGP, in the three existing colleges, in eight of the nine great areas (with the exception of Linguistics, Letters and Arts) and in 33 of the 49 areas of CAPES assessment. In percentual, the publications were distributed in all colleges, 88.88% of the great areas and 67.34% of the areas of CAPES, demonstrating that BABT comprises a very broad target audience. With the analysis of articles published in BABT from 2017 to 2020, it was also found that 28.15% of the authors who published in the journal are professors associated with the SSGP. It can be confirmed, from the results obtained, that the main area that determined the classification of the journal in Qualis 2017-2020 was Agricultural Sciences I.

DISCUSSION

It can be seen that Agrarian Sciences I is the main area of BABT, with 81 occurrences of publications without redundancy by professors of SSGP recognized by CAPES, differing from the study carried out previously [⁹], in which it was not possible to identify precisely which area constitutes the main area in the journal's classification, given that the results showed a tie between the areas of Agrarian Sciences I and Food Science.

According to Costa, Canto and Pinto [¹⁰], the changes in Qualis will be positive, as they will make the process more objective and simpler, especially with the adoption of the h5 index, which makes it possible to analyze the impact of journals that do not have an Impact Factor or CiteScore, like most Brazilian journals. Corroborating this, Lisboa and coauthors [¹¹] comment that the advances will be undeniable and that the new evaluation system will be aligned with metrics accepted by the international scientific community, highlighting two positive aspects: (i) less difficulty for CAPES to carry out its evaluation activities and; (ii) reduction in the number of journals that were evaluated in different areas and associated with different types of stratification.

On the other hand, there are numerous aspects that may generate uncertainties at first, such as: (i) the adoption of a single criterion for evaluation may be questionable, leading to negative results for already consolidated journals [¹²]; (ii) the universalization of evaluation and classification criteria for journals associated with the decentralization of evaluation may be inappropriate, as it is proposed by a single area. Such proposal may be valid for unidisciplinary journals, but not for journals that cover a significant number of areas [⁹] and; (iii) if the metric used to classify Brazilian and international journals is the same, considering the much lower citation rates of Brazilian journals, the tendency is that these journals will be classified in the lower strata of Qualis [¹³].

Finally, two points are worth highlighting. First, internationalization represents a factor that has been valued by CAPES during its process of reformulating the evaluation SSGP, even considering exclusively publications of professors associated to SSGP recognized by CAPES, disregarding publications by foreign authors, which constitute an absolute majority of the total number of publications in several journals, especially in those considered of excellence [⁹]. Second, the current version of Qualis influenced the choice of journals by researchers, since several journals benefited from such classification ended up publicizing their classification in their official websites in what, in theory, would be the new Qualis [¹³]. However, such classification did not materialize, and several inconsistencies were corrected.

After the publication of Qualis 2017-2020 it was confirmed that the evaluation of BABT was in fact assigned by Agrarian Sciences I area, and in the evaluation report of Agrarian Sciences I area in the quadrennial evaluation 2021 [¹⁴], it was possible to have a clear understanding of all the criteria used for classifying journals on Qualis 2017-2020, and in case of BABT, it can be considered that the A4 stratum assigned to it is aligned with the method proposed by the documents published by CAPES.

CONCLUSION

BABT had previously been classified with stratum B3, which did not match the journal's indicators, even considering the scenario in which the main areas had autonomy to rearrange the classification of 20% of journals into up to two strata. Finally, when Qualis 2017-2020 was officially released, it was concluded that BABT is classified in an appropriate stratum (A4), since its indicators, both in this study and the previous one [⁹], demonstrated that it meets all the requirements to be allocated in the upper strata of Qualis.

Acknowledgments

The authors also thank the UTFPR research laboratory “Organizações e Sociedade” (Organizations and Society).

REFERENCES

• ¹
  Schifini LRC, Rodrigues RS. [Journals evaluation policy in the medical fields: impacts on brazilian editorial production]. Perspect. Ciênc. Inf. 2020;24(4):78-111. https://doi.org/10.1590/1981-5344/3745
  » https://doi.org/10.1590/1981-5344/3745
• ²
  Picinin CT, Pilatti LA, Kovaleski JL, Graeml AR, Pedroso B. Comparison of performance of researchers recipients of CNPq productivity grants in the field of Brazilian production engineering. Scientometrics. 2016;109(2):855-870. https://doi.org/10.1007/s11192-016-2070-7
  » https://doi.org/10.1007/s11192-016-2070-7
• ³
  Gheno EM, Guaragna RM, Da Mata LFS, Duarte LF, Souza DO, Calabró L. [CAPES evaluation system: indicators and strategies for monitoring and evaluating performance]. Em Quest. 2019;25(3):184-213. https://doi.org/10.19132/1808-5245253.184-213
  » https://doi.org/10.19132/1808-5245253.184-213
• ⁴
  Castro ALS. [Periodic Qualis/CAPES system: reflections on the distortions of evaluation criteria and their multiple uses]. Multi-Sci. J. 2015;1(3):1-2. https://doi.org/10.33837/msj.v1i3.161
  » https://doi.org/10.33837/msj.v1i3.161
• ⁵
  Amoras FC. [The unique qualis and parent area]. Est. Cient. (UNIFAP). 2019;9(2):7-8. http://dx.doi.org/10.18468/estcien.2019v9n2.p07-08
  » http://dx.doi.org/10.18468/estcien.2019v9n2.p07-08
• ⁶
  Coordenação de Aperfeiçoamento de Pessoal de Nível Superior [Internet]. [About the assessment areas]; c2017 [cited 2022 Nov 20]; Available from: http://www.capes.gov.br/avaliacao/sobre-as-areas-de-avaliacao
  » http://www.capes.gov.br/avaliacao/sobre-as-areas-de-avaliacao
• ⁷
  Barata RCB. [Ten things you should know about the Qualis]. Rev. Bras. Pós-Grad. 2016;13(30):13-40. https://doi.org/10.21713/2358-2332.2016.v13.947
  » https://doi.org/10.21713/2358-2332.2016.v13.947
• ⁸
  Coordenação de Aperfeiçoamento de Pessoal de Nível Superior [Internet]. [official letter No. 6/2019-CGAP/DAV/CAPES]; c2019 [cited 2022 Nov 21]; Available from: http://www.prppg.ufpr.br/site/wp-content/uploads/2019/07/oficio_1015556.pdf
  » http://www.prppg.ufpr.br/site/wp-content/uploads/2019/07/oficio_1015556.pdf
• ⁹
  Moreira SM, Picinin CT, Amarante CC, Pilatti LA, Pedroso B. Simulation of the New Proposed Method by CAPES for the Qualis 2017-2020 Classification of the Brazilian Archives of Biology and Technology. Braz. Arch. Bio. Technol. 2020;63(e20200249). https://doi.org/10.1590/1678-4324-2020200249
  » https://doi.org/10.1590/1678-4324-2020200249
• ¹⁰
  Costa H, Canto FL do, Pinto AL. [Google scholar metrics and new the new qualis evaluation system: impact of the brazilian journals of information science]. Inf. Soc. [Internet]. 2020 Mar. 31 [cited 2022 Nov. 15];30(1). Available from: https://periodicos.ufpb.br/ojs2/index.php/ies/article/view/50676
  » https://periodicos.ufpb.br/ojs2/index.php/ies/article/view/50676
• ¹¹
  Lisboa EM, Alves FL, Barbosa TK, Souza MC de, Franco ML, Cabral SC, Alves WM, Barros GF, Silva CF da CR da. The dissemination of scientific knowledge and the new qualis periodicals of the Coordination for the Improvement of Higher Education Personnel (CAPES). Res., Soc. Dev. [Internet]. 2021 Aug. 9 [cited 2022 Nov. 10];10(10):e252101018834. Available from: https://rsdjournal.org/index.php/rsd/article/view/18834
  » https://rsdjournal.org/index.php/rsd/article/view/18834
• ¹²
  Cadernos de Saúde Pública et al. [Contributions to the debate on the assessment of scientific research production]. Cad Saúde Pública. 2019;35(10). https://doi.org/10.1590/0102-311X00173219
  » https://doi.org/10.1590/0102-311X00173219
• ¹³
  Perez OC. [The New Qualis Periodicals: possible guidelines, impacts and resistance]. Nov. Deb. 2020;6(1-2). https://doi.org/10.48006/2358-0097-6212
  » https://doi.org/10.48006/2358-0097-6212
• ¹⁴
  Coordenação de Aperfeiçoamento de Pessoal de Nível Superior [Internet]. [Evaluation report: Agricultural Sciences I - Quadrennial Assessment 2021]; c2022 [cited 2022 Dec 22]; Available from: https://www.gov.br/capes/pt-br/centrais-de-conteudo/documentos/avaliacao/19122022_RELATORIODAAVALIACAOQUADRIENALCIENCIASAGRARIASIFINAL.pdf
  » https://www.gov.br/capes/pt-br/centrais-de-conteudo/documentos/avaliacao/19122022_RELATORIODAAVALIACAOQUADRIENALCIENCIASAGRARIASIFINAL.pdf

• Funding:

  This study was partially funded by the National Council for Scientific and Technological Development (CNPq) and Araucária Foundation to Support the Scientific and Technological Development of the State of Paraná (FA)

Edited by

Editor-in-Chief:

Alexandre Rasi Aoki

Associate Editor:

Alexandre Rasi Aoki

Publication Dates

• Publication in this collection
  20 Oct 2023
• Date of issue
  2023

History

• Received
  09 Dec 2022
• Accepted
  19 June 2023