The effects of the Covid-19 pandemic on the performance of academic journals: the case of Turkey

Os efeitos da pandemia do Covid-19 no desempenho das revistas acadêmicas: o caso da Turquia

Abstract

Since the beginning of 2020, “Covid-19” has affected the whole world in an unprecedented way in modern times. It is inevitable that this pandemic, which has negatively affected many fields, will also have an impact on academic journals. The aim of this study is to determine the effect of the Covid-19 pandemic on the performance of academic journals. In our study, a “Data Envelopment Analysis” methodology with 3 inputs and 3 outputs was used to determine the relative “performance of the journals”. Within the scope of the study, 109 journals published in “Turkey” and scanned in “Web of Science” indexes were examined. Results show that eleven journals were efficient in 2019, while in 2020 this number decreased to seven. Four fields have been positively affected by the pandemic and journals publishing in these fields have increased their efficiencies. Eighteen fields were adversely affected by the pandemic and the efficiency of journal publishing in these fields decreased. Eleven fields and journals publishing in these fields maintained their efficiency both before and during the pandemic. As the Covid-19 pandemic is not over yet, our data is limited. In the coming years, more detailed and comprehensive studies can be carried out with more extensive data and a further number of journals from different countries.

Keywords
Data envelopment analysis; Multi-criteria decision making; Peer-reviewed journals; Ranking

Resumo

Desde o início de 2020, a Covid-19 afetou o mundo inteiro de uma forma sem precedentes nos tempos modernos. É inevitável que essa pandemia, que afetou negativamente muitos campos, também tenha impacto nos periódicos acadêmicos. O objetivo deste estudo é determinar o efeito da pandemia de Covid-19 no desempenho de revistas acadêmicas. Neste estudo, foi utilizada uma metodologia de “Análise Envoltória de Dados” com 3 entradas e 3 saídas para determinar o “desempenho relativo dos periódicos”. No âmbito do estudo, foram examinados 109 periódicos publicados na Turquia e indexados na Web of Science. Os resultados mostram que onze periódicos foram eficientes em 2019, enquanto em 2020 esse número diminuiu para sete. Quatro áreas foram afetadas positivamente pela pandemia e os periódicos que publicam nessas áreas aumentaram sua eficiência. Dezoito áreas foram afetadas negativamente pela pandemia e a eficiência dos periódicos que publicam nessas áreas diminuiu. Onze áreas e periódicos que publicam nessas respectivas áreas mantiveram sua eficiência antes e durante a pandemia. Como a pandemia do Covid-19 ainda não acabou, nossos dados são limitados. Nos próximos anos, estudos mais detalhados e abrangentes poderão ser realizados com dados mais extensos e maior número de periódicos de diferentes países.

Palavras-chave
Análise envoltória de dados; Tomada de decisão multicritério; Revistas revisadas por pares; Classificação

Introduction

Today, many institutions, including higher education institutions, are giving more and more importance to rankings in order to increase their quality. Universities give importance to academic journal rankings in order to better determine their academic output (Rosenthal; Weiss, 2017). This article proposes a ranking procedure for journals in the Emerging Sources Citation Index (ESCI), Social Sciences Citation Index (SSCI), Arts and Humanities Citation Index (A&HCI) and Science Citation Index (SCI) indexes published in Turkey based on Data Envelopment Analysis (DEA), using data from Scimago Journal & Country Rank. One of the advantages of our research is that it offers the opportunity to compare journals published in different disciplines with each other. Another advantage is that it is not based on subjective personal opinions, but on real data from past years.

From the beginning of 2020, the world is facing an unprecedented health crisis affecting almost all segments of society. When faced with global crises such as Covid-19, which concern all humanity, obtaining and disseminating accurate and rapid scientific information is of great importance. The fact that the publication processes of academic journals sometimes take years is an obstacle to the rapid dissemination of this new scientific knowledge (Horbach, 2020). In this study, the data from 2019 and 2020 were used and analysed. In this way, it will be revealed what effect the Covid-19 pandemic has on this ranking, and which types of journals are affected by this pandemic.

In general, there are two approaches to ranking journals: stated preference and expressed preference. The first of these, expert evaluation, is the academic community’s evaluation of journals according to their own personal views. The second is the method that takes into account the actual publication and citation values of the journals. Many of the current assessments are based on the first method, expert assessments. Since these evaluations are ultimately based on individual opinions, it is difficult to talk about 100% objectivity (Mingers; Harzing, 2007). In the literature, there is an ongoing debate regarding the superiority or suitability of such methods and new method proposals (Garfield, 1972; Pinski; Narin, 1976; DuBois; Reeb, 2000; Katerattanakul; Han; Hong, 2003; Gursoy; Sandstorm, 2016; Barrick et al., 2019). Rather than add to these discussions, this study will use data from Scimago Institutions Rankings and Science Journal as inputs and outputs in our DEA application.

DEA is a non-parametric method widely used for efficiency and/or productivity measurement (Doğan; Ersoy, 2017). The advantage of using DEA to rank business journals is that it allows comparing similar outputs (e.g., citations, impact factors, and others) against similar inputs (e.g., articles) across different disciplines (Rosenthal; Weiss, 2017). That is, the power of DEA models that rank journals using citation data is thought to come from the use of such rankings as input/output data directly from the Scimago Institutions Rankings and Science Journal database; hence, these rankings are based on the concept of efficiency in comparing the various outputs of the article bases in different journals.

Relevant journal ranking literature is reviewed before proceeding with an immediate analysis. Following this, the different citation measures used in the DEA model are described and discussed before applying the methodology. As explained in this section, there are gaps in the literature that we aim to fill. Although there are some studies in the literature on journal ranking; to best to our knowledge, a study using DEA and focusing on Turkey does not exist in the literature. More importantly, the issue of how the Covid-19 outbreak affected this ranking has not been encountered in the current literature.

There have been numerous scholarly attempts to rank journals in the field of operations management (OM) and/or other business, science, and social sciences. For the most part, these studies either used citation-based data or investigated respondents’ perceptions of journal quality using survey methods.

Vokurka (1996) conducted a research based on the number of citations to evaluate the performance of journal publishing in the field of production management. He compiled the journals cited in articles published in Decision Sciences, Journal of Operations Management, and Management Science and ranked these journals according to their citation performance.

Wing (1997) made an evaluation of the survey method in her study in which twenty-two journals publishing in the field of construction management have been dealt with. The participants were asked to evaluate only the journals that were published in the fields they were knowledgeable about. Despite the small sample, statistically significant results were obtained as a result of the research.

DuBois and Reeb (2000) evaluated thirty internationally published business journals by using both the survey method and analysing the citation numbers in their studies.

Pechlaner et al. (2004) tried to rank tourism and hospitality journals according to the readership of the journals, their scientific and practical relevance, their general reputation, and the importance of their publication in the journals for the academic career of the participants.

There are many studies in the literature using DEA. DEA is one of the most widely used methods in efficiency, productivity or performance measurement in its most general concept. However, only one article (Rosenthal; Weiss, 2017) was found that aimed to rank journals. This was the biggest reason for us to do the study by focusing on Turkey.

Apart from ranking journals, DEA has been used in many fields for ranking and efficiency measurement. In the literature, there are studies aiming to measure the efficiency of schools and universities using DEA (Bessent; Bessent, 1980; Athanassopoulos; Shale, 1997; Avkiran, 2001; Johnes, 2006; Kuah; Wong, 2011). In their study, Abbott and Doucouliagos (2003) used nonparametric techniques to estimate the technical and scale efficiency of each of the Australian universities. Various output and input measures are used. The results showed that, regardless of the output-input mix, Australian universities as a whole recorded a high level of productivity relative to each other. In addition, similar studies were conducted on university preferences (Sarrico et al., 1997).

The method was used in bank efficiency studies (Sherman; Gold, 1985; Sherman; Ladino, 1995; Golany; Storbeck, 1999; Paradi; Rouatt; Zhu, 2011; Paradi; Zhu, 2013), hospital efficiency and quality assessments (Vassiloglou; Giokas, 1990; Thanassoulis, 1999; Nayar; Ozcan, 2008) and airport efficiency and performance (Gillen; Lall, 1997; Barros; Dieke, 2007).

According to Baysal; Uygur; Toklu (2004); in international trade, ports have a role that directly affects the economy of the country. Depending on the increasing trade volume, ports should be well planned and developed in order to obtain optimum port capacities. Existing ports and their capacities should be used in the most effective way, and scientific methods and techniques should be used for this. In their study, DEA was applied to determine the efficiency of 7 State Railways of the Republic of Turkey (TCDD) ports serving in Turkey. As a result of the implementation, the efficiency value of each port and the recommended potential improvements for inactive ports are given. There are other studies in this area as well (Roll; Hayuth, 1993; Martinez-Budria et al., 1999).

Mansourirad (2013) proposed a fuzzy DEA method that enables grouping and measurement according to appropriate categories in comparing the efficiency of hotels. The method compares hotels in different star classes in their categories and does not allow the comparison of small-scale hotels with large-scale hotels or internationally branded hotels. The application of the proposed method is shown in 20 hotels with different star classifications from 1 to 5. There are other studies in this area as well (Johns; Howcroft; Drake, 1997; Hwang; Chang, 2003).

Studies on the effects of the Covid-19 pandemic have become one of the most popular topics more recently. In this context, there are many studies on different subjects, such as its effects on the environment (Eroğlu, 2020; Kroll et al., 2020; Zambrano-Monserrate; Ruano; Sanchez-Alcalde, 2020; Liu; Wang; Zheng, 2021), businesses (Donthu; Gustafsson, 2020; Gursoy; Chi, 2020; Jiang; Wen, 2020), daily life (Chakraborty; Maity, 2020; Haleem; Jayaid; Vaishya, 2020), scientists (Korbel; Stegle, 2020; Myers et al., 2020). In addition, the economic effects (Ceylan; Ozkan; Mulazimogullari, 2020; Jackson et al., 2020; Topcu; Gulal, 2020) and psychological effects of Covid-19 (Cullen; Gulati; Kelly, 2020; Durankuş; Aksu, 2020; Forte et al., 2020; Gualano et al., 2020; Kontoangelos; Economou; Papageorgiou, 2020; Orgilés et al., 2020; Ammar et al., 2021) are dealt with. However, there is no study on how this pandemic affects the performance of academic journals. Therefore, this study aims to fill this gap.

Methodological Procedures

In this study, the performance of academic journals operating in Turkey and scanned in Web of Science indexes was revealed by using the DEA method. DEA is a non-parametric and widely used method in the measurement of organizational performance, specifically the dimensions of efficiency and/or productivity.

Data Envelopment Analysis

DEA is a data-driven approach, which started with Farrell’s basic work in 1957 and was brought to the field of operations research by Charnes, Cooper and Rhodes in 1978. It can be considered as a new tool that is designed to evaluate the performance of a set of similar units called Decision Making Unit (DMU) and make it possible to transform a large number of inputs into a large number of outputs (Doğan; Tanç, 2008). DEA is originally designed to measure the relative efficiency of public sector activities and non-profit organizations, such as educational institutions and healthcare facilities. However, later on, this method has also been applied to many profit-oriented organizations. Hospitals, universities, military units, local governments, courts, businesses, etc. have been the subject of DEA applications (Basso; Funari, 2001; Cooper; Seiford; Zhu, 2004).

DEA comes to the forefront as a method that does not have the weaknesses of other methods in cases where the relationships between multiple inputs and multiple outputs among the DMUs are complex. In addition, DEA is a proven method for the efficiency and/or productivity of the public sector and service sector, since it covers the relevant units regardless of the unit of measurement (Prieto; Zofio, 2001; Cooper; Seiford; Zhu, 2004).

The Input-Oriented Classical CCR Model and Its Super Efficiency Version

A model developed by Charnes, Cooper and Rhodes became known as CCR model, which is the abbreviation of the names of these three authors. The CCR model is based on the assumption of constant returns to scale. The return to scale structure, which is fixed in CCR models, has been relaxed by Banker, Charnes and Cooper, and models in which it is possible to deal with variable returns to scale have been developed. These models are known as BCC models. Both the CCR and BCC models are the most basic models in DEA, and each has both input and output oriented versions. In this study, analyses were carried out using the input-oriented CCR model and its super-efficiency extension. Classical CCR models can be easily used in efficiency measurement. However, when there is more than one efficient unit, the order of priority among the efficient ones cannot be determined, and the models assign an efficiency score of “1” corresponding to 100% to each of the efficient DMUs. Therefore, super efficiency models are used to rank the efficient DMUs between each other (Doğan, 2015).

$$\begin{gathered} min {ϴ}_0 \\ s.t. \\ n \\ \Sigma {\lambda }_j{x}_{ij}\le {\theta }_0{x}_{i0}, i=1,\dots ,m \\ j=1 \\ n \\ \Sigma {\lambda }_j{y}_{rj}\ge y_{r}0, r=1,\dots ,s \\ j=1 \\ {\lambda }_j\ge 0, j=1,\dots ,n \end{gathered}$$ (1)

The input-oriented CCR model is Model (1) (Xu; Ouenniche, 2012). In this model, λj represents non-negative scalars where j=1,…,n, and ϴ₀ represent DMU0 whose efficiency is evaluated. In this model, which takes into account the assumption of constant returns to scale (CRS), DMU0 should take a value of 1 to be efficient. When this value is less than 1, DMU0 is not efficient.

$$\begin{array}{rl}& min{\theta }_0\\ & \mathit{\text{s.t}}\text{.}\\ & \underset{}{\overset{n}{\Sigma }}{\lambda }_j{x}_{ij}\le {\theta }_0{x}_{i0},i=1,\dots ,m\\ & \underset{}{\overset{n}{\Sigma }}{\lambda }_j{y}_{rj}\ge y_{r0},r=1,\dots ,s\end{array}$$ (2)

As explained in the paragraphs above; Model (1) and Model (2) are “input-oriented CCR” and “input-oriented CCR super-efficiency” models, respectively.

Two important points should be considered while applying the DEA. The first is the selection of comparable units, known as DMUs, with a similar input-output structure. At this point, homogeneity should be taken into account when selecting the DMUs. That is, DMUs must work for the same or similar targets under the same or similar conditions. The second is that the number of DMUs must be large enough to obtain meaningful results. DEA can be problematic when the total number of inputs and outputs in a dataset approaches the total number of DMUs. In the literature, the number of DMUs is taken as at least 2 or 3 times the sum of the number of inputs and outputs (Colbert; Levary; Shaner, 2000; Haas; Murphy 2003).

Data and Procedure

In our analysis, 3 inputs and 3 outputs were used. Inputs are; Total Docs. (2019, 2020), Total Docs. (3 years), Total Refs. (2019, 2020); and the outputs are H index, Total Cites (3 years), Cites/Doc. (2 years). The inputs and outputs in Table 1 refer to:

• Total Docs. (2019): Journal’s published articles in 2019. All types of documents are considered.

• Total Docs. (3 years): Journal’s published articles in 2018, 2017 and 2016. All types of documents are considered.

• Total Refs. (2019): Number of references included in the journal’s published articles in 2019.

• Total Cites (3 years): Citations in 2019 received by journal’s documents published in 2018, 2017, and 2016.

• Cites/Doc. (2 years): Average citation per document in a 2-year period. This metric is widely used as impact index.

• H index: Journal’s number of articles (h) that have received at least h citations over the whole period.

Table 1
Journal Statistics for 2019.

Rank	DMU	SJR Best Quartile	Total Docs. (2020) Input 1	Total Docs. (3years) Input 2	Total Refs. Input 3	H index Output 1	Total Cites (3 years) Output 2	Cites / Doc. (2 years) Output 3	Categories
1	Acta Orthopaedica et Traumatologica Turcica	Q2	96	339	2318	28	388	1,22	Surgery (Q2); Medicine (miscellaneous) (Q3); Orthopedics and Sports Medicine (Q3)
2	Acta Veterinaria Eurasia	Q4	14	21	501	2	8	0,38	Veterinary (miscellaneous) (Q4)
3	Advances in Hospitality and Tourism Research	Q3	13	11	693	3	5	0,45	Development (Q3); Tourism, Leisure and Hospitality Management (Q4)
4	Agri Dergisi	Q3	41	110	583	19	60	0,54	Anesthesiology and Pain Medicine (Q3)
5	All Azimuth	Q3	25	39	1140	4	13	0,41	Political Science and International Relations (Q3)
6	Amme Idaresi Dergisi	Q4	21	71	1430	5	5	0,02	Public Administration (Q4); Sociology and Political Science (Q4)
7	Anatolian journal of cardiology	Q3	197	768	3155	28	466	0,61	Cardiology and Cardiovascular Medicine (Q3)
8	Ankara Universitesi Veteriner Fakultesi Dergisi	Q3	60	178	1976	12	79	0,45	Veterinary (miscellaneous) (Q3); Animal Science and Zoology (Q4)
9	Archives of Rheumatology	Q4	72	221	1872	11	119	0,55	Rheumatology (Q4)
10	Balkan Medical Journal	Q3	80	351	1583	16	346	0,99	Medicine (miscellaneous) (Q3)
11	Belleten	Q3	27	89	1303	3	4	0,05	Cultural Studies (Q3); History (Q3)
12	Bilig	Q3	39	154	1524	7	13	0,07	Arts and Humanities (miscellaneous) (Q3); Social Sciences (miscellaneous) (Q3)
13	Borsa Istanbul Review	Q2	40	66	2117	21	225	3,22	Economics and Econometrics (Q2); Finance (Q2)
14	Bulletin of the Mineral Research and Exploration	Q4	41	85	2374	8	45	0,43	Geology (Q4); Geotechnical Engineering and Engineering Geology (Q4)
15	Cocuk Enfeksiyon Dergisi	Q4	54	146	765	9	6	0,03	Infectious Diseases (Q4); Pediatrics, Perinatology and Child Health (Q4)
16	Cumhuriyet Ilahiyat Dergisi	Q1	46	171	2190	2	8	0,06	Religious Studies (Q1); Philosophy (Q2); Social Sciences (miscellaneous) (Q3)
17	Diagnostic and Interventional Radiology	Q2	72	249	2117	43	447	1,74	Cardiology and Cardiovascular Medicine (Q2); Radiology, Nuclear Medicine and Imaging (Q2)
18	Dusunen Adam	Q4	43	156	1404	12	42	0,29	Neurology (clinical) (Q4); Psychiatry and Mental Health (Q4)
19	Egitim Arastirmalari: Eurasian Journal of Educational Research	Q3	62	217	3168	17	133	0,57	Education (Q3)
20	Egitim ve Bilim	Q3	80	247	5805	20	168	0,53	Education (Q3)
21	Eurasian Journal of Applied Linguistics	Q2	20	19	920	3	7	0,37	Language and Linguistics (Q2); Linguistics and Language (Q2); Education (Q4); Social Psychology (Q4)
22	Eurasian Journal of Medicine	Q3	65	165	1784	13	183	1,10	Medicine (miscellaneous) (Q3)
23	European Endodontic Journal		25	69	738	6	31	0,41	Dentistry (miscellaneous)
24	Experimental and Clinical Transplantation	Q3	209	542	4444	29	403	0,77	Transplantation (Q3)
25	Gazi Medical Journal	Q4	114	258	2400	7	32	0,08	Medicine (miscellaneous) (Q4)
26	Gazi University Journal of Science	Q3	98	221	2730	16	123	0,67	Engineering (miscellaneous) (Q3); Multidisciplinary (Q3)
27	Guncel Pediatri	Q4	19	109	628	4	4	0,02	Pediatrics, Perinatology and Child Health (Q4)
28	Hacettepe Journal of Mathematics and Statistics	Q3	150	353	3427	26	278	0,76	Analysis (Q3); Statistics and Probability (Q3); Algebra and Number Theory (Q4); Geometry and Topology (Q4)
29	Haseki Tip Bulteni	Q4	81	190	1750	4	9	0,06	Medicine (miscellaneous) (Q4)
30	Ilahiyat Studies	Q3	10	28	381	1	1	0,05	Religious Studies (Q3); Cultural Studies (Q4)
31	Insight Turkey	Q4	51	153	2054	20	35	0,25	Political Science and International Relations (Q4); Sociology and Political Science (Q4)
32	International Electronic Journal of Algebra	Q4	31	69	541	5	29	0,35	Algebra and Number Theory (Q4)
33	International Journal of Early Childhood Special Education	Q4	19	25	933	9	4	0,12	Developmental and Educational Psychology (Q4); Education (Q4)
34	International Journal of Education in Mathematics, Science and Technology	Q3	18	28	984	5	35	1,25	Education (Q3); Mathematics (miscellaneous) (Q3)
35	International Journal of Instruction	Q2	254	289	10356	19	584	2,02	Education (Q2)
36	International Journal of Renewable Energy Research	Q3	203	634	7501	30	1185	1,97	Energy Engineering and Power Technology (Q3); Renewable Energy, Sustainability and the Environment (Q3)
37	International Journal of Thermodynamics	Q2	24	78	575	32	62	0,60	Engineering (miscellaneous) (Q2); Condensed Matter Physics (Q3)
38	Isi Bilimi Ve Teknigi Dergisi/ Journal of Thermal Science and Technology	Q3	20	65	614	11	14	0,20	Engineering (miscellaneous) (Q3); Atomic and Molecular Physics, and Optics (Q4); Materials Science (miscellaneous) (Q4)
39	JCRPE Journal of Clinical Research in Pediatric Endocrinology	Q2	65	238	1879	35	387	1,84	Pediatrics, Perinatology and Child Health (Q2); Endocrinology (Q3); Endocrinology, Diabetes and Metabolism (Q3)
40	Journal of Critical and Intensive Care		17	60	389	5	4	0,06	Critical Care and Intensive Care Medicine
41	Journal of International Advanced Otology	Q2	92	272	1826	13	230	0,74	Medicine (miscellaneous) (Q2); Otorhinolaryngology (Q2)
42	Journal of Modern Turkish History	Q4	31	57	1316	1	2	0,03	History (Q4)
43	Journal of Sports Science and Medicine	Q2	97	255	3913	65	560	1,85	Orthopedics and Sports Medicine (Q2); Physical Therapy, Sports Therapy and Rehabilitation (Q2); Sports Science (Q2)
44	Journal of Tekirdag Agricultural Faculty	Q4	25	92	789	2	19	0,21	Agricultural and Biological Sciences (miscellaneous) (Q4); Pollution (Q4)
45	Journal of the Entomological Research Society	Q4	30	96	1191	12	44	0,42	Ecology, Evolution, Behavior and Systematics (Q4); Insect Science (Q4)
46	Journal of the Faculty of Engineering and Architecture of Gazi University	Q2	160	372	5355	18	269	0,77	Architecture (Q2); Engineering (miscellaneous) (Q2)
47	Journal of the Turkish German Gynecology Association	Q3	47	131	1058	16	121	0,77	Obstetrics and Gynecology (Q3)
48	Journal of Thermal Engineering	Q3	42	169	966	11	126	0,75	Building and Construction (Q3); Energy Engineering and Power Technology (Q3); Fluid Flow and Transfer Processes (Q3)
49	Kafkas Universitesi Veteriner Fakultesi Dergisi	Q3	127	437	4038	20	247	0,57	Veterinary (miscellaneous) (Q3)
50	Klimik Dergisi	Q4	86	137	2616	8	29	0,19	Infectious Diseases (Q4); Microbiology (medical) (Q4)
51	Klinik Psikiyatri Dergisi	Q4	28	87	1107	3	13	0,15	Psychiatry and Mental Health (Q4)
52	Marmara Medical Journal	Q4	27	90	687	11	13	0,11	Medicine (miscellaneous) (Q4)
53	Medical Journal of Bakirkoy	Q4	65	144	1603	4	12	0,05	Medicine (miscellaneous) (Q4)
54	Metu Journal of the Faculty of Architecture	Q3	12	72	491	11	20	0,22	Architecture (Q3)
55	Mikrobiyoloji Bulteni	Q3	49	157	1034	21	100	0,52	Medicine (miscellaneous) (Q3); Immunology and Microbiology (miscellaneous) (Q4); Infectious Diseases (Q4); Microbiology (medical) (Q4)
56	Milli Folklor	Q2	60	165	1652	7	9	0,05	Cultural Studies (Q2); Arts and Humanities (miscellaneous) (Q3)
57	Molecular Imaging and Radionuclide Therapy	Q3	26	54	402	7	60	1,11	Radiology, Nuclear Medicine and Imaging (Q3)
58	Neurological Sciences and Neurophysiology	Q4	33	140	1079	8	9	0,05	Neurology (Q4); Neurology (clinical) (Q4)
59	Nobel Medicus	Q4	22	105	630	7	7	0,12	Medicine (miscellaneous) (Q4)
60	Noropsikiyatri Arsivi	Q3	64	225	1798	14	176	0,75	Psychiatry and Mental Health (Q3); Neuroscience (miscellaneous) (Q4)
61	Organic Communications	Q4	21	69	715	15	44	0,69	Organic Chemistry (Q4)
62	Osmanli Arastirmalari - Journal of Ottoman Studies	Q2	10	73	593	4	13	0,15	Cultural Studies (Q2); History (Q2)
63	Pegem Egitim ve Ogretim Dergisi	Q4	28	72	1790	3	17	0,24	Education (Q4)
64	Records of Natural Products	Q3	72	210	2039	31	293	1,40	Drug Discovery (Q3); Organic Chemistry (Q3); Pharmacology (Q3); Plant Science (Q3)
65	Research in Educational Administration and Leadership	Q4	24	35	1050	4	12	0,31	Education (Q4)
66	Teknik Dergi/Technical Journal of Turkish Chamber of Civil Engineers	Q4	30	65	959	7	10	0,08	Building and Construction (Q4); Civil and Structural Engineering (Q4)
67	Tekstil ve Konfeksiyon	Q3	10	154	216	15	57	0,25	Industrial and Manufacturing Engineering (Q3); Materials Science (miscellaneous) (Q3)
68	Tuberkuloz ve Toraks	Q3	44	170	1138	18	45	0,25	Critical Care and Intensive Care Medicine (Q3); Pulmonary and Respiratory Medicine (Q4); Surgery (Q4)
69	Turk Dermatoloji Dergisi	Q4	37	132	743	5	16	0,12	Dermatology (Q4)
70	Turk Geriatri Dergisi	Q4	66	165	1384	16	45	0,27	Geriatrics and Gerontology (Q4)
71	Turk Kardiyoloji Dernegi Arsivi	Q4	145	561	1989	24	179	0,31	Cardiology and Cardiovascular Medicine (Q4)
72	Turk Noroloji Dergisi	Q4	62	205	1042	4	19	0,10	Neurology (clinical) (Q4)
73	Turk Onkoloji Dergisi	Q4	62	97	1713	7	9	0,08	Oncology (Q4)
74	Turk Osteoporoz Dergisi	Q4	20	101	424	5	8	0,02	Endocrinology, Diabetes and Metabolism (Q4)
75	Turk Patoloji Dergisi	Q3	42	120	902	14	73	0,67	Pathology and Forensic Medicine (Q3)
76	Turk Psikiyatri Dergisi	Q3	48	178	1757	28	79	0,40	Medicine (miscellaneous) (Q3); Psychiatry and Mental Health (Q4)
77	Turk Tarim ve Ormancilik Dergisi/Turkish Journal of Agriculture and Forestry	Q2	58	192	2413	43	324	1,67	Ecology (Q2); Food Science (Q2); Forestry (Q2)
78	Turkish Journal of Anaesthesiology and Reanimation	Q3	86	252	2330	12	181	0,84	Anesthesiology and Pain Medicine (Q3); Emergency Medicine (Q3)
79	Turkish Journal of Biochemistry	Q4	112	241	3325	15	75	0,36	Biochemistry (Q4); Biochemistry (medical) (Q4); Clinical Biochemistry (Q4); Molecular Biology (Q4)
80	Turkish Journal of Biology	Q3	33	279	1506	38	231	0,79	Agricultural and Biological Sciences (miscellaneous) (Q3); Cell Biology (Q4); Genetics (Q4); Microbiology (Q4); Molecular Biology (Q4); Physiology (Q4)
81	Turkish Journal of Botany	Q2	69	195	3161	37	196	1,00	Plant Science (Q2)
82	Turkish Journal of Chemistry	Q3	118	300	4668	46	337	1,01	Chemistry (miscellaneous) (Q3)
83	Turkish Journal of Earth Sciences	Q2	47	77	3564	43	114	1,34	Earth and Planetary Sciences (miscellaneous) (Q2)
84	Turkish Journal of Electrical Engineering and Computer Sciences	Q3	333	1028	9739	30	1112	1,06	Computer Science (miscellaneous) (Q3); Electrical and Electronic Engineering (Q3)
85	Turkish Journal of Emergency Medicine	Q3	37	138	828	11	164	1,28	Critical Care and Intensive Care Medicine (Q3); Emergency Medicine (Q3)
86	Turkish Journal of Endocrinology and Metabolism		43	113	1072	5	10	0,11	Endocrinology, Diabetes and Metabolism; Internal Medicine
87	Turkish Journal of Field Crops	Q2	34	80	1230	19	91	1,17	Agronomy and Crop Science (Q2)
88	Turkish Journal of Fisheries and Aquatic Sciences	Q3	93	403	3828	29	417	1,00	Animal Science and Zoology (Q3); Aquatic Science (Q3)
89	Turkish Journal of Gastroenterology	Q3	209	425	3914	28	290	0,59	Gastroenterology (Q3)
90	Turkish Journal of Haematology	Q3	79	269	795	16	184	0,72	Hematology (Q3)
91	Turkish Journal of Immunology	Q4	25	34	1145	4	8	0,27	Immunology (Q4); Immunology and Allergy (Q4)
92	Turkish Journal of Mathematics	Q3	223	491	4110	27	443	0,83	Mathematics (miscellaneous) (Q3)
93	Turkish Journal of Medical Sciences	Q3	254	767	7177	27	579	0,71	Medicine (miscellaneous) (Q3)
94	Turkish Journal of Orthodontics	Q4	39	23	1023	4	15	0,65	Orthodontics (Q4)
95	Turkish Journal of Pediatrics	Q3	165	402	3273	36	173	0,37	Pediatrics, Perinatology and Child Health (Q3)
96	Turkish Journal of Pharmaceutical Sciences	Q3	67	139	2121	14	104	0,64	Pharmaceutical Science (Q3); Molecular Medicine (Q4)
97	Turkish Journal of Physical Medicine and Rehabilitation	Q4	57	54	1712	5	14	0,26	Physical Therapy, Sports Therapy and Rehabilitation (Q4); Rehabilitation (Q4)
98	Turkish Journal of Physics	Q4	61	163	1791	26	105	0,55	Physics and Astronomy (miscellaneous) (Q4)
99	Turkish Journal of Physiotherapy and Rehabilitation	Q4	23	21	657	1	2	0,10	Orthopedics and Sports Medicine (Q4); Physical Therapy, Sports Therapy and Rehabilitation (Q4); Rehabilitation (Q4)
100	Turkish Journal of Thoracic and Cardiovascular Surgery	Q4	87	420	1547	10	44	0,12	Cardiology and Cardiovascular Medicine (Q4); Pulmonary and Respiratory Medicine (Q4); Surgery (Q4)
101	Turkish Journal of Urology	Q3	129	160	2596	8	108	0,68	Urology (Q3)
102	Turkish Journal of Veterinary and Animal Sciences	Q3	109	321	3471	35	216	0,64	Veterinary (miscellaneous) (Q3)
103	Turkish Journal of Zoology	Q3	73	336	2497	28	263	0,70	Animal Science and Zoology (Q3)
104	Turkish Neurosurgery	Q3	149	463	3869	27	431	0,87	Neurology (clinical) (Q3); Surgery (Q3)
105	Turkish Online Journal of Distance Education	Q3	44	157	1988	22	177	1,10	Education (Q3)
106	Turkish World Mathematical Society Journal of Applied and Engineering Mathematics	Q4	46	50	803	5	25	0,50	Applied Mathematics (Q4); Computational Mathematics (Q4); Control and Optimization (Q4); Discrete Mathematics and Combinatorics (Q4); Mathematical Physics (Q4); Numerical Analysis (Q4); Statistics and Probability (Q4)
107	Turkiye Entomoloji Dergisi	Q3	30	84	1206	12	57	0,54	Insect Science (Q3)
108	UHOD: Uluslararasi Hematoloji-Onkoloji Dergisi	Q4	27	112	811	10	17	0,14	Hematology (Q4); Oncology (Q4)
109	Ulusal Travma ve Acil Cerrahi Dergisi	Q3	104	304	2020	22	201	0,62	Anesthesiology and Pain Medicine (Q3); Emergency Medicine (Q3); Surgery (Q3)

The inputs and outputs in Table 2 refer to:

• Total Docs. (2020): Journal’s published articles in 2020. All types of documents are considered.

• Total Docs. (3 years): Journal’s published articles in 2019, 2018 and 2017. All types of documents are considered.

• Total Refs. (2020): Number of references included in the journal’s published articles in 2020.

• Total Cites (3 years): Citations in 2020 received by journal’s documents published in 2019, 2018, 2017.

• Cites/Doc. (2 years): Average citation per document in a 2-year period. This metric is widely used as impact index.

• H index: Journal’s number of articles (h) that have received at least h citations over the whole period.

Table 2
Journal Statistics for 2020.

Rank	DMU	SJR Best Quartile	Total Docs. (2020) Input 1	Total Docs. (3years) Input 2	Total Refs. Input 3	H index Output 1	Total Cites (3 years) Output 2	Cites/Doc. (2 years) Output 3	Categories
1	Acta Orthopaedica et Traumatologica Turcica	Q2	107	288	2308	28	826	2,81	Surgery (Q2); Medicine (miscellaneous) (Q3); Orthopedics and Sports Medicine (Q3)
2	Acta Veterinaria Eurasia	Q4	20	35	667	2	22	0,63	Veterinary (miscellaneous) (Q4)
3	Advances in Hospitality and Tourism Research	Q3	1	24	52	3	38	1,58	Development (Q3); Tourism, Leisure and Hospitality Management (Q4)
4	Agri Dergisi	Q3	45	113	928	19	182	1,55	Anesthesiology and Pain Medicine (Q3)
5	All Azimuth	Q2	8	52	551	4	78	1,80	Political Science and International Relations (Q3)
6	Amme Idaresi Dergisi	Q4	11	72	603	5	8	0,13	Public Administration (Q4); Sociology and Political Science (Q4)
7	Anatolian journal of cardiology	Q3	190	663	2989	28	1005	1,55	Cardiology and Cardiovascular Medicine (Q3)
8	Ankara Universitesi Veteriner Fakultesi Dergisi	Q3	61	174	1992	12	192	0,98	Veterinary (miscellaneous) (Q3); Animal Science and Zoology (Q4)
9	Archives of Rheumatology	Q4	88	220	2351	11	470	2,27	Rheumatology (Q4)
10	Balkan Medical Journal	Q3	87	290	1664	16	764	2,04	Medicine (miscellaneous) (Q3)
11	Belleten	Q3	24	92	1164	3	6	0,07	Cultural Studies (Q3); History (Q3)
12	Bilig	Q3	38	140	1538	7	52	0,33	Arts and Humanities (miscellaneous) (Q3); Social Sciences (miscellaneous) (Q3)
13	Borsa Istanbul Review	Q2	60	86	3330	21	801	7,92	Economics and Econometrics (Q2); Finance (Q2)
14	Bulletin of the Mineral Research and Exploration	Q3	41	102	2171	8	120	1,34	Geology (Q4); Geotechnical Engineering and Engineering Geology (Q4)
15	Cocuk Enfeksiyon Dergisi	Q4	58	148	905	9	24	0,18	Infectious Diseases (Q4); Pediatrics, Perinatology and Child Health (Q4)
16	Cumhuriyet Ilahiyat Dergisi	Q2	43	180	2129	2	26	0,18	Religious Studies (Q1); Philosophy (Q2); Social Sciences (miscellaneous) (Q3)
17	Diagnostic and Interventional Radiology	Q2	102	224	2415	43	1126	4,82	Cardiology and Cardiovascular Medicine (Q2); Radiology, Nuclear Medicine and Imaging (Q2)
18	Dusunen Adam	Q4	45	157	1777	12	185	1,07	Neurology (clinical) (Q4); Psychiatry and Mental Health (Q4)
19	Egitim Arastirmalari: Eurasian Journal of Educational Research	Q3	67	195	3859	17	351	1,98	Education (Q3)
20	Egitim ve Bilim	Q3	49	224	3809	20	308	1,00	Education (Q3)
21	Eurasian Journal of Applied Linguistics	Q2	25	39	1474	3	52	1,33	Language and Linguistics (Q2); Linguistics and Language (Q2); Education (Q4); Social Psychology (Q4)
22	Eurasian Journal of Medicine	Q3	64	179	1829	13	441	2,29	Medicine (miscellaneous) (Q3)
23	European Endodontic Journal	Q3	47	88	1030	6	117	1,49	Dentistry (miscellaneous)
24	Experimental and Clinical Transplantation	Q3	193	596	3859	29	1033	1,49	Transplantation (Q3)
25	Gazi Medical Journal	Q4	201	306	4454	7	78	0,24	Medicine (miscellaneous) (Q4)
26	Gazi University Journal of Science	Q3	80	214	2409	16	293	1,33	Engineering (miscellaneous) (Q3); Multidisciplinary (Q3)
27	Guncel Pediatri	Q4	39	101	1020	4	14	0,12	Pediatrics, Perinatology and Child Health (Q4)
28	Hacettepe Journal of Mathematics and Statistics	Q3	174	367	3761	26	768	1,95	Analysis (Q3); Statistics and Probability (Q3); Algebra and Number Theory (Q4); Geometry and Topology (Q4)
29	Haseki Tip Bulteni	Q4	79	215	1916	4	60	0,25	Medicine (miscellaneous) (Q4)
30	Ilahiyat Studies	Q3	9	29	351	1	2	0,00	Religious Studies (Q3); Cultural Studies (Q4)
31	Insight Turkey	Q3	15	154	335	20	94	0,52	Political Science and International Relations (Q4); Sociology and Political Science (Q4)
32	International Electronic Journal of Algebra	Q4	34	82	535	5	98	1,27	Algebra and Number Theory (Q4)
33	International Journal of Early Childhood Special Education	Q4	74	36	2633	9	28	0,57	Developmental and Educational Psychology (Q4); Education (Q4)
34	International Journal of Education in Mathematics, Science and Technology	Q2	11	46	721	5	132	2,87	Education (Q3); Mathematics (miscellaneous) (Q3)
35	International Journal of Instruction	Q2	297	512	12232	19	2437	4,67	Education (Q2)
36	International Journal of Renewable Energy Research	Q3	168	659	5910	30	2443	3,60	Energy Engineering and Power Technology (Q3); Renewable Energy, Sustainability and the Environment (Q3)
37	International Journal of Thermodynamics	Q3	22	74	930	32	102	1,29	Engineering (miscellaneous) (Q2); Condensed Matter Physics (Q3)
38	Isi Bilimi Ve Teknigi Dergisi/Journal of Thermal Science and Technology	Q3	29	69	868	11	98	1,76	Engineering (miscellaneous) (Q3); Atomic and Molecular Physics, and Optics (Q4); Materials Science (miscellaneous) (Q4)
39	JCRPE: Journal of Clinical Research in Pediatric Endocrinology	Q2	72	202	2197	35	802	3,22	Pediatrics, Perinatology and Child Health (Q2); Endocrinology (Q3); Endocrinology, Diabetes and Metabolism (Q3)
40	Journal of Critical and Intensive Care	Q4	32	53	565	5	4	0,03	Critical Care and Intensive Care Medicine
41	Journal of International Advanced Otology	Q3	88	289	1853	13	679	1,91	Medicine (miscellaneous) (Q2); Otorhinolaryngology (Q2)
42	Journal of Modern Turkish History	Q4	22	67	1075	1	4	0,08	History (Q4)
43	Journal of Sports Science and Medicine	Q1	89	260	3806	65	1478	5,63	Orthopedics and Sports Medicine (Q2); Physical Therapy, Sports Therapy and Rehabilitation (Q2); Sports Science (Q2)
44	Journal of Tekirdag Agricultural Faculty	Q3	37	117	1336	2	70	0,59	Agricultural and Biological Sciences (miscellaneous) (Q4); Pollution (Q4)
45	Journal of the Entomological Research Society	Q4	18	94	692	12	97	0,92	Ecology, Evolution, Behavior and Systematics (Q4); Insect Science (Q4)
46	Journal of the Faculty of Engineering and Architecture of Gazi University	Q2	174	399	6484	18	826	2,05	Architecture (Q2); Engineering (miscellaneous) (Q2)
47	Journal of the Turkish German Gynecology Associationw	Q3	55	128	1195	16	269	1,82	Obstetrics and Gynecology (Q3)
48	Journal of Thermal Engineering	Q3	78	159	2633	11	396	2,61	Building and Construction (Q3); Energy Engineering and Power Technology (Q3); Fluid Flow and Transfer Processes (Q3)
49	Kafkas Universitesi Veteriner Fakultesi Dergisi	Q3	126	411	3787	20	652	1,51	Veterinary (miscellaneous) (Q3)
50	Klimik Dergisi	Q4	65	188	1575	8	102	0,50	Infectious Diseases (Q4); Microbiology (medical) (Q4)
51	Klinik Psikiyatri Dergisi	Q4	46	115	1608	3	52	0,51	Psychiatry and Mental Health (Q4)
52	Marmara Medical Journal	Q4	27	72	608	11	36	0,43	Medicine (miscellaneous) (Q4)
53	Medical Journal of Bakirkoy	Q4	31	178	694	4	58	0,35	Medicine (miscellaneous) (Q4)
54	Metu Journal of the Faculty of Architecture	Q3	20	63	776	11	36	0,65	Architecture (Q3)
55	Mikrobiyoloji Bulteni	Q4	32	137	851	21	164	1,05	Medicine (miscellaneous) (Q3); Immunology and Microbiology (miscellaneous) (Q4); Infectious Diseases (Q4); Microbiology (medical) (Q4)
56	Milli Folklor	Q2	80	164	2158	7	46	0,26	Cultural Studies (Q2); Arts and Humanities (miscellaneous) (Q3)
57	Molecular Imaging and Radionuclide Therapy	Q3	25	80	434	7	168	1,49	Radiology, Nuclear Medicine and Imaging (Q3)
58	Neurological Sciences and Neurophysiology	Q4	39	106	1113	8	58	0,69	Neurology (Q4); Neurology (clinical) (Q4)
59	Nobel Medicus	Q4	28	81	809	7	10	0,19	Medicine (miscellaneous) (Q4)
60	Noropsikiyatri Arsivi	Q3	66	207	1968	14	490	2,24	Psychiatry and Mental Health (Q3); Neuroscience (miscellaneous) (Q4)
61	Organic Communications	Q4	22	75	701	15	142	1,37	Organic Chemistry (Q4)
62	Osmanli Arastirmalari: Journal of Ottoman Studies	Q4	8	56	603	4	4	0,06	Cultural Studies (Q2); History (Q2)
63	Pegem Egitim ve Ogretim Dergisi	Q4	29	100	1809	3	80	0,81	Education (Q4)
64	Records of Natural Products	Q2	49	215	1263	31	775	3,61	Drug Discovery (Q3); Organic Chemistry (Q3); Pharmacology (Q3); Plant Science (Q3)
65	Research in Educational Administration and Leadership	Q3	31	50	1809	4	48	0,86	Education (Q4)
66	Teknik Dergi/Technical Journal of Turkish Chamber of Civil Engineers	Q4	8	80	292	7	64	1,00	Building and Construction (Q4); Civil and Structural Engineering (Q4)
67	Tekstil ve Konfeksiyon	Q3	53	103	1590	15	122	0,98	Industrial and Manufacturing Engineering (Q3); Materials Science (miscellaneous) (Q3)
68	Tuberkuloz ve Toraks	Q4	62	158	1159	18	129	0,78	Critical Care and Intensive Care Medicine (Q3); Pulmonary and Respiratory Medicine (Q4); Surgery (Q4)
69	Turk Dermatoloji Dergisi	Q4	23	122	462	5	38	0,32	Dermatology (Q4)
70	Turk Geriatri Dergisi	Q4	64	186	1397	16	144	0,72	Geriatrics and Gerontology (Q4)
71	Turk Kardiyoloji Dernegi Arsivi	Q4	125	527	3104	24	388	0,67	Cardiology and Cardiovascular Medicine (Q4)
72	Turk Noroloji Dergisi	Q4	65	209	1793	4	88	0,44	Neurology (clinical) (Q4)
73	Turk Onkoloji Dergisi	Q4	72	138	1942	7	46	0,42	Oncology (Q4)
74	Turk Osteoporoz Dergisi	Q4	29	70	745	5	6	0,11	Endocrinology, Diabetes and Metabolism (Q4)
75	Turk Patoloji Dergisi	Q3	40	123	835	14	252	2,25	Pathology and Forensic Medicine (Q3)
76	Turk Psikiyatri Dergisi	Q4	53	160	1799	28	182	1,2	Medicine (miscellaneous) (Q3); Psychiatry and Mental Health (Q4)
77	Turk Tarim ve Ormancilik Dergisi/Turkish Journal of Agriculture and Forestry	Q1	63	154	2807	43	676	5,17	Ecology (Q2); Food Science (Q2); Forestry (Q2)
78	Turkish Journal of Anaesthesiology and Reanimation	Q3	102	262	1689	12	263	1,01	Anesthesiology and Pain Medicine (Q3); Emergency Medicine (Q3)
79	Turkish Journal of Biochemistry	Q4	107	283	3260	15	236	0,71	Biochemistry (Q4); Biochemistry (medical) (Q4); Clinical Biochemistry (Q4); Molecular Biology (Q4)
80	Turkish Journal of Biology	Q2	54	182	2866	38	490	2,87	Agricultural and Biological Sciences (miscellaneous) (Q3); Cell Biology (Q4); Genetics (Q4); Microbiology (Q4); Molecular Biology (Q4); Physiology (Q4)
81	Turkish Journal of Botany	Q3	54	198	2920	37	521	2,81	Plant Science (Q2)
82	Turkish Journal of Chemistry	Q3	128	332	5411	46	917	2,44	Chemistry (miscellaneous) (Q3)
83	Turkish Journal of Earth Sciences	Q3	58	94	4221	43	205	2,14	Earth and Planetary Sciences (miscellaneous) (Q2)
84	Turkish Journal of Electrical Engineering and Computer Sciences	Q3	230	989	7035	30	2220	2,25	Computer Science (miscellaneous) (Q3); Electrical and Electronic Engineering (Q3)
85	Turkish Journal of Emergency Medicine	Q2	29	126	601	11	790	8,57	Critical Care and Intensive Care Medicine (Q3); Emergency Medicine (Q3)
86	Turkish Journal of Endocrinology and Metabolism	Q4	53	117	1377	5	40	0,31	Endocrinology, Diabetes and Metabolism; Internal Medicine
87	Turkish Journal of Field Crops	Q2	31	76	1242	19	166	1,72	Agronomy and Crop Science (Q2)
88	Turkish Journal of Fisheries and Aquatic Sciences	Q3	88	384	4022	29	993	2,49	Animal Science and Zoology (Q3); Aquatic Science (Q3)
89	Turkish Journal of Gastroenterology	Q3	153	510	3254	28	986	1,97	Gastroenterology (Q3)
90	Turkish Journal of Haematology	Q3	77	258	1186	16	378	1,49	Hematology (Q3)
91	Turkish Journal of Immunology	Q4	23	51	817	4	28	0,54	Immunology (Q4); Immunology and Allergy (Q4)
92	Turkish Journal of Mathematics	Q2	161	595	3439	27	1141	1,86	Mathematics (miscellaneous) (Q3)
93	Turkish Journal of Medical Sciences	Q3	268	734	8662	27	1356	1,74	Medicine (miscellaneous) (Q3)
94	Turkish Journal of Orthodontics	Q3	35	62	1072	4	98	1,58	Orthodontics (Q4)
95	Turkish Journal of Pediatrics	Q3	152	441	3696	36	478	0,97	Pediatrics, Perinatology and Child Health (Q3)
96	Turkish Journal of Pharmaceutical Sciences	Q3	97	167	3281	14	360	1,90	Pharmaceutical Science (Q3); Molecular Medicine (Q4)
97	Turkish Journal of Physical Medicine and Rehabilitation	Q3	55	111	1385	5	210	1,89	Physical Therapy, Sports Therapy and Rehabilitation (Q4); Rehabilitation (Q4)
98	Turkish Journal of Physics	Q4	26	194	1211	26	246	1,38	Physics and Astronomy (miscellaneous) (Q4)
99	Turkish Journal of Physiotherapy and Rehabilitation	Q4	36	44	1047	1	12	0,27	Orthopedics and Sports Medicine (Q4); Physical Therapy, Sports Therapy and Rehabilitation (Q4); Rehabilitation (Q4)
100	Turkish Journal of Thoracic and Cardiovascular Surgery	Q4	79	339	1432	10	132	0,42	Cardiology and Cardiovascular Medicine (Q4); Pulmonary and Respiratory Medicine (Q4); Surgery (Q4)
101	Turkish Journal of Urology	Q3	94	289	2324	8	659	2,11	Urology (Q3)
102	Turkish Journal of Veterinary and Animal Sciences	Q3	164	313	5758	35	445	1,25	Veterinary (miscellaneous) (Q3)
103	Turkish Journal of Zoology	Q3	59	288	2366	28	411	1,33	Animal Science and Zoology (Q3)
104	Turkish Neurosurgery	Q3	139	461	3592	27	929	1,88	Neurology (clinical) (Q3); Surgery (Q3)
105	Turkish Online Journal of Distance Education	Q3	45	160	2280	22	476	2,47	Education (Q3)
106	Turkish World Mathematical Society Journal of Applied and Engineering Mathematics	Q4	102	96	1832	5	110	1,15	Applied Mathematics (Q4); Computational Mathematics (Q4); Control and Optimization (Q4); Discrete Mathematics and Combinatorics (Q4); Mathematical Physics (Q4); Numerical Analysis (Q4); Statistics and Probability (Q4)
107	Turkiye Entomoloji Dergisi	Q4	19	78	820	12	94	0,96	Insect Science (Q3)
108	UHOD: Uluslararasi Hematoloji-Onkoloji Dergisi	Q4	30	99	833	10	30	0,26	Hematology (Q4); Oncology (Q4)
109	Ulusal Travma ve Acil Cerrahi Dergisi	Q3	150	304	3694	22	576	1,81	Anesthesiology and Pain Medicine (Q3); Emergency Medicine (Q3); Surgery (Q3)

Journals with zero in their data in 2019 or 2020 are not included in the analysis in order to obtain accurate results. After excluding such journals, the analysis was made in 109 journals. Efficiency measurement calculations of journals were made using the input-oriented CCR model (Model 1) and its supper efficiency version (Model 2) with the help of the Efficiency Measurement System (EMS) package program (Scheel, 2006). Since it is aimed at minimizing the inputs without any change in the outputs (while the outputs are kept constant), the input-oriented model is used. As mentioned earlier, the CCR model considers “constant returns to scale”.

Results

The EMS output of 2019 is shown in Table 3. The values in this table are obtained by applying Model (2). When Table 3 is examined; it is seen that a total of eleven journals, numbered 13, 17, 37, 39, 43, 57, 67, 80, 83, 85, and 90, are above the efficiency limit and therefore efficient. It is possible to see this in the “Efficiency Scores” column of the table. The efficiency scores of these journals are 195.14%, 108.08%, 219,63%, 101.29%, 103,78%, 179,30%, 171,09%, 117,24%, 139,75%, 102,38% and 100.32%, respectively. When we examine the categories in which these journals publish, it is seen that there is a journal in each of the following categories (The numbers in parentheses indicate which numbered journal is in that category): Agricultural and Biological Sciences (miscellaneous) (80), Cardiology and Cardiovascular Medicine (17), Cell Biology (80), Condensed Matter Physics (37), Critical Care and Intensive Care Medicine (85), Earth and Planetary Sciences (miscellaneous) (83), Economics and Econometrics (13), Emergency Medicine (85), Endocrinology (39), Endocrinology, Diabetes and Metabolism (39), Engineering (miscellaneous) (37), Finance (13), Genetics (80), Hematology (90), Industrial and Manufacturing Engineering (67), Materials Science (miscellaneous) (67), Microbiology (80), Molecular Biology (80), Nuclear Medicine and Imaging (57), Orthopedics and Sports Medicine (43), Pediatrics, Perinatology and Child Health (39), Physical Therapy, Sports Therapy and Rehabilitation (43), Physiology (80), Radiology (57), Radiology, Nuclear Medicine and Imaging (17), Sports Science (43).

Table 3
EMS Outputs for 2019.

DMU	Efficiency Score	Total Docs. (2019) {I} {V}	Total Docs. (3 years) {I} {V}	Total Refs. {I} {V}	H-Index {O} {V}	Total Cites 3 years {O} {V}	(Cites/Doc 2 years) {O} {V}	Benchmarks	{S} (Total Docs. 2019) {I}	{S} (Total Docs. 3 years) {I}	{S} (Total Refs.) {I}	{S} (H-Index) {O}	{S} (Total Cites 3 years) {O}	{S} (Cites/Doc 2 years) {O}
37	219,63%	0,19	0,40	0,41	2,20	0,00	0,00	67
13	195,14%	0,00	1,00	0,00	0,00	1,43	0,52	54
57	179,30%	0,00	0,02	0,98	0,00	0,00	1,79	15
67	171,09%	0,05	0,00	0,95	0,75	0,96	0,00	34
83	139,75%	0,00	1,00	0,00	1,31	0,00	0,09	18
80	117,24%	0,80	0,20	0,00	0,43	0,75	0,00	14
17	108,08%	0,00	0,53	0,47	0,24	0,84	0,00	45
43	103,78%	0,74	0,20	0,06	0,36	0,67	0,00	22
85	102,38%	0,09	0,07	0,85	0,00	0,78	0,25	3
39	101,29%	0,42	0,03	0,55	0,00	0,76	0,26	1
90	100,32%	0,00	0,13	0,87	0,00	1,00	0,00	8
10	99,10%	0,00	0,09	0,91	0,00	0,99	0,00	17 (0,50) 67 (0,63) 90 (0,47)	0,00	0,00	0,00	22,52	0,00	0,38
77	97,99%	0,67	0,20	0,13	0,36	0,59	0,03	13 (0,21) 17 (0,01) 37 (0,16) 43 (0,37) 80 (0,23)	0,00	0,00	0,00	0,00	0,00	0,00
36	95,69%	0,83	0,17	0,00	0,00	0,96	0,00	13 (0,90) 17 (2,20)	0,00	0,00	619,29	83,41	0,00	4,75
34	91,50%	0,00	1,00	0,00	0,00	0,00	0,92	13 (0,39)	0,94	0,00	78,58	3,15	52,34	0,00
3	84,71%	0,00	1,00	0,00	0,39	0,00	0,45	13 (0,14) 83 (0,00)	5,36	0,00	286,07	0,00	26,48	0,00
1	78,30%	0,00	0,06	0,94	0,00	0,78	0,00	17 (0,76) 67 (0,05) 90 (0,26)	0,00	0,00	0,00	9,41	0,00	0,29
64	76,22%	0,00	0,62	0,38	0,21	0,54	0,02	13 (0,10) 17 (0,56) 37 (0,10) 43 (0,03)	5,74	0,00	0,00	0,00	0,00	0,00
61	71,35%	0,41	0,03	0,56	0,26	0,00	0,45	13 (0,14) 37 (0,33) 57 (0,02) 67 (0,08)	0,00	0,00	0,00	0,00	13,77	0,00
4	71,25%	0,00	0,20	0,80	0,29	0,32	0,11	17 (0,03) 37 (0,43) 57 (0,17) 67 (0,19)	10,61	0,00	0,00	0,00	0,00	0,00
88	70,65%	0,89	0,11	0,00	0,00	0,71	0,00	17 (0,75) 80 (0,35)	0,00	0,00	585,75	16,62	0,00	0,59
33	68,66%	0,00	0,73	0,27	0,69	0,00	0,00	37 (0,05) 83 (0,17)	3,77	0,00	0,00	0,00	18,70	0,14
54	68,19%	0,95	0,05	0,00	0,53	0,00	0,15	13 (0,01) 37 (0,25) 67 (0,19)	0,00	0,00	135,16	0,00	7,87	0,00
105	67,70%	0,77	0,23	0,00	0,27	0,4	0,01	13 (0,20) 37 (0,02) 43 (0,16) 80 (0,19)	0,00	0,00	16,06	0,00	0,00	0,00
87	67,18%	0,00	0,32	0,68	0,20	0,29	0,18	13 (0,26) 17 (0,01) 37 (0,40) 57 (0,07)	0,55	0,00	0,00	0,00	0,00	0,00
7	66,35%	0,00	0,10	0,90	0,00	0,66	0,00	17 (0,52) 67 (0,52) 90 (1,12)	0,00	0,00	0,00	19,89	0,00	1,22
48	63,07%	0,00	0,08	0,92	0,00	0,55	0,08	17 (0,12) 67 (0,15) 85 (0,40)	1,86	0,00	0,00	0,61	0,00	0,00
47	61,31%	0,00	0,14	0,86	0,14	0,38	0,09	17 (0,21) 37 (0,15) 57 (0,28) 67 (0,00)	2,63	0,00	0,00	0,00	0,00	0,00
81	60,37%	0,81	0,19	0,00	0,30	0,30	0,01	13 (0,05) 37 (0,58) 43 (0,26) 80 (0,01)	0,00	0,00	430,15	0,00	0,00	0,00
35	59,28%	0,00	1,00	0,00	0,00	0,59	0,00	13 (2,60)	46,74	0,00	643,80	35,51	0,00	6,34
41	58,85%	0,00	0,06	0,94	0,00	0,59	0,00	17 (0,44) 90 (0,19)	7,62	0,00	0,00	8,80	0,00	0,16
84	58,10%	0,00	0,65	0,35	0,00	0,58	0,00	13 (0,37) 17 (2,30)	12,97	0,00	0,00	76,73	0,00	4,14
94	57,93%	0,00	1,00	0,00	0,00	0,00	0,58	13 (0,20)	14,52	0,00	165,24	0,24	30,42	0,00
22	57,40%	0,00	0,37	0,63	0,00	0,46	0,11	13 (0,14) 17 (0,33) 57 (0,06)	6,31	0,00	0,00	4,59	0,00	0,00
103	56,38%	0,89	0,11	0,00	0,00	0,56	0,00	17 (0,44) 80 (0,29)	0,00	0,00	44,63	1,81	0,00	0,29
55	55,85%	0,00	0,18	0,82	0,22	0,34	0,00	17 (0,15) 37 (0,39) 67 (0,12)	5,66	0,00	0,00	0,00	0,00	0,01
82	53,90%	0,00	0,58	0,42	0,20	0,34	0,00	13 (0,21) 37 (0,32) 43 (0,48)	0,75	0,00	0,00	0,00	0,00	0,76
104	52,71%	0,00	0,05	0,95	0,00	0,53	0,00	17 (0,95) 90 (0,02)	7,94	0,00	0,00	14,42	0,00	0,81
92	51,01%	0,00	0,05	0,95	0,00	0,51	0,00	17 (0,98) 90 (0,02)	41,30	0,00	0,00	15,58	0,00	0,89
75	50,33%	0,00	0,15	0,85	0,14	0,26	0,09	17 (0,08) 37 (0,25) 57 (0,34) 67 (0,02)	0,38	0,00	0,00	0,00	0,00	0,00
76	47,54%	0,76	0,24	0,00	0,31	0,16	0,00	37 (0,73) 43 (0,03) 80 (0,07)	0,00	0,00	187,96	0,00	0,00	0,15
109	46,69%	0,00	0,18	0,82	0,12	0,35	0,00	17 (0,42) 37 (0,01) 67 (0,24)	15,78	0,00	0,00	0,00	0,00	0,17
60	46,59%	0,00	0,06	0,94	0,00	0,42	0,04	17 (0,35) 67 (0,02) 85 (0,10)	0,39	0,00	0,00	2,62	0,00	0,00
98	46,01%	0,00	0,67	0,33	0,24	0,22	0,00	13 (0,02) 37 (0,56) 43 (0,12)	2,42	0,00	0,00	0,00	0,00	0,07
2	45,96%	0,00	0,31	0,69	0,00	0,00	0,46	13 (0,10) 57 (0,06)	0,99	0,00	0,00	0,46	17,51	0,00
21	44,38%	0,00	1,00	0,00	0,23	0,00	0,22	13 (0,11) 83 (0,02)	3,76	0,00	119,09	0,00	19,20	0,00
107	43,17%	0,81	0,19	0,00	0,22	0,20	0,01	13 (0,10) 37 (0,23) 43 (0,03) 80 (0,01)	0,00	0,00	34,01	0,00	0,00	0,00
24	42,87%	0,00	0,05	0,95	0,00	0,43	0,00	17 (0,88) 90 (0,05)	22,35	0,00	0,00	9,69	0,00	0,80
28	41,94%	0,00	0,65	0,35	0,00	0,42	0,00	13 (0,11) 17 (0,56)	17,70	0,00	0,00	0,67	0,00	0,59
38	41,25%	0,60	0,40	0,00	0,41	0,00	0,00	37 (0,34)	0,00	0,00	55,62	0,00	7,31	0,01
93	40,72%	0,00	0,65	0,35	0,00	0,41	0,00	13 (0,17) 17 (1,21)	9,53	0,00	0,00	28,59	0,00	1,95
102	40,30%	0,00	0,58	0,42	0,15	0,25	0,00	17 (0,18) 37 (0,46) 43 (0,19)	1,32	0,00	0,00	0,00	0,00	0,30
71	39,85%	0,00	0,11	0,89	0,00	0,40	0,00	17 (0,22) 67 (0,75) 90 (0,19)	18,71	0,00	0,00	0,00	0,00	0,41
78	39,09%	0,00	0,41	0,59	0,00	0,33	0,06	13 (0,05) 17 (0,38) 57 (0,03)	3,86	0,00	0,00	5,41	0,00	0,00
45	37,43%	0,79	0,21	0,00	0,22	0,15	0,01	13 (0,07) 37 (0,28) 43 (0,01) 80 (0,03)	0,00	0,00	65,41	0,00	0,00	0,00
89	37,04%	0,00	0,66	0,34	0,05	0,32	0,00	13 (0,06) 17 (0,60) 43 (0,01)	30,40	0,00	0,00	0,00	0,00	0,67
106	36,99%	0,00	0,39	0,61	0,12	0,00	0,25	13 (0,10) 37 (0,06) 57 (0,13)	8,16	0,00	0,00	0,00	8,90	0,00
19	36,48%	0,77	0,23	0,00	0,15	0,21	0,00	13 (0,05) 37 (0,02) 43 (0,17) 80 (0,11)	0,00	0,00	208,95	0,00	0,00	0,00
96	36,23%	0,00	0,50	0,50	0,12	0,24	0,01	13 (0,10) 17 (0,03) 37 (0,11) 43 (0,11)	4,88	0,00	0,00	0,00	0,00	0,00
20	35,69%	0,84	0,16	0,00	0,06	0,29	0,00	17 (0,02) 43 (0,26) 80 (0,06)	0,00	0,00	926,00	0,00	0,00	0,03
62	35,14%	0,94	0,06	0,00	0,23	0,00	0,12	13 (0,03) 37 (0,04) 67 (0,13)	0,00	0,00	94,86	0,00	3,59	0,00
32	34,81%	0,00	0,47	0,53	0,09	0,16	0,09	13 (0,00) 17 (0,02) 37 (0,07) 57 (0,23)	1,26	0,00	0,00	0,00	0,00	0,00
23	34,46%	0,31	0,09	0,60	0,08	0,12	0,14	13 (0,06) 17 (0,00) 37 (0,10) 57 (0,12) 67 (0,03)	0,00	0,00	0,00	0,00	0,00	0,00
46	32,89%	0,00	0,55	0,45	0,00	0,33	0,00	13 (0,46) 17 (0,37)	7,55	0,00	0,00	7,58	0,00	1,36
68	32,04%	0,58	0,15	0,27	0,20	0,12	0,00	17 (0,01) 37 (0,50) 67 (0,02) 80 (0,03)	0,00	0,00	0,00	0,00	0,00	0,10
95	31,81%	0,00	0,65	0,35	0,14	0,18	0,00	17 (0,27) 37 (0,74) 43 (0,01)	14,19	0,00	0,00	0,00	0,00	0,56
49	31,36%	0,82	0,18	0,00	0,00	0,31	0,00	13 (0,01) 17 (0,55)	0,00	0,00	86,27	3,75	0,00	0,41
31	30,92%	0,39	0,61	0,00	0,31	0,00	0,00	37 (0,56) 83 (0,05)	0,00	0,00	130,40	0,00	5,35	0,15
9	30,81%	0,00	0,43	0,57	0,00	0,26	0,05	13 (0,00) 17 (0,25) 57 (0,09)	1,48	0,00	0,00	0,56	0,00	0,00
52	30,53%	0,97	0,03	0,00	0,31	0,00	0,00	37 (0,34) 67 (0,01)	0,00	0,00	12,38	0,00	8,47	0,10
26	29,35%	0,00	0,56	0,44	0,09	0,19	0,01	13 (0,07) 17 (0,12) 37 (0,13) 43 (0,08)	6,33	0,00	0,00	0,00	0,00	0,00
101	29,05%	0,00	0,52	0,48	0,00	0,29	0,00	13 (0,23) 17 (0,13)	19,21	0,00	0,00	2,24	0,00	0,28
65	28,46%	0,00	0,73	0,27	0,20	0,00	0,08	13 (0,08) 37 (0,03) 83 (0,03)	1,47	0,00	0,00	0,00	10,94	0,00
5	28,14%	0,00	0,73	0,27	0,18	0,00	0,10	13 (0,12) 37 (0,03) 83 (0,02)	0,98	0,00	0,00	0,00	16,53	0,00
108	27,52%	0,96	0,04	0,00	0,28	0,00	0,00	37 (0,29) 67 (0,05)	0,00	0,00	46,47	0,00	3,91	0,05
91	27,28%	0,00	0,70	0,30	0,20	0,00	0,07	13 (0,06) 37 (0,02) 83 (0,05)	1,65	0,00	0,00	0,00	12,27	0,00
70	26,11%	0,00	0,64	0,36	0,15	0,11	0,00	17 (0,02) 37 (0,44) 43 (0,02)	3,69	0,00	0,00	0,00	0,00	0,06
8	25,83%	0,00	0,58	0,42	0,09	0,16	0,01	13 (0,04) 17 (0,09) 37 (0,15) 43 (0,04)	0,20	0,00	0,00	0,00	0,00	0,00
14	24,79%	0,58	0,42	0,00	0,15	0,10	0,00	13 (0,16) 37 (0,11) 83 (0,03)	0,00	0,00	100,31	0,00	0,00	0,17
18	24,79%	0,76	0,24	0,00	0,15	0,10	0,00	13 (0,02) 37 (0,27) 43 (0,02) 80 (0,04)	0,00	0,00	10,56	0,00	0,00	0,00
66	24,57%	0,00	0,87	0,13	0,25	0,00	0,00	37 (0,17) 83 (0,04)	1,54	0,00	0,00	0,00	4,77	0,07
59	23,50%	0,95	0,05	0,00	0,23	0,00	0,00	37 (0,19) 67 (0,06)	0,00	0,00	25,71	0,00	8,37	0,01
40	22,91%	0,00	0,07	0,93	0,23	0,00	0,00	37 (0,15) 67 (0,01)	0,16	0,00	0,00	0,00	6,06	0,03
15	20,65%	0,00	0,08	0,92	0,21	0,00	0,00	37 (0,25) 67 (0,07)	4,49	0,00	0,00	0,00	13,39	0,14
97	20,52%	0,00	0,72	0,28	0,16	0,00	0,04	13 (0,05) 37 (0,04) 83 (0,06)	5,86	0,00	0,00	0,00	6,26	0,00
74	20,30%	0,00	0,10	0,9	0,20	0,00	0,00	37 (0,12) 67 (0,07)	0,40	0,00	0,00	0,00	3,66	0,07
79	18,42%	0,00	0,61	0,39	0,09	0,10	0,00	13 (0,09) 37 (0,27) 43 (0,07)	3,93	0,00	0,00	0,00	0,00	0,22
58	18,00%	0,96	0,04	0,00	0,18	0,00	0,00	37 (0,23) 67 (0,05)	0,00	0,00	53,05	0,00	7,86	0,10
6	17,83%	0,97	0,03	0,00	0,18	0,00	0,00	37 (0,15) 67 (0,00)	0,00	0,00	165,39	0,00	4,80	0,07
44	16,58%	0,33	0,05	0,62	0,01	0,08	0,08	13 (0,04) 39 (0,00) 57 (0,03) 67 (0,04) 85 (0,04)	0,00	0,00	0,00	0,00	0,00	0,00
73	16,06%	0,00	0,85	0,15	0,16	0,00	0,00	37 (0,15) 83 (0,05)	3,92	0,00	0,00	0,00	6,20	0,08
27	15,40%	0,94	0,06	0,00	0,15	0,00	0,00	37 (0,10) 67 (0,06)	0,00	0,00	28,01	0,00	5,42	0,05
100	14,82%	0,00	0,29	0,71	0,06	0,09	0,00	17 (0,05) 37 (0,12) 67 (0,26)	3,84	0,00	0,00	0,00	0,00	0,10
69	14,61%	0,00	0,19	0,81	0,06	0,07	0,02	17 (0,01) 37 (0,12) 57 (0,02) 67 (0,04)	0,88	0,00	0,00	0,00	0,00	0,00
50	14,15%	0,00	0,81	0,19	0,11	0,03	0,00	13 (0,07) 37 (0,15) 83 (0,04)	3,97	0,00	0,00	0,00	0,00	0,17
63	13,62%	0,98	0,02	0,00	0,06	0,00	0,07	13 (0,07) 37 (0,04) 67 (0,01)	0,00	0,00	77,25	0,00	1,19	0,00
12	13,37%	0,96	0,04	0,00	0,13	0,00	0,00	37 (0,20) 67 (0,03)	0,00	0,00	79,51	0,00	1,40	0,06
99	12,71%	0,00	0,72	0,28	0,08	0,00	0,04	13 (0,03) 37 (0,00) 83 (0,01)	1,41	0,00	0,00	0,00	5,21	0,00
51	10,97%	0,94	0,06	0,00	0,06	0,02	0,03	13 (0,03) 37 (0,06) 67 (0,01) 80 (0,01)	0,00	0,00	5,38	0,00	0,00	0,00
86	10,70%	0,00	0,89	0,11	0,10	0,00	0,01	13 (0,00) 37 (0,15) 83 (0,01)	0,65	0,00	0,00	0,00	0,72	0,00
30	10,16%	0,98	0,02	0,00	0,06	0,00	0,04	13 (0,01) 37 (0,02) 67 (0,00)	0,00	0,00	1,58	0,00	3,04	0,00
56	10,09%	0,00	0,91	0,09	0,10	0,00	0,00	37 (0,20) 83 (0,01)	0,59	0,00	0,00	0,00	5,01	0,09
72	10,02%	0,00	0,21	0,79	0,03	0,06	0,01	17 (0,03) 37 (0,06) 57 (0,00) 67 (0,06)	2,20	0,00	0,00	0,00	0,00	0,00
25	8,93%	0,00	0,62	0,38	0,04	0,05	0,00	17 (0,03) 37 (0,14) 43 (0,02)	2,90	0,00	0,00	0,00	0,00	0,09
11	8,33%	0,97	0,03	0,00	0,08	0,00	0,00	37 (0,09) 67 (0,00)	0,00	0,00	54,67	0,00	1,84	0,01
53	7,01%	0,00	0,88	0,12	0,06	0,01	0,00	13 (0,02) 37 (0,11) 83 (0,00)	0,99	0,00	0,00	0,00	0,00	0,09
29	5,14%	0,00	0,90	0,10	0,04	0,01	0,00	13 (0,01) 37 (0,12) 83 (0,00)	0,98	0,00	0,00	0,00	0,00	0,03
16	4,06%	0,76	0,24	0,00	0,02	0,02	0,00	13 (0,01) 37 (0,04) 43 (0,00) 80 (0,01)	0,00	0,00	24,51	0,00	0,00	0,00
42	3,82%	0,00	0,78	0,22	0,03	0,00	0,01	13 (0,00) 37 (0,02) 83 (0,01)	0,23	0,00	0,00	0,00	0,63	0,00

The EMS output of 2020, obtained as a result of running Model (2), is shown in Table 4. It is seen from Table 4 that a total of seven journals, numbered 3, 13, 31, 37, 43, 83 and 85, are found as efficient. The efficiency scores of these journals are 591,14%, 162,36%, 103,68%, 188,29%, 108,28%, 109,47% and 210,05%, respectively. When we examine the categories in which these journals publish, it is again seen that there is a journal in each of the following categories: Condensed Matter Physics (37), Critical Care and Intensive Care Medicine (85), Development (3), Earth and Planetary Sciences (miscellaneous) (83), Economics and Econometrics (13), Emergency Medicine (85), Engineering (miscellaneous) (37), Finance (13), Orthopedics and Sports Medicine (43), Physical Therapy, Sports Therapy and Rehabilitation (43), Political Science and International Relations (31), Sociology and Political Science (31), Sports Science (43), Tourism, Leisure and Hospitality Management (3).

Table 4
EMS Outputs for 2020.

DMU	Efficiency Score	(Total Docs. 2019) {I} {V}	(Total Docs. 3 years) {I} {V}	(Total Refs.) {I} {V}	(H-Index) {O} {V}	(Total Cites 3 years) {O} {V}	(Cites / Doc 2 years) {O} {V}	Benchmarks	{S} (Total Docs. 2019) {I}	{S} (Total Docs. 3 years) {I}	{S} (Total Refs.) {I}	{S} (H-Index) {O}	{S} (Total Cites 3 years) {O}	{S} (Cites / Doc 2 years) {O}
3	591,14%	1,00	0,00	0,00	1,74	0,00	4,17	51	-	-	-	-	-	-
85	210,05%	0,00	0,59	0,41	0,00	1,76	0,34	65	-	-	-	-	-	-
37	188,29%	0,11	0,45	0,43	1,88	0,00	0,00	88	-	-	-	-	-
13	162,36%	0,00	1,00	0,00	0,38	0,86	0,39	38	-	-	-	-	-	-
83	109,47%	0,00	1,00	0,00	0,98	0,11	0,00	2	-	-	-	-	-	-
43	108,28%	0,46	0,54	0,00	0,46	0,63	0,00	50	-	-	-	-	-
31	103,68%	0,00	0,49	0,51	1,04	0,00	0,00	10
17	92,05%	0,00	0,69	0,31	0,40	0,52	0,00	37 (0,28) 43 (0,42) 85 (0,60)	32,8	0,00	0,00	0,00	0,00	3,09
77	90,62%	0,00	0,70	0,30	0,49	0,36	0,06	13 (0,33) 37 (0,71) 43 (0,20) 85 (0,06)	2,42	0,00	0,00	0,00	0,00	0,00
64	85,35%	0,00	0,37	0,63	0,53	0,32	0,00	3 (1,64) 37 (0,53) 85 (0,83)	4,37	0,00	0,00	0,00	0,00	6,81
34	84,61%	0,13	0,87	0,00	0,00	0,00	0,85	3 (1,13) 13 (0,14)	0,00	0,00	97,46	1,26	20,20	0,00
39	77,05%	0,00	0,69	0,31	0,36	0,41	0,00	37 (0,36) 43 (0,29) 85 (0,43)	9,42	0,00	0,00	0,00	0,00	2,53
80	70,13%	0,68	0,32	0,00	0,41	0,29	0,00	3 (0,54) 37 (0,57) 43 (0,28)	0,00	0,00	391,63	0,00	0,00	0,29
81	68,93%	0,66	0,34	0,00	0,39	0,30	0,00	3 (1,10) 37 (0,46) 43 (0,29)	0,00	0,00	415,75	0,00	0,00	1,17
87	67,84%	0,00	0,72	0,28	0,46	0,18	0,04	13 (0,10) 37 (0,50) 43 (0,01) 85 (0,03)	2,50	0,00	0,00	0,00	0,00	0,00
61	66,06%	0,00	0,27	0,73	0,54	0,12	0,00	3 (0,17) 37 (0,41) 85 (0,12)	1,87	0,00	0,00	0,00	0,00	0,45
55	64,87%	0,00	0,36	0,64	0,55	0,10	0,00	3 (2,06) 37 (0,45) 85 (0,05)	7,42	0,00	0,00	0,00	0,00	30,22
98	63,81%	0,49	0,51	0,00	0,42	0,22	0,00	3 (3,39) 37 (0,39) 43 (0,05)	0,00	0,00	36,10	0,00	0,00	4,78
105	61,27%	0,67	0,33	0,00	0,28	0,33	0,00	3 (0,83) 43 (0,30)	0,00	0,00	211,14	0,00	0,00	0,54
4	61,07%	0,00	0,30	0,70	0,50	0,11	0,00	3 (0,67) 37 (0,48) 85 (0,14)	12,23	0,00	0,00	0,00	0,00	1,30
35	59,61%	0,00	0,73	0,27	0,00	0,60	0,00	13 (2,00) 85 (1,06)	26,44	0,00	0,00	34,62	v0,00	20,23
36	57,64%	0,26	0,74	0,00	0,00	0,58	0,00	13 (0,23) 85 (2,86)	0,00	0,00	911,41	6,32	0,00	22,72
33	56,63%	0,00	1,00	0,00	0,47	0,00	0,09	13 (0,02) 83 (0,20)	29,20	0,00	585,05	0,00	27,35	0,00
38	54,90%	0,00	0,84	0,16	0,31	0,00	0,24	3 (0,59) 13 (0,07) 37 (0,25)	6,01	0,00	0,00	0,00	1,62	0,00
75	54,88%	0,00	0,34	0,66	0,38	0,17	0,00	3 (0,57) 37 (0,30) 85 (0,25)	7,50	0,00	0,00	0,00	0,00	1,20
1	54,22%	0,00	0,75	0,25	0,22	0,32	0,00	37 (0,38) 43 (0,11) 85 (0,79)	16,92	0,00	0,00	0,00	0,00	5,04
57	53,60%	0,00	0,39	0,61	0,34	0,20	0,00	3 (0,55) 37 (0,11) 85 (0,17)	5,48	0,00	0,00	0,00	0,00	0,99
66	51,81%	0,00	0,39	0,61	0,50	0,02	0,00	3 (1,46) 31 (0,00) 37 (0,08)	0,88	0,00	0,00	0,00	0,00	1,42
5	51,05%	0,25	0,75	0,00	0,18	0,12	0,21	3 (0,84) 13 (0,04) 37 (0,01) 85 (0,01)	0,00	0,00	74,66	0,00	0,00	0,00
82	50,85%	0,00	0,79	0,21	0,27	0,24	0,00	13 (0,16) 37 (0,29) 43 (0,52)	3,47	0,00	0,00	0,00	0,00	2,07
88	50,54%	0,62	0,38	0,00	0,18	0,33	0,00	3 (1,65) 43 (0,25) 85 (0,71)	0,00	0,00	569,54	0,00	0,00	7,61
47	49,96%	0,00	0,72	0,28	0,27	0,23	0,00	37 (0,35) 43 (0,04) 85 (0,23)	10,00	0,00	0,00	0,00	0,00	0,80
107	49,29%	0,63	0,37	0,00	0,35	0,15	0,00	3 (0,40) 37 (0,27) 43 (0,03)	0,00	0,00	2,85	0,00	0,00	0,22
52	48,91%	0,00	0,21	0,79	0,49	0,00	0,00	3 (0,01) 31 (0,09) 37 (0,29)	5,54	0,00	0,00	0,00	2,00	0,00
45	48,55%	0,00	0,32	0,68	0,41	0,08	0,00	3 (0,76) 37 (0,29) 85 (0,05)	0,24	0,00	0,00	0,00	0,00	1,07
76	47,86%	0,00	0,68	0,32	0,36	0,12	0,00	37 (0,82) 43 (0,01) 85 (0,11)	3,41	0,00	0,00	0,00	0,00	0,83
10	45,20%	0,00	0,81	0,19	0,14	0,32	0,00	37 (0,15) 43 (0,01) 85 (0,92)	7,96	0,00	0,00	0,00	0,00	6,12
103	43,71%	0,59	0,41	0,00	0,24	0,19	0,00	3 (1,97) 37 (0,38) 43 (0,13) 85 (0,13)	0,00	0,00	0,00	0,00	0,00	4,12
54	43,62%	0,00	0,84	0,16	0,34	0,00	0,10	3 (0,10) 13 (0,01) 37 (0,33)	0,89	0,00	0,00	0,00	7,99	0,00
68	42,62%	0,00	0,32	0,68	0,36	0,06	0,00	3 (1,22) 37 (0,43) 85 (0,05)	14,30	0,00	0,00	0,00	0,00	2,12
90	41,66%	0,00	0,43	0,57	0,27	0,15	0,00	3 (2,03) 37 (0,19) 85 (0,36)	15,58	0,00	0,00	0,00	0,00	5,01
22	41,31%	0,00	0,82	0,18	0,11	0,30	0,00	13 (0,00) 43 (0,15) 85 (0,27)	4,82	0,00	0,0	0,00	0,00	0,89
60	40,02%	0,00	0,72	0,28	0,15	0,25	0,00	37 (0,02) 43 (0,15) 85 (0,34)	2,92	0,00	0,00	0,00	0,00	1,53
20	39,42%	0,61	0,39	0,00	0,21	0,18	0,00	3 (1,44) 37 (0,17) 43 (0,16)	0,00	0,00	664,06	0,00	0,00	2,38
67	38,81%	0,00	0,80	0,20	0,28	0,10	0,00	13 (0,04) 37 (0,37) 43 (0,03)	6,81	0,00	0,00	0,00	0,00	0,03
41	38,41%	0,00	0,79	0,21	0,11	0,28	0,00	37 (0,02) 43 (0,07) 85 (0,73)	6,21	0,00	0,00	0,00	0,00	4,76
21	37,97%	0,29	0,71	0,00	0,00	0,00	0,38	3 (0,05) 13 (0,16)	0,00	0,00	33,03	0,46	76,02	0,00
28	36,98%	0,00	0,70	0,30	0,15	0,22	0,00	37 (0,12) 43 (0,26) 85 (0,46)	24,85	0,00	0,00	0,00	0,00	3,67
48	36,89%	0,00	0,73	0,27	0,10	0,27	0,00	13 (0,16) 43 (0,09) 85 (0,17)	6,30	0,00	0,00	0,00	0,00	0,65
104	36,12%	0,00	0,76	0,24	0,13	0,23	0,00	37 (0,26) 43 (0,14) 85 (0,88)	6,58	0,00	0,00	0,00	0,00	6,81
89	35,94%	0,00	0,79	0,21	0,13	0,23	0,00	37 (0,44) 43 (0,02) 85 (1,16)	10,15	0,00	0,00	0,00	0,00	8,65
84	35,71%	0,24	0,76	0,00	0,00	0,36	0,00	13 (0,02) 85 (2,79)	0,00	0,00	766,14	1,12	0,00	21,82
101	34,74%	0,00	0,89	0,11	0,00	0,35	0,00	13 (0,11) 85 (0,72)	5,03	0,00	0,00	2,28	0,00	4,95
94	34,43%	0,00	0,79	0,21	0,12	0,00	0,23	3 (0,49) 13 (0,10) 37 (0,01)	5,30	0,00	0,00	0,00	1,61	0,00
92	34,20%	0,00	0,37	0,63	0,17	0,17	0,00	3 (0,04) 37 (0,36) 85 (1,40)	6,61	0,00	0,00	0,00	0,00	10,63
7	34,10%	0,00	0,43	0,57	0,18	0,16	0,00	3 (2,96) 37 (0,22) 85 (1,10)	25,08	0,00	0,00	0,00	0,00	12,85
19	33,76%	0,44	0,56	0,00	0,16	0,16	0,02	13 (0,04) 37 (0,13) 43 (0,17) 85 (0,06)	0,00	0,00	340,41	0,00	0,00	0,00
96	33,58%	0,00	0,76	0,24	0,15	0,18	0,00	13 (0,14) 37 (0,00) 43 (0,17)	9,24	0,00	0,00	0,00	0,00	0,14
9	33,55%	0,00	0,81	0,19	0,08	0,26	0,00	13 (0,08) 43 (0,08) 85 (0,36)	7,04	0,00	0,00	0,00	0,00	1,90
109	33,39%	0,00	0,67	0,33	0,15	0,19	0,00	37 (0,05) 43 (0,28) 85 (0,20)	18,29	0,00	0,00	0,00	0,00	1,52
102	33,08%	0,00	0,77	0,23	0,21	0,12	0,00	13 (0,24) 37 (0,68) 43 (0,12)	13,80	0,00	0,00	0,00	0,00	2,22
70	32,87%	0,00	0,31	0,69	0,27	0,06	0,00	3 (0,88) 37 (0,39) 85 (0,09)	9,05	0,00	0,00	0,00	0,00	1,95
108	32,43%	0,00	0,21	0,79	0,32	0,00	0,00	31 (0,08) 37 (0,26)	2,75	0,00	0,00	0,00	4,40	0,12
24	31,99%	0,00	0,79	0,21	0,12	0,20	0,00	37 (0,44) 43 (0,03) 85 (1,20)	14,86	0,00	0,00	0,00	0,00	9,50
95	31,35%	0,00	0,29	0,71	0,24	0,08	0,00	3 (0,50) 37 (0,92) 85 (0,46)	13,52	0,00	0,00	0,00	0,00	4,97
32	30,99%	0,00	0,35	0,65	0,21	0,10	0,00	3 (0,22) 37 (0,10) 85 (0,10)	5,18	0,00	0,00	0,00	0,00	0,07
23	30,48%	0,00	0,80	0,20	0,13	0,03	0,15	3 (0,20) 13 (0,05) 37 (0,11) 85 (0,08)	6,66	0,00	0,00	0,00	0,00	0,00
26	29,40%	0,00	0,68	0,32	0,16	0,14	0,00	37 (0,24) 43 (0,11) 85 (0,14)	4,78	0,00	0,00	0,00	0,00	0,79
46	29,10%	0,00	0,74	0,26	0,06	0,23	0,00	13 (0,41) 43 (0,06) 85 (0,53)	5,90	0,00	0,00	0,00	0,00	6,01
62	28,95%	0,70	0,30	0,00	0,29	0,00	0,00	3 (0,41) 37 (0,09)	0,00	0,00	72,71	0,00	20,36	0,70
97	28,92%	0,00	0,77	0,23	0,09	0,17	0,03	13 (0,06) 37 (0,00) 43 (0,03) 85 (0,15)	5,32	0,00	0,00	0,00	0,00	0,00
18	28,41%	0,67	0,33	0,00	0,15	0,13	0,00	3 (0,13) 37 (0,19) 43 (0,07) 85 (0,06)	0,00	0,00	0,00	0,00	0,00	0,35
49	27,79%	0,00	0,72	0,28	0,11	0,17	0,00	37 (0,10) 43 (0,17) 85 (0,49)	3,12	0,00	0,00	0,00	0,00	3,77
93	27,48%	0,00	0,79	0,21	0,05	0,22	0,00	13 (0,40) 43 (0,09) 85 (1,13)	8,27	0,00	0,00	0,00	0,00	11,69
6	26,85%	0,71	0,29	0,00	0,27	0,00	0,00	3 (0,46) 37 (0,11)	0,00	0,00	32,62	0,00	20,89	0,74
2	25,86%	0,00	0,77	0,23	0,10	0,00	0,16	3 (0,16) 13 (0,04) 37 (0,02)	1,96	0,00	0,00	0,00	21,40	0,00
14	25,57%	0,47	0,53	0,00	0,13	0,07	0,06	3 (0,07) 13 (0,10) 37 (0,17) 85 (0,03)	0,00	0,00	49,8	0,00	0,00	0,00
65	25,51%	0,15	0,85	0,00	0,15	0,00	0,10	13 (0,10) 37 (0,03) 83 (0,03)	0,00	0,00	4,90	0,00	37,92	0,00
8	25,36%	0,00	0,68	0,32	0,14	0,11	0,00	37 (0,21) 43 (0,07) 85 (0,09)	2,29	0,00	0,00	0,00	0,00	0,43
40	25,03%	0,00	0,18	0,82	0,25	0,00	0,00	31 (0,02) 37 (0,15)	4,55	0,00	0,00	0,00	12,42	0,17
15	24,86%	0,00	0,27	0,73	0,25	0,00	0,00	31 (0,15) 37 (0,19)	8,05	0,00	0,00	0,00	9,18	0,14
78	24,36%	0,00	0,35	0,65	0,16	0,08	0,00	3 (0,46) 37 (0,24) 85 (0,28)	11,07	0,00	0,00	0,00	0,00	2,43
59	24,19%	0,00	0,19	0,81	0,24	0,00	0,00	31 (0,03) 37 (0,20)	1,92	0,00	0,00	0,00	13,27	0,08
91	24,07%	0,00	0,80	0,20	0,14	0,00	0,10	3 (0,11) 13 (0,03) 37 (0,09)	1,49	0,00	0,00	0,00	10,60	0,00
69	23,56%	0,00	0,38	0,62	0,23	0,01	0,00	3 (0,73) 31 (0,04) 37 (0,06)	2,70	0,00	0,00	0,00	0,00	0,93
71	23,33%	0,00	0,37	0,63	0,17	0,07	0,00	3 (1,95) 37 (0,45) 85 (0,34)	7,47	0,00	0,00	0,00	0,00	5,90
58	21,87%	0,00	0,25	0,75	0,19	0,03	0,00	3 (0,03) 37 (0,23) 85 (0,04)	2,16	0,00	0,00	0,00	0,00	0,01
106	20,22%	0,00	0,71	0,29	0,09	0,02	0,09	3 (0,00) 13 (0,08) 37 (0,09) 85 (0,05)	12,59	0,00	0,00	0,00	0,00	0,00
79	19,31%	0,00	0,68	0,32	0,11	0,08	0,00	37 (0,26) 43 (0,09) 85 (0,11)	4,30	0,00	0,00	0,00	0,00	1,01
74	18,98%	0,00	0,18	0,82	0,19	0,00	0,00	31 (0,02) 37 (0,15)	2,05	0,00	0,00	0,00	10,42	0,09
100	17,44%	0,00	0,45	0,55	0,13	0,04	0,00	3 (1,71) 37 (0,13) 85 (0,07)	7,25	0,00	0,00	0,00	0,00	3,03
50	16,19%	0,00	0,29	0,71	0,12	0,04	0,00	3 (0,13) 37 (0,20) 85 (0,10)	3,10	0,00	0,00	0,00	0,00	0,80
63	15,23%	0,40	0,60	0,00	0,06	0,08	0,02	13 (0,01) 37 (0,05) 43 (0,01) 85 (0,07)	0,00	0,00	123,52	0,00	0,00	0,00
12	14,71%	0,66	0,34	0,00	0,10	0,04	0,00	3 (0,08) 37 (0,18) 43 (0,01) 85 (0,02)	0,00	0,00	0,00	0,00	0,00	0,25
53	14,21%	0,00	0,47	0,53	0,11	0,04	0,00	3 (0,75) 37 (0,04) 85 (0,03)	1,78	0,00	0,00	0,00	0,00	1,17
73	13,09%	0,00	0,75	0,25	0,10	0,03	0,01	13 (0,01) 37 (0,19) 43 (0,01) 85 (0,00)	3,58	0,00	0,00	0,00	0,00	0,00
86	12,16%	0,00	0,67	0,33	0,09	0,03	0,00	37 (0,13) 43 (0,01) 85 (0,02)	2,25	0,00	0,00	0,00	0,00	0,05
56	11,11%	0,00	0,83	0,17	0,09	0,03	0,00	13 (0,00) 37 (0,18) 43 (0,02)	3,24	0,00	0,00	0,00	0,00	0,08
27	10,99%	0,00	0,19	0,81	0,11	0,00	0,00	3 (0,03) 31 (0,01) 37 (0,11)	1,55	0,00	0,00	0,00	0,00	0,08
44	9,83%	0,00	0,79	0,21	0,03	0,05	0,01	13 (0,01) 37 (0,01) 43 (0,02) 85 (0,05)	0,06	0,00	0,00	0,00	0,00	0,00
51	9,71%	0,00	0,75	0,25	0,05	0,04	0,01	13 (0,02) 37 (0,06) 43 (0,00) 85 (0,03)	0,69	0,00	0,00	0,00	0,00	0,00
72	8,91%	0,00	0,74	0,26	0,04	0,05	0,00	37 (0,08) 43 (0,01) 85 (0,08)	0,82	0,00	0,00	0,00	0,00	0,42
99	8,89%	0,00	0,73	0,27	0,04	0,00	0,05	3 (0,04) 13 (0,02) 37 (0,01)	1,46	0,00	0,00	0,00	10,00	0,00
11	8,39%	0,81	0,19	0,00	0,08	0,00	0,00	3 (0,05) 37 (0,09)	0,00	0,00	12,09	0,00	4,87	0,12
30	8,23%	0,00	0,16	0,84	0,08	0,00	0,00	31 (0,00) 37 (0,03)	0,05	0,00	0,00	0,00	1,21	0,04
29	7,16%	0,00	0,73	0,27	0,04	0,03	0,00	37 (0,10) 43 (0,00) 85 (0,06)	1,55	0,00	0,00	0,00	0,00	0,39
25	6,78%	0,00	0,81	0,19	0,04	0,02	0,00	13 (0,01) 37 (0,13) 43 (0,04)	6,7	0,00	0,00	0,00	0,00	0,22
16	4,32%	0,63	0,37	0,00	0,03	0,02	0,00	3 (0,10) 37 (0,03) 43 (0,01)	0,00	0,00	11,89	0,00	0,00	0,09
42	3,88%	0,23	0,77	0,00	0,03	0,00	0,01	3 (0,01) 13 (0,00) 37 (0,03)	0,00	0,00	2,49	0,00	2,26	0,00

Although the data are limited as it is still an ongoing pandemic, some deductions can be drawn from present data. It was observed that the efficiency of the journals in the fields of Development, Political Science and International Relations, Tourism, Leisure and Hospitality Management, Sociology and Political Science increased with the effect of the Covid-19 pandemic. Furthermore, the efficiency of the journals in the fields of Agricultural and Biological Sciences (miscellaneous), Cardiology and Cardiovascular Medicine, Cell Biology, Endocrinology, Endocrinology, Diabetes and Metabolism, Genetics, Hematology, Industrial and Manufacturing Engineering, Materials Science (miscellaneous), Microbiology, Molecular Biology, Nuclear Medicine and Imaging, Pediatrics, Perinatology and Child Health, Physiology, Physical Therapy, Sports Therapy and Rehabilitation, Radiology, Radiology, Nuclear Medicine and Imaging decreased with the same effect. Condensed Matter Physics, Critical Care and Intensive Care Medicine, Earth and Planetary Sciences (miscellaneous), Economics and Econometrics, Emergency Medicine, Engineering (miscellaneous), Finance, Orthopedics and Sports Medicine, Physical Therapy, Sports Therapy and Rehabilitation, Sports Science fields found efficient both before and during the pandemic.

Considering the year 2019, the closest journals to the efficiency frontier are Journal No. 10 with an efficiency score of 99.10%, Journal No. 77 with an efficiency score of 97.99%, and Journal No. 36 with an efficiency score of 95.69%. These journals are inefficient. However, since they are very close to the efficiency frontier, it will be easier for them to make improvements to be efficient in comparison to the journals that are far or too far away from the efficiency frontier.

If we look at the efficiency scores again, it can be seen that journal No. 42 is in last place with an efficiency score of 3.82%. Journal No. 16 with 4.06%, Journal No. 29 with 5.14%, and Journal No. 53 with 7.01% are other journals having very low efficiency scores. Like these journals, other inefficient journals (all journals with efficiency scores less than 100%) should reconsider their performance.

The results also include information on the reference groups of inefficient journals and how often efficient journals are cited as a reference for inefficient journals. It is possible to see this information in the “Benchmarks” column and reach the information about which inefficient decision units are similar to which units in the reference groups and how much improvement should be made here. However, this is not the case for efficient decision units. Because it is against the nature of DEA to write an efficient decision unit as a linear combination of other efficient units, as well as an inefficient unit (Doğan; Tanç, 2008).

Looking at the “Benchmarks” column; journal numbered 37 is sixty-seven times referenced by inactive journals for comparison while journal numbered 13 is fifty-four times, journal number 57 is fifteen times, journal number 67 is thirty-four times, journal number 83 is eighteen times, journal number 80 is fourteen times, journal number 17 is forty-five times, journal number 43 is twenty-two times, journal number 85 is three times, journal number 39 is one time, and journal number 90 is eight times. Keeping in mind that journal No. 37, Journal No. 13, Journal No. 83, Journal No. 17, Journal No. 43 and Journal No. 39 are Q2, then it is noteworthy that journal No. 67 is a reference unit for thirty-four journals. While most of the Q2 or even Q1 journals are inefficient, it can be concluded that journal No. 67, which is a Q3 journal, has succeeded in obtaining its outputs by using the inputs/resources optimally. It is possible to make similar comments for other journals.

If we look again at the “Benchmarks” column, focusing on some of the inefficient journals; It is seen that journals 13 and 17 are included in the reference group of the inefficient journal No. 36. The values in parentheses show how the inefficient unit is similar to the units in the reference set and how much it should take them as an example. Journal No. 36; is similar to journal No. 13 by 90%, and journal No. 17 by 220%, and if it makes improvements by taking these journals as an example at these rates, it will be efficient. If we look at journal numbered 88; It is seen that there are journals No. 17 and 80 in their reference group. Journal No. 88 can make improvements by taking journal No. 17 as a sample of 75% and journal No. 80 by 35%. It is possible to make similar comments for other inefficient journals like journals No. 36 and 88. As an example, the calculation of the target value for one of the entries in journal No. 36, “Total Docs. 2019” is given below:

$$194,4=(0,90*40)+(2,2*72)$$

The value of “194.4” in the above equation shows the value that Journal No. 36 should target in order to be efficient. “0.90” indicates the weight of journal No. 13, “40” indicates the input value of Journal No. 13; “2.2” indicates the weight of Journal No. 17, and “72” represents the input value of Journal No. 17. The model proposes that Journal No. 36 reduce the number of articles published annually from 203 to 194. This indicates a reduction in the number of articles by 4.42%. Target input values for other inactive journals can be found using a similar approach. In Table 5, real input values and targeted input values for all journals are presented. The same comments and calculations made for 2019 can also be made for 2020.

Table 5
Real Input Values and Target Input Values for Journals (2019-2020).

DMU	Efficiency Score	Real Values (2019)				Target Values (2019)
		(Total Docs. 2019) Input 1 {I}	(Total Docs. 3 years) Input 2 {I}	(Total Refs.) Input 3 {I}		(Total Docs. 2019) Input 1 {I}	(Total Docs. 3 years) Input 2 {I}	(Total Refs.) Input 3 {I}
1	78,30%	96	339	2318		75,76	266,88	1826,42
2	45,96%	14	21	501		5,56	9,84	235,82
3	84,71%	13	11	693		5,60	9,24	296,38
4	71,25%	41	110	583		18,80	79,45	420,14
5	28,14%	25	39	1140		6,46	11,80	342,57
6	17,83%	21	71	1430		3,60	11,70	86,25
7	66,35%	197	768	3155		131,12	510,84	2103,56
8	25,83%	60	178	1976		15,56	46,95	517,98
9	30,81%	72	221	1872		20,34	67,11	565,43
10	99,10%	80	351	1583		79,43	347,95	1568,23
11	8,33%	27	89	1303		2,16	7,02	51,75
12	13,37%	39	154	1524		5,10	20,22	121,48
13	195,14%	40	66	2117		-	-	-
14	24,79%	41	85	2374		10,45	21,45	508,89
15	20,65%	54	146	765		6,70	30,28	158,87
16	4,06%	46	171	2190		1,69	6,57	59,23
17	108,08%	72	249	2117		-	-	-
18	24,79%	43	156	1404		10,54	38,64	336,09
19	36,48%	62	217	3168		22,60	78,90	948,22
20	35,69%	80	247	5805		28,64	88,02	1150,08
21	44,38%	20	19	920		5,34	8,8	304,15
22	57,40%	65	165	1784		30,92	94,65	1019,11
23	34,46%	25	69	738		8,22	22,86	239,24
24	42,87%	209	542	4444		67,31	232,57	1902,71
25	8,93%	114	258	2400		7,46	23,49	222,27
26	29,35%	98	221	2730		22,32	65,04	790,02
27	15,40%	19	109	628		3,00	17,04	70,46
28	41,94%	150	353	3427		44,72	146,70	1418,39
29	5,14%	81	190	1750		3,28	10,02	90,17
30	10,16%	10	28	381		0,88	2,22	32,67
31	30,92%	51	153	2054		15,79	47,53	500,20
32	34,81%	31	69	541		9,10	22,86	175,05
33	68,66%	19	25	933		9,19	16,99	634,63
34	91,50%	18	28	984		15,60	25,74	825,63
35	59,28%	254	289	10356		104,00	171,60	5504,20
36	95,69%	203	634	7501		-	-	-
37	219,63%	24	78	575		-	-	-
38	41,25%	20	65	614		8,16	26,52	195,50
39	101,29%	65	238	1879		-	-	-
40	22,91%	17	60	389		3,70	13,24	88,41
41	58,85%	92	272	1826		46,69	160,67	1082,53
42	3,82%	31	57	1316		0,95	2,33	47,14
43	103,78%	97	255	3913		-	-	-
44	16,58%	25	92	789		4,26	15,94	138,50
45	37,43%	30	96	1191		11,48	37,38	393,50
46	32,89%	160	372	5355		45,04	122,49	1757,11
47	61,31%	47	131	1058		26,00	79,11	643,38
48	63,07%	42	169	966		24,94	108,18	617,64
49	31,36%	127	437	4038		40,00	137,61	1185,52
50	14,15%	86	137	2616		8,28	19,40	377,00
51	10,97%	28	87	1107		3,07	10,99	115,23
52	30,53%	27	90	687		8,26	28,06	197,66
53	7,01%	65	144	1603		3,44	9,90	105,59
54	68,19%	12	72	491		8,30	49,42	205,96
55	55,85%	49	157	1034		21,36	86,25	567,72
56	10,09%	60	165	1652		5,27	16,37	150,64
57	179,30%	26	54	402		-	-	-
58	18,00%	33	140	1079		6,02	25,64	143,05
59	23,50%	22	105	630		5,16	24,06	122,21
60	46,59%	64	225	1798		29,10	104,03	828,07
61	71,35%	21	69	715		14,84	48,38	511,45
62	35,14%	10	73	593		3,46	25,12	114,59
63	13,62%	28	72	1790		3,86	9,28	173,35
64	76,22%	72	210	2039		49,63	161,49	1572,11
65	28,46%	24	35	1050		5,33	9,93	293,53
66	24,57%	30	65	959		5,96	16,34	240,31
67	171,09%	10	154	216		-	-	-
68	32,04%	44	170	1138		13,91	52,94	358,17
69	14,61%	37	132	743		4,52	19,09	106,85
70	26,11%	66	165	1384		13,94	44,40	373,60
71	39,85%	145	561	1989		38,35	221,39	778,79
72	10,02%	62	205	1042		4,20	21,39	110,97
73	16,06%	62	97	1713		5,95	15,55	264,45
74	20,30%	20	101	424		3,58	20,14	84,12
75	50,33%	42	120	902		20,80	60,86	454,11
76	47,54%	48	178	1757		22,74	84,12	642,56
77	97,99%	58	192	2413		56,44	187,35	2351,93
78	39,09%	86	252	2330		30,14	99,54	922,37
79	18,42%	112	241	3325		16,87	44,85	619,69
80	117,24%	33	279	1506		-	-	-
81	60,37%	69	195	3161		41,47	117,63	1471,79
82	53,90%	118	300	4668		62,64	161,22	2506,81
83	139,75%	47	77	3564		-	-	-
84	58,10%	333	1028	9739		180,40	597,12	5652,39
85	102,38%	37	138	828		-	-	-
86	10,70%	43	113	1072		4,07	12,47	121,89
87	67,18%	34	80	1230		22,54	54,63	829,73
88	70,65%	93	403	3828		65,55	284,40	2114,85
89	37,04%	209	425	3914		46,57	155,91	1436,35
90	100,32%	79	269	795		-	-	-
91	27,28%	25	34	1145		5,23	9,37	316,72
92	51,01%	223	491	4110		72,14	249,40	2090,56
93	40,72%	254	767	7177		93,92	312,51	2921,46
94	57,93%	39	23	1023		8,00	13,20	423,40
95	31,81%	165	402	3273		38,17	127,5	1036,22
96	36,23%	67	139	2121		19,47	50,70	768,89
97	20,52%	57	54	1712		0v05,78	11,04	342,69
98	46,01%	61	163	1791		25,88	75,60	833,90
99	12,71%	23	21	657		1,67	2,75	99,15
100	14,82%	87	420	1547		9,08	61,85	231,01
101	29,05%	129	160	2596		18,56	47,55	762,12
102	40,30%	109	321	3471		42,43	129,15	1389,03
103	56,38%	73	336	2497		41,25	190,47	1368,22
104	52,71%	149	463	3869		69,98	241,93	2027,05
105	67,70%	44	157	1988		30,27	108,57	1347,12
106	36,99%	46	50	803		8,82	18,30	298,46
107	43,17%	30	84	1206		12,76	34,98	476,40
108	27,52%	27	112	811		7,46	30,32	177,55
109	46,69%	104	304	2020		32,88	142,32	946,73
DMU	Efficiency Score	Real Values (2020)				Target Values (2020)
		(Total Docs. 2019) Input 1 {I}	(Total Docs. 3 years) Input 2 {I}	(Total Refs.) Input 3 {I}		(Total Docs. 2019) Input 1 {I}	(Total Docs. 3 years) Input 2 {I}	(Total Refs.) Input 3 {I}
1	54,22%	107	288	2308		41,06	156,26	1246,85
2	25,86%	20	35	667		3,00	8,76	160,12
3	591,14%	1	24	52
4	61,07%	45	113	928		15,29	69,24	565,38
5	51,05%	8	52	551		3,75	25,60	192,19
6	26,85%	11	72	603		2,88	19,18	126,22
7	34,10%	190	663	2989		39,70	225,92	1019,62
8	25,36%	61	174	1992		13,46	45,08	515,81
9	33,55%	88	220	2351		22,36	73,04	787,24
10	45,20%	87	290	1664		237,02	1009,40	7369,45
11	8,39%	24	92	1164		2,03	7,86	86,30
12	14,71%	38	140	1538		5,51	20,36	221,64
13	162,36%	60	86	3330
14	25,57%	41	102	2171		10,68	26,64	512,77
15	24,86%	58	148	905		6,43	37,16	226,95
16	4,32%	43	180	2129		1,65	7,22	71,16
17	92,05%	102	224	2415		60,94	205,52	2219,52
18	28,41%	45	157	1777		12,28	42,94	485,94
19	33,76%	67	195	3859		22,13	64,82	937,18
20	39,42%	49	224	3809		19,42	88,74	841,94
21	37,97%	25	39	1474		9,65	14,96	535,40
22	41,31%	64	179	1829		21,18	73,02	733,17
23	30,48%	47	88	1030		7,94	27,32	327,28
24	31,99%	193	596	3859		47,15	191,56	1244,58
25	6,78%	201	306	4454		7,02	20,88	306,44
26	29,40%	80	214	2409		19,13	64,00	726,00
27	10,99%	39	101	1020		2,60	10,40	107,21
28	36,98%	174	367	3761		39,12	134,44	1377,62
29	7,16%	79	215	1916		3,94	14,96	129,06
30	8,23%	9	29	351		0,66	2,22	27,90
31	103,68%	15	154	335		-	-	-
32	30,99%	34	82	535		5,32	25,28	164,54
33	56,63%	74	36	2633		12,80	20,52	910,80
34	84,61%	11	46	721		9,53	39,16	524,96
35	59,61%	297	512	12232		150,74	305,56	7297,06
36	57,64%	168	659	5910		96,74	380,14	2484,76
37	188,29%	22	74	930		-	-
38	54,90%	29	69	868		10,29	38,68	496,28
39	77,05%	72	202	2197		46,20	156,22	1696,97
40	25,03%	32	53	565		3,60	14,18	146,20
41	38,41%	88	289	1853		27,84	111,66	723,75
42	3,88%	22	67	1075		0,67	2,46	28,42
43	108,28%	89	260	3806		-	-	-
44	9,83%	37	117	1336		4,05	13,10	148,77
45	48,55%	18	94	692		8,59	46,00	339,27
46	29,10%	174	399	6484		45,31	117,64	1912,19
47	49,96%	55	128	1195		17,93	65,28	615,97
48	36,89%	78	159	2633		22,54	58,58	977,51
49	27,79%	126	411	3787		31,54	113,34	1034,51
50	16,19%	65	188	1575		7,43	30,52	252,86
51	9,71%	46	115	1608		3,39	9,94	140,43
52	48,91%	27	72	608		7,74	35,56	300,37
53	14,21%	31	178	694		2,50	24,74	94,23
54	43,62%	20	63	776		7,96	27,68	345,40
55	64,87%	32	137	851		13,41	89,04	555,67
56	11,11%	80	164	2158		5,74	18,52	243,52
57	53,60%	25	80	434		7,90	42,76	233,07
58	21,87%	39	106	1113		6,25	22,78	239,50
59	24,19%	28	81	809		4,85	19,42	196,05
60	40,02%	66	207	1968		23,65	83,32	793,84
61	66,06%	22	75	701		12,67	49,54	462,26
62	28,95%	8	56	603		2,39	16,50	105,02
63	15,23%	29	100	1809		4,62	15,98	159,93
64	85,35%	49	215	1263		37,37	183,16	1077,01
65	25,51%	31	50	1809		8,40	13,64	487,53
66	51,81%	8	80	292		3,22	40,96	150,32
67	38,81%	53	103	1590		13,21	38,62	591,48
68	42,62%	62	158	1159		12,13	67,40	493,39
69	23,56%	23	122	462		2,65	28,12	107,16
70	32,87%	64	186	1397		12,07	61,32	462,55
71	23,33%	125	527	3104		21,71	122,94	724,24
72	8,91%	65	209	1793		4,97	18,60	160,54
73	13,09%	72	138	1942		5,67	17,52	248,06
74	18,98%	29	70	745		3,60	14,18	146,20
75	54,88%	40	123	835		14,42	67,38	458,89
76	47,86%	53	160	1799		22,12	77,14	866,77
77	90,62%	63	154	2807		54,96	140,48	2556,46
78	24,36%	102	262	1689		13,38	54,72	352,28
79	19,31%	107	283	3260		16,92	56,50	650,45
80	70,13%	54	182	2866		38,00	127,94	1623,86
81	68,93%	54	198	2920		37,03	135,84	1588,74
82	50,85%	128	332	5411		62,26	170,42	2781,62
83	109,47%	58	94	4221		-	-	-
84	35,71%	230	989	7035		82,11	353,26	1743,39
85	210,05%	29	126	601		-	-	-
86	12,16%	53	117	1377		4,33	14,74	170,98
87	67,84%	31	76	1242		18,76	51,98	854,09
88	50,54%	88	384	4022		44,49	194,06	1464,01
89	35,94%	153	510	3254		45,10	183,92	1182,48
90	41,66%	77	258	1186		16,65	108,14	498,62
91	24,07%	23	51	817		3,89	11,88	189,32
92	34,20%	161	595	3439		48,56	204,00	1178,28
93	27,48%	268	734	8662		64,78	200,18	2353,67
94	34,43%	35	62	1072		6,71	21,10	367,78
95	31,35%	152	441	3696		34,08	138,04	1158,06
96	33,58%	97	167	3281		23,53	56,24	1113,22
97	28,92%	55	111	1385		10,62	31,86	404,13
98	63,81%	26	194	1211		16,42	123,22	729,28
99	8,89%	36	44	1047		1,46	3,420	77,98
100	17,44%	79	339	1432		6,60	59,48	251,89
101	34,74%	94	289	2324		27,48	100,18	799,02
102	33,08%	164	313	5758		40,04	102,16	1888,32
103	43,71%	59	288	2366		25,67	125,58	1028,75
104	36,12%	139	461	3592		43,70	166,52	1303,52
105	61,27%	45	160	2280		27,53	97,92	1184,96
106	20,22%	102	96	1832		8,23	19,84	380,15
107	49,29%	19	78	820		9,01	37,38	386,08
108	32,43%	30	99	833		6,92	31,56	268,60
109	33,39%	150	304	3694		31,82	101,70	1232,38

Discussion

The findings obtained as a result of the DEA method give very realistic results in terms of guiding the journal editors and reviewers. DEA is a very useful tool in evaluating the relative efficiency of decision-making units. In this study, efficiency measurements of academic journals operating in Turkey and scanned in Web of Science indexes for the years 2019 and 2020 were made. It is expected that the results obtained from the study will lead the editors of the journals to be more selective about whether or not to evaluate the articles submitted for the review, and also to the referees to be more rigorous in their evaluations. With carefully selected indicators (input/output variables), DEA identifies the variables that cause inefficiency and makes suggestions about the improvements that inefficient units can do to be efficient.

In the study, 109 journals operating in Turkey and scanned in Web of Science indexes were evaluated within the scope of efficiency measurement, 11 of them for 2019 and 7 for 2020 were found to be efficient. Six of these 11 journals efficient in 2019 are Q2 and seven are Q3 journals. In 2020, one of the 7 journals is Q1, two of them are Q2, and the other four are Q3. Inefficient 98 (2019) and 102 (2020) journals obtained efficiency scores according to their proximity to and distance from the efficiency frontier. However, it is easier for those who are very close to the efficiency frontier to make improvements to be efficient compared to the journals that are far or too far from the efficiency limit. A number of suggestions have been made for inefficient units to be efficient. Due to the use of the input-oriented DEA model in the study, while the output variables are kept constant, it is stated at what rate reductions will be made in the input variables. It is also shown to what extent inefficient journals can reduce the “Total Documents.”, “Total Documents. (3 years)” and “Total References.” entries. It is important to what extent the resources are used optimally to be efficient. In this context, journal editors are shown how to optimize the usage levels of these three entries.

Conclusion

Today’s intensely competitive environment and changing economic conditions require efficient use of resources. The survival of the decision-making units depends on the extent to which they achieve their set goals and use their resources efficiently. Information about efficiency can be obtained by making a comparison between the inputs and outputs of the observed values and the inputs and outputs of optimal values. Nowadays, the concept of “performance” is gaining importance in all sectors. In the most general terms, performance can be defined as the degree of success achieved by a business in a certain period of time. In other words, performance is the quantitative and qualitative expression of where an individual, a group, or an enterprise can reach the intended goal with that job. The aim of this study is to evaluate the performance of journals operating in Turkey and included in the Web of Science indexes by DEA method, and then to compare the data of 2019 and 2020 to determine the effect of the Covid-19 outbreak on the performance of these journals.

In future studies on this subject, more comprehensive DEA analyses can be made by including both the following years due to the ongoing Covid-19 pandemic and previous years to make the study more consistent. In addition, the scope of the study can be further expanded by including journals operating in other countries.

Como citar este artigo/How to cite this article

• Doğan, N. Ö.; Bora, B. The effects of the Covid-19 pandemic on the performance of academic journals: the case of Turkey. Transinformação, v. 34, e220015, 2022. https://doi.org/10.1590/2318-0889202234e220015

References

• Abbott, M.; Doucouliagos, C. The efficiency of Australian universities: a data envelopment analysis. Economics of Education Review, v. 22 n. 1, p. 89-97, 2003.
• Ammar, A. et al Effects of home confinement on mental health and lifestyle behaviours during the Covid-19 outbreak: insights from the ECLB-Covid19 multicentre study. Biology of Sport, v. 38, n. 1, p. 9, 2021.
• Athanassopoulos, A. D.; Shale, E. Assessing the comparative efficiency of higher education institutions in the UK by the means of data envelopment analysis. Education Economics, v. 5 n. 2, p. 117-134, 1997.
• Avkiran, N. K. Investigating technical and scale efficiencies of Australian universities through data envelopment analysis. Socio-Economic Planning Sciences, v. 35 n. 1, p. 57-80, 2001.
• Barrick, J. A. et al Ranking accounting journals by topical area and methodology. Journal of Information Systems, v. 33, n. 2, p. 1-22, 2019.
• Barros, C. P.; Dieke, P. U. Performance evaluation of Italian airports: a data envelopment analysis. Journal of Air Transport Management, v. 13 n. 4, p. 184-191, 2007.
• Basso, A.; Funari, S. A data envelopment analysis approach to measure the mutual fund performance. European Journal of Operational Research, v. 135, n. 3, p. 477-492, 2001.
• Baysal, M. E.; Uygur, M.; Toklu, B. Veri Zarflama Analizi İle Tcdd Limanlarinda Bir Etkinlik Ölçümü Çalişmasi. Gazi Üniversitesi Mühendislik Mimarlık Fakültesi Dergisi, v. 19 n. 4, 2004.
• Bessent, A. M.; Bessent, E. W. Determining the comparative efficiency of schools through data envelopment analysis. Educational Administration Quarterly, v. 16 n. 2, p. 57-75, 1980.
• Ceylan, R. F.; Ozkan, B.; Mulazimogullari, E. Historical evidence for economic effects of Covid-19. The European Journal of Health Economics, v. 21, p. 817-823, 2020.
• Chakraborty, I.; Maity, P. Covid-19 outbreak: migration, effects on society, global environment and prevention. Science of the Total Environment, v. 728, p. 138882, 2020.
• Colbert, A.; Levary, R. R.; Shaner, M. C. Determining the relative efficiency of MBA programs using DEA. European Journal of Operational Research, v. 125, n. 3, p. 656-669, 2000.
• Cooper, W. W.; Seiford L. M.; Zhu J. Handbook on Data Envelopment Analysis Boston: Springer, 2004.
• Cullen, W.; Gulati, G.; Kelly, B. D. Mental health in the Covid-19 pandemic. QJM: An International Journal of Medicine, v. 113, n. 5, p. 311-312, 2020.
• Doğan, N. Ö. Vza süper etkinlik modelleri ile etkinlik ölçümü: Kapadokya da faaliyet gösteren balon işletmeleri üzerine bir uygulama. Atatürk Üniversitesi İktisadi ve İdari Bilimler Dergisi, v. 29, n. 1, p. 187-204, 2015.
• Doğan, N. Ö.; Ersoy, Y. Etkinlik ölçümü: Tekstil sektöründen bir işletme örneği. Hitit Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, v. 10, n. 1, p. 35-44, 2017.
• Dogan, N. Ö.; Tanç, A. Konaklama isletmelerinde veri zarflama analizi yöntemiyle faaliyet denetimi: kapadokya örnegi. Atatürk Üniversitesi Iktisadi ve Idari Bilimler Dergisi, v. 22, n. 1, p. 239-259, 2008.
• Donthu, N.; Gustafsson, A. Effects of Covid-19 on business and research. Journal of Business Research, v. 117, p. 284, 2020.
• DuBois, F. L.; Reeb, D. Ranking the international business journals. Journal of International Business Studies, v. 31, n. 4, p. 689-704, 2000.
• Durankuş, F.; Aksu, E. Effects of the Covid-19 pandemic on anxiety and depressive symptoms in pregnant women: a preliminary study. The Journal of Maternal-Fetal & Neonatal Medicine, p. 1-7, 2020.
• Eroğlu, H. Effects of Covid-19 outbreak on environment and renewable energy sector. Environment, Development and Sustainability, p. 1-9, 2020.
• Forte, G. et al The enemy which sealed the world: effects of Covid-19 diffusion on the psychological state of the Italian population. Journal of Clinical Medicine, v. 9, n. 6, p. 1802, 2020.
• Garfıeld, E. Citation analysis as a tool in journal evaluation. Science, v. 178, p. 471-479, 1972.
• Gillen, D.; Lall, A. Developing measures of airport productivity and performance: an application of data envelopment analysis. Transportation Research Part E: Logistics and Transportation Review, v. 33 n. 4, p. 261-273, 1997.
• Golany, B.; Storbeck, J. E. A data envelopment analysis of the operational efficiency of bank branches. Interfaces, v. 29 n. 3, p. 14-26, 1999.
• Gualano, M. R. et al Effects of Covid-19 lockdown on mental health and sleep disturbances in Italy. International Journal of Environmental Research and Public Health, v. 17, n. 13, p. 4779, 2020.
• Gursoy, D.; Chi, C. G. Effects of Covid-19 pandemic on hospitality industry: review of the current situations and a research agenda. Journal of Hospitality Marketing & Management, v. 29, n. 5, p. 527-529, 2020.
• Gursoy, D.; Sandstrom, J. K. An updated ranking of hospitality and tourism journals. Journal of Hospitality & Tourism Research, v. 40, n. 1, p. 3-18, 2016.
• Haas, D. A.; Murphy H. F. Compensating for non-homogeneity İn decision-making units in Data Envelopment Analysis. European Journal of Operational Research, v. 144, p. 530–544, 2003.
• Haleem, A.; Javaid, M.; Vaishya, R. Effects of Covid-19 pandemic in daily life. Current Medicine Research and Practice, v. 10, n. 2, p. 78, 2020.
• Horbach, S. P. Pandemic publishing: medical journals strongly speed up their publication process for Covid-19. Quantitative Science Studies, v. 1, n. 3, p. 1056-1067, 2020.
• Hwang, S. N.; Chang, T. Y. Using data envelopment analysis to measure hotel managerial efficiency change in Taiwan. Tourism Management, v. 24 n. 4, p. 357-369, 2003.
• Jackson, J. K. et al Global economic effects of Covid-19. Congressional Research Service, 2020.
• Jiang, Y.; Wen, J. Effects of Covid-19 on hotel marketing and management: a perspective article, International Journal of Contemporary Hospitality Management, v. 32, n. 8, p. 2563-2573, 2020.
• Johnes, J. Data Envelopment Analysis and its application to the measurement of efficiency in higher education. Economics of Education Review, v. 25 n. 3, p. 273-288, 2006.
• Johns, N.; Howcroft, B.; Drake, L. The use of data envelopment analysis to monitor hotel productivity. Progress in Tourism and Hospitality Research, v. 3 n. 2, p. 119-127, 1997.
• Katerattanakul, P.; Han, B.; Hong, S. Objective quality ranking of computing journals. Communications of the ACM, v. 46, n. 10, p. 111-114, 2003.
• Kontoangelos, K.; Economou, M.; Papageorgiou, C. Mental health effects of Covid-19 pandemia: a review of clinical and psychological traits. Psychiatry Investigation, v. 17, n. 6, p. 491, 2020.
• Korbel, J. O.; Stegle, O. Effects of the Covid-19 pandemic on life scientists. Genome Biology, v. 21, p. 113, 2020.
• Kroll, J. H. et al The complex chemical effects of Covid-19 shutdowns on air quality. Nature Chemistry, v. 12, n. 9, p. 777-779, 2020.
• Kuah, C. T.; Wong, K. Y. Efficiency assessment of universities through data envelopment analysis. Procedia Computer Science, v. 3, p. 499-506, 2011.
• Liu, F.; Wang, M.; Zheng, M. Effects of Covid-19 lockdown on global air quality and health. Science of the Total Environment, v. 755, p. 142533, 2021.
• Mansourirad, E. A categorical fuzzy DEA method to evaluate efficiency of hotels based on stars rating. Applied Mathematical Sciences, v. 7, n.73, p. 3625-3628, 2013.
• Martinez-Budria, E. et al A study of the efficiency of Spanish port authorities using data envelopment analysis. International Journal of Transport Economics, p. 237-253, 1999.
• Mingers, J.; Harzing, A. W. Ranking journals in business and management: a statistical analysis of the Harzing data set. European Journal of Information Systems, v. 16, n. 4, p. 303-316, 2007.
• Myers, K. R. et al Unequal effects of the Covid-19 pandemic on scientists. Nature Human Behaviour, v. 4, n. 9, p. 880-883, 2020.
• Nayar, P.; Ozcan, Y. A. Data envelopment analysis comparison of hospital efficiency and quality. Journal of Medical Systems, v. 32 n. 3, p. 193-199, 2008.
• Orgilés, M. et al Immediate psychological effects of the Covid-19 quarantine in youth from Italy and Spain. Frontiers in Psychology, v. 11, p. 2986, 2020.
• Paradi, J. C.; Rouatt, S.; Zhu, H. Two-stage evaluation of bank branch efficiency using data envelopment analysis. Omega, v. 39 n. 1, p. 99-109, 2011.
• Paradi, J. C.; Zhu, H. A survey on bank branch efficiency and performance research with data envelopment analysis. Omega, v. 41 n. 1, p. 61-79, 2013.
• Pechlaner, H. et al A ranking of international tourism and hospitality journals. Journal of Travel Research, v. 42, n. 4, p. 328-332, 2004.
• Pinski, G.; Narin F. Citation influence for journal aggregates of scientific publications: theory, with application to the literatüre of physics. Information Processing & Management, v. 12, n. 5, p. 297-312, 1976.
• Prıeto, A. M.; Zofio L. J. Evaluating efficientness in public provision of infrastructure and equipment: the case of spanish municipalities. Journal of Productivity Analysis, v. 15, p. 41-58, 2001.
• Roll, Y.; Hayuth, Y. E. H. U. D. A. Port performance comparison applying data envelopment analysis (DEA). Maritime Policy and Management, v. 20 n. 2, p. 153-161, 1993.
• Rosenthal, E. C.; Weiss, H. J. A data envelopment analysis approach for ranking journals. Omega, v. 70, p. 135-147, 2017.
• Sarrico, C. S. et al Data envelopment analysis and university selection. Journal of the Operational Research Society, v. 48 n. 12, p. 1163-1177, 1997.
• Scheel, H. Efficiency measurement system (Ems). Holger Scheel’s DEA Page, 2006.
• Sherman, H. D.; Gold, F. Bank branch operating efficiency: evaluation with data envelopment analysis. Journal of Banking & Finance, v. 9 n. 2, p. 297-315, 1985.
• Sherman, H. D.; Ladino, G. Managing bank productivity using data envelopment analysis (DEA). Interfaces, v. 25 n. 2, p. 60-73, 1995.
• Thanassoulis, E. Data envelopment analysis and its use in banking. Interfaces, v. 29 n. 3, p. 1-13, 1999.
• Topcu, M.; Gulal, O. S. The impact of Covid-19 on emerging stock markets. Finance Research Letters, v. 36, p. 101691, 2020.
• Vassiloglou, M.; Giokas, D. A study of the relative efficiency of bank branches: an application of data envelopment analysis. Journal of the Operational Research Society, v. 41 n. 7, p. 591-597, 1990.
• Vokurka, R. The relative importance of journals used in operations management research: a citation analysis. Journal of Operations Management, v. 14, n. 4, p. 345-55, 1996.
• Wing, C. K. The ranking of construction management journals. Construction Management & Economics, v. 15, n. 4, p. 387-398, 1997.
• Xu, B.; Ouenniche, J. A data envelopment analysis-based framework for the relative performance evaluation of competing crude oil prices’ volatility forecasting models. Energy Economics, v. 34, p. 576-583, 2012.
• Zambrano-Monserrate, M. A.; Ruano, M. A.; Sanchez-Alcalde, L. Indirect effects of Covid-19 on the environment. Science of the Total Environment, v. 728, p.138813, 2020.

Publication Dates

• Publication in this collection
  02 Dec 2022
• Date of issue
  2022

History

• Received
  25 Apr 2022
• Accepted
  11 May 2022