Inquiring into the Exemplary Spanish Theses of Mathematics Education: analysis of their scientific impact

Indagando en las tesis ejemplares españolas de educación matemática: análisis de su impacto científico

Abstract

Establishing criteria that determine the quality of any scientific publication is a fact agreed on by the entire scientific community in its different fields of knowledge. In the case of doctoral theses, these criteria may vary between universities, doctoral schools, or evaluation committees. This paper presents an evaluation criteria proposal for doctoral theses in the field of Mathematics Education. A total of 199 theses indexed in the TESEO database are analyzed, throughout the 2015-2020 period. Likewise, after applying these criteria, those theses that have been denominated as “exemplary” are identified in this evaluative proposal. The main results include the consolidation of two university research centers in the field of Mathematics Education. However, results show the scarce dissemination that is made of the investigations carried out from the doctoral works (102 out of 199 doctoral theses have not had any associated publication) apparently having, as the main objective, the achievement of the Doctor’s degree instead of the search for development and scientific advancement.

Exemplary theses; Mathematics education; Evaluation science; Bibliometrics

Resumen

El establecimiento de criterios que determinen la calidad de cualquier publicación científica es un hecho consensuado, por toda la comunidad científica, en sus diferentes ámbitos de conocimiento. En el caso de las tesis doctorales, estos criterios pueden variar entre universidades, escuelas de doctorado o comités de evaluación. Este trabajo presenta una propuesta de criterios de evaluación, para tesis doctorales en el ámbito de la educación matemática. Se analizan un total de 199 tesis indexadas en la base de datos TESEO, para el periodo 2015-2020. Asimismo, tras aplicar estos criterios, se identifican aquellas tesis que han sido denominadas como ejemplares a partir de esta propuesta evaluativa. Entre los principales resultados, se puede destacar la consolidación de dos centros universitarios de investigación en el campo de la Educación Matemática. Sin embargo, los resultados muestran la escasa difusión que se hace de las investigaciones realizadas a partir de los trabajos de doctorado (102 de 199 tesis doctorales no han tenido ninguna publicación asociada) teniendo, al parecer, como principal objetivo la obtención del título de Doctor en lugar de la búsqueda del desarrollo y avance científico.

Tesis ejemplares; Educación matemática; Evaluación en Ciencia; Bibliometría

1 Introduction

The study of the doctoral thesis as a product of scientific interest began in the 90’s. At that time, authors such as López-López (1996) stated that one of the most pertinent documentary sources for the state of research in a country through its scientific literature is the literature corresponding to the doctoral theses that doctoral students must read in order to obtain the Doctorate Degree; thus leaving part of that nebulous world of gray literature. This denomination of gray literature is given by the ways in which doctoral theses are published and disseminated; that is, it is part of those scientific documents that are produced and disseminated through non-conventional channels and that do not have an ISBN (as in the case of books) or an ISSN (as in the case of journals). Also, and frequently, doctoral dissertations are not published (and if they are, they are not published in a commercial sense) or they are incompletely published (COOPER; HEDGES; VALENTINE; 2009; PAPPAS; WILLIAMS; 2011 ). Specifically, within scientometric research, doctoral theses have been the object of analysis in studies such as those by Ajanovic; Çizel (2021) , Anderson, Alexander and Saunders (2020), Cleary (2000) , Vallejo, Torralbo and Fernández-Cano (2016), Xie; Li; Li (2020), or González-Pérez; Peralta-González and Meneses-Placeres (2019).

The different studies cited are a clear example of how doctoral theses are a scientific document of great interest for the different scientific branches. In these studies, there are exhaustive reviews on the epistemological positioning of doctoral theses, contemporary research approaches, the interdisciplinary character of doctoral theses or typologies (structures) for the production of these research reports. An inherent issue in these studies has been the deliberation on the quality of the doctoral theses and in this regard, we coined the concept of exemplarity, which has been widely used in Education when referring to one of the values that a good teacher should have. For this study, we will adopt the conception described by Kant (1781 , p.87) when he establishes that “there is exemplarity as soon as there is a rule to follow, what to determine, but exemplarity is without example”.

In other words, we will propose a series of criteria, identified as exemplary, to determine the rule to be followed in the preparation and subsequent publication of the doctoral thesis, considered exemplary. Likewise, and in consequence with what was established by the great philosopher, we will analyze the theses that, following certain criteria, can be considered exemplary theses in the field of Mathematics Education.

1.1 Indicators of exemplarity for doctoral theses

There are numerous studies that have tried to establish and/or adapt certain scientific indicators (up to now applied to manuals, articles, or research projects) to doctoral theses in order to assess their scientific impact (SHARMINI et al. , 2015; UNNIKRISHNAN, 2019 ); separating them from the academic quality that is judged in the public act of reading and defense carried out within the universities. Specifically, in the case of doctoral theses, we can list different studies or organizations that identify evaluative indicators to assess the quality of doctoral theses. A first study would be the one developed by Ortiz-Torres et al. (2010), focused on evaluating the scientific impact of doctoral theses in pedagogical sciences, based on ten indicators:

1. Citation index: frequency with which the thesis is cited in other publications (thesis, monographs, books, articles, etc.) after the dissertation.

2. Visibility index: presence of partial or final results of doctoral theses published in different scientific journals and academic sites on the Internet and their ease of access.

3. Economic-social impact: awards, recognitions, distinctions granted to the partial or final results of the thesis by different entities, agencies, or organizations.

4. Publication in journals (peer review): scientific articles derived from the doctoral thesis, either from their partial or total results that have been published in journals that have an editorial board, which controls the quality of articles.

5. Publication in internationally prestigious databases: specific site where partial or final results of the thesis are published, such as EBSCO, Latindex, the Virtual Library of the Organization of Ibero-American States (OEI), among others.

6. Publication of books and monographs as a continuity of the doctoral thesis: either digital or on paper, but with the requirement that they have an ISBN.

7. Participation in national and international events: presentation of partial or final results of the doctoral thesis in scientific events (national or international) and official certification of their presentation.

8. Participation in projects, research groups and/or thematic networks: the thesis as an output from a project or thematic network.

9. Recognized scientific leadership: that the author directs a group of researchers on a subject related to that developed in the doctoral thesis.

10. Results introduced in social practice: the proof that the contribution of the doctoral thesis has been applied by means of the existence of endorsements from the introducers that certify it (transfer).

Another proposal of evaluative indicators used could be the one used by different Spanish universities to evaluate the quality of their doctoral theses. In most of them, the unanimous criterion to accredit this quality is the publication of an article in journals with impact indexes. Each university, through its doctoral school, constitutes a series of interdisciplinary commissions (one commission for each branch of knowledge) where they agree upon the evaluative criteria for judging the formative and research merits necessary for the presentation and defense of the doctoral thesis at that university.

As an example, the scale used by the University of Malaga (Spain) that establishes:

1. 1 point

   • Journal Citation Report (JCR)

   • Scimago (SJR)

   • Integrated Classification of Scientific Journals (Clasificación Integrada de Revistas Científicas-CIRC), grade A

   • National Commission for the Evaluation of Research Activity (Comisión Nacional Evaluadora de la Actividad Investigadora- CNEAI)

2. 0.75 points

   • CIRC (grade B)

   • CNEAI (score 12-14 points)

   • Latindex (score + 33 criteria)

3. 0.50 points

   • CIRC (grade C)

   • CNEAI (score 9-11points)

   • Latindex (score +30-32 criteria)

4. 0.25 points

   • CNEAI (score 8 points)

Likewise, this university makes it clear that the Academic Committee of the Doctorate Program will not consider the following as quality publications:

• Articles published in non-indexed journals.

• Articles published in journals cataloged in Latindex that meet less than 30 criteria.

• Bibliographic reviews or book prologues.

• Books or book chapters published in publishers not included in the Scholarly Publishers Indicators catalogue (hereafter, SPI) (http://epuc.cchs.csic.es/SPI/disciplinas.php).

• Conference proceedings.

• Classroom materials.

Another study, related to assessing the quality of doctoral theses, is that published by De Miguel (2010) where it establishes that the impact generated by a thesis must be evaluated after a period of time (which is not specified). In this case, the criteria relating to the impact of the thesis are related to questions such as: has the work carried out had any kind of projection to date? Have new lines and research projects been generated from the thesis? Do they provide methodologies of interest to researchers? Impact on the community from the work done? Type of publications made? Presentations at conferences and seminars? Invitations received? or dissemination of the thesis in the scientific community?

Finally, we wish to highlight the doctoral thesis of Fernández Bautista (2018) where he identifies the concept of exemplarity based on two criteria: web citations (Google Scholar) and expert judgment. This author performs an analysis of more than 7,000 doctoral theses in the field of education between 1840-2015 and identifies twenty exemplary doctoral theses applying these two criteria.

As can be seen, the studies collected propose the evaluation of the quality of doctoral theses from the same evaluative criteria that govern the rest of scientific publications, as well as science in general.

In the specific case of this study, the following evaluative criteria have been applied to determine this exemplarity in the Kantian sense of the term rule to follow:

• Resulting publications: total number of publications making a distinction between books or book chapters, magazine articles, and conference proceedings.

• Direct citations: number of citations received for the doctoral thesis.

• Indirect citations: number of citations received by the publications made from the doctoral thesis.

• Latency time: time elapsed since the defence of the thesis and the first publication of the results of the doctoral thesis. Specifically, in this case, it would be desirable for this time span to be as short as possible; this would reflect the importance and impact of the findings of the doctoral theses.

• Validity of the doctoral thesis: period elapsed between the first and last publication. This criterion, unlike the previous one, would be advisable to be as broad as possible. This would imply that the results achieved in the doctoral theses have been considered relevant due to the extent of subsequent publications over the longest possible time.

Undoubtedly, these are criteria based on quantitative parameters on citation, visibility, and accreditation.

The main objective of this study is to identify a series of evaluation criteria for doctoral theses that can be used to assess their quality and impact in their field of study, as it is done in the field of research with other types of reports: papers, books, or proceedings. In this way, and with the aim of contextualizing these evaluation criteria, the following are the specific objectives:

• Identifying the Spanish doctoral theses on Mathematics Education defended during the period 2015-2020.

• Carrying out a longitudinal and productivity study of universities and departments.

• Finding the most productive doctoral thesis directors and the possible existence of in-visible schools.

• Applying the criteria for evaluating the quality of doctoral theses and checking their fit and relevance.

2 Methodology

As said before, this study examines the doctoral theses defended in Spanish universities during the period 2015-2020. Specifically, we carried out a double analysis: on the one hand, we analyzed the longitudinal production of doctoral theses during this period of time, the universities where they were read and defended, and the directors who supervised them. On the other hand, we carried out an analysis of exemplarity, taking into account the previously identified indicators.

The database used for the indexing of doctoral theses is the TESEO database of the Ministry of Education, Culture and Sports, as the main data source that compiles bibliographic records of the doctoral theses defended in Spain (https://www.educacion.gob.es/teseo/irGestionarConsulta.do).

The identification of the sample was carried out in January 2020 by introducing the following search descriptors ( Frame 1 ). Likewise, in order to complete the census, a search was carried out in the databases of national university libraries, in catalogues, and repositories. As a result of this search and, after a task to confirm the object of study (theses focused on the teaching and learning of mathematics), a total of 199 doctoral theses related to the field of mathematics education were identified ( Frame 1 ).

Frame 1
– Descriptors used in TESEO

List 1	List 2	List 3
educ* y matem*.	Logic* y matem*.	Educ* y primaria
Bibliometri* y matem*.	Prof* y matem*	Educ* y secundaria
(didactic* educacion*) y matemática*.	Calcul* y matem *.	Contar y matem*
Matematica.	Evalua* y matem *.	Instruccion y matem*
Numer*.	Azar y matem*.	Historia* y matem*
Razon* y matem*	Aritmetica	Escuela y matem*.
Problema* y matem*.	Geometria y educ*	Escolar y matem*.
(enseña* aprend*) y matem*.	Educ* y infan*	Estadistica y educ*
		Algebra y educ*

Source: prepared by the authors

This study is a quantitative type of research, due to the nature of the data collected and the type of analysis carried out. The method used is descriptive and explanatory, as it is a study that quantitatively describes an available sample based on multiple variables relating to productivity and citation. Similarly, it is a secondary, integrative, and documentary analysis study. In other words, based on a series of search methods and techniques, the information contained in the doctoral theses is analyzed in order to present new systematic, coherent, and sufficiently argued information to respond to the study objectives ( CASASEMPERE-SATORRES; VERCHER-FERRÁNDIZ, 2020 ). The population and the sample coincide; therefore, it is a census study. In other words, we have retrieved all the documents available for research, in this case, all the doctoral theses belonging to the field of Mathematics Education, defended in Spain and indexed in the TESEO database, corresponding to the years between 2015 and 2020.

The computer program used for data analysis was IBM SPSS, version 21.

3 Results and Discussion

3.1 Bibliometrics analysis of doctoral theses production in Mathematics Education

This first analysis enables us to identify the main research centers and main agents of the production of Spanish doctoral theses in Mathematics Education (production by universities, departments, directors, and tribunals). From these analyses it will be possible to infer the main lines of research followed in these studies, epistemological perspectives, research agendas, and collaborative networks (invisible college; CRANE, 1972 ). The latter term refers to communities of experts that create complex networks of scientific collaboration between members from different centres, universities, countries, etc.

3.1.1. Longitudinal analysis

The first variable we examine is the year of reading and defense of each of the doctoral theses that make up the study population. In this way, we can conduct an analysis of the longitudinal development of doctoral theses production in Mathematics Education in Spain (during the established period). For the period studied (2015-2020), a total of 199 doctoral theses related to the field have been recovered from Mathematics Education.

The production analysis allows us to conclude that there is a slight decrease in the number of doctoral theses from 2015 to the present time ( Figure 1 ). These data can be justified by the period of extinction of the old doctoral programs of Spanish universities. In 2011, Royal Decree 99/2011 of 28 January 2011 ( ESPAÑA, 2011 ) which regulates official doctoral studies, was published in Spain, marking the final transformation stage of the European Higher Education Area (EHEA). The basis of this new doctorate is based on the recommendations made at the Salzburg meeting (2005), reformulated in 2010, in an attempt to improve the quality of doctoral studies through collaboration with other institutions and entities involved in R+D+i. Among the main changes included in these regulations are: for accessing these studies it will be necessary to have at least 300 ECTS credits of basic training, the establishment of a maximum period for the presentation of the doctoral thesis (no longer than six years), research training courses throughout the period of enrolment in doctoral studies, the establishment of certain training activities that must be carried out prior to the presentation of the doctoral thesis (scientific publications, attendance at expert meetings and conferences, research stays, membership of research groups, among others).

Figure 1
Longitudinal productivity of doctoral theses on Mathematics Education

This fact meant that, in previous years, there was a great increase in the reading of doctoral theses so that doctoral students did not have to adapt to the new regulations of doctoral programs and their numerous requirements .

Specifically, the approximate average of doctoral theses read during this period was 32 doctoral theses per year, with a maximum of 55 theses in 2015 and only four in 2020. Note that with the descriptor education , in the academic year 20/21, only nineteen doctoral theses appear.

This drastic decrease in production observed in 2020 is probably due to the problem of updating the TESEO database, already manifested in multiple studies (FERNÁNDEZ-CANO; TORRALBO; VALLEJO, 2012; MORALEJO-ÁLVAREZ, 2000 ; VALLEJO; TORRALBO; FERNÁNDEZ-CANO, 2016).

Finally, it should be emphasized that the decelerating trend of the last 5 years is consistent with scientometric studies carried out in previous research (VALLEJO; FERNÁNDEZ-CANO; TORRALBO; MAZ, 2007), which showed a decrease in the exponential growth of Mathematics Education doctoral theses, thus reaching the saturation level proposed by Price (1986) .

3.1.2 Institutional production of doctoral theses on Mathematics Education

As can be seen in Table 1 , the universities with the highest production of doctoral theses on Mathematics Education in the last five years have been the University of Granada (whose production represents 13.07% of the total theses defended between 2015- 2020), followed by the Autonomous University of Barcelona (with 10.05%) and the University of Alicante (with 8.54%). Subsequently, the University of Valencia and the University of Salamanca are placed with 5.52% and 4.52%, respectively, of the total production.

Table 1
– Productivity of doctoral theses by university (min. 5 theses)

Institutions	Initials	Theses
University of Granada	UGR	26
Autonomous University of Barcelona	UAB	20
University of Alicante	UA	17
University of Valencia	UV	11
University of Salamanca	USAL	9
University of Barcelona	UB	8
National University of Distance Education	UNED	7
University of Extremadura	UNEX	7
University of Zaragoza	UNIZAR	6
University of Murcia	UM	6
Complutense University of Madrid	UCM	6
University of Vigo	UVIGO	5

Source: prepared by the authors

The concentration of doctoral thesis production between the University of Granada and the Autonomous University of Barcelona makes these two universities the two major Mathematics Education thesis production and research centers in Spain. Likewise, it should be stated that these two universities have been references in the realization of doctoral theses on Mathematics Education since 1976 - the year in which doctoral theses began to be indexed in the TESEO database. This is confirmed by the studies by Vallejo, et al . (2007) or Vallejo et al . (2016).

3.1.3 Production by departments

Given that the departmental organization of the different Spanish universities does not have a similar nomenclature, in this type of studies with doctoral theses it is difficult to group them by department. In this research, for organizational purposes, a grouping of departments was made based on the following classification: Exclusively educational departments, Exclusively psychological departments, Mixed departments (Education, Psychology and others) , and Other departments (neither educational, nor psychological, miscellaneous category that includes departments of specific didactics (such as didactics of science or arts), or Mathematics departments (not linked to the educational field) where most of the research is done on pure or applied mathematics).

In Figure 2 , we can see that the largest number of doctoral theses carried out on Mathematics Education were assigned to exclusively educational departments (specifically, 64.2% of the theses). Specifically, almost half of these theses were elaborated in departments that make up the Mathematics Didactics knowledge area (48.3%). A much lower percentage of theses (21.9%) were carried out in departments of other fields rather than the educational and psychological realms (for example, departments of Statistics, Mathematics, Doctoral Schools etc.).

Figure 2
Productivity of doctoral theses on Mathematics Education according to departments

On the other hand, thesis production has been much lower in departments related to some knowledge area typical of the Psychology field (7%) and in departments that are constituted from the areas of psychology, education, and/or other related areas of knowledge (7%).

These results show that almost 50% of the production of doctoral theses on Mathematics Education is carried out within the knowledge area closest to this research topic. Likewise, this finding is ratified by the results found in the following section (thesis supervisors).

3.1.4. Production by thesis supervisors

A total of 252 doctoral thesis supervisors were identified for the period 2015/2020. The number of theses produced per supervisor varied between 1 and 5, as can be seen in Table 2 , finding that 82.14% of the supervisors oversaw only one doctoral thesis in the period of time studied; these supervisors may be called - according to Lotka’s law (1926) - occasional supervisors.

Table 2
– Number of theses produced per supervisor

Number of supervised theses	Supervisors	Percentage
1	207	82,14%
2	27	10,71%
3	12	4,76%
4	2	0.79%
5	4	1,59%
Total	252	100%

Source: prepared by the authors

In contrast to these occasional supervisors are those who in a period of six years (2015/2020) supervised four or five doctoral theses (great supervisors). Table 3 shows the nominal relationship of those who supervised between 4 and 5 doctoral theses on mathematics education throughout these years.

Table 3
– Supervisors who supervised 4 or 5 doctoral theses (2015 and 2020)

Number of supervised theses		Supervisors
5		Josep María Fontuny Aymemich
5		Encarnación Castro Martínez
5		Alexander Maz Machado
5		María Consuelo Cañadas Santiago
4		Jordi Deulofeu Piquet
4		Pablo Flores Martínez

Source: prepared by the authors

The highest production in the direction of doctoral works (five doctoral theses) corresponds to Mr. Josep María Fontuny Aymemich (Professor at the Autonomous University of Barcelona), Ms. Encarnación Castro Martínez (Retired Professor at the University of Granada), Mr. Alexander Maz Machado (Professor at the University of Córdoba), and Ms. María Consuelo Cañadas Santiago (Professor at the University of Granada).

It is interesting to highlight the relationship found between the most productive thesis supervisors (with a higher doctoral supervision) and the universities with a higher production. This is the case of Ms. Encarnación Castro Martínez and Ms. María Consuelo Cañadas Santiago from the University of Granada, and Mr. Josep María Fontuny Aymemich from the Autonomous University of Barcelona; it is a finding that shows the consolidation of research groups in the field of Mathematics education at these two universities.

As a complement to the study of the supervision of doctoral theses, a network analysis (see Figure 3 ) of the members that made up the evaluation boards of the theses was carried out. The objective was to analyze the possible existence of invisible college ( CRANE, 1972 ); that is, groups of researchers who carry out their scientific activity under the influence of a renowned researcher.

Figure 3
Invisible college in the committees of exemplary theses of mathematics education (2015-2019)

From this analysis, a network of 33 members of doctoral thesis committees has been obtained that are interrelated and give rise to an appropriate graph for their analysis. The rest of the components (committee members) are not part of this main network, so they were discarded.

Within this main network, six different clusters have been identified:

Cluster 1, identified in Figure 3 in red, in the center of the image: it is the majority cluster, formed by 8 members: Alicia Bruno Castañeda, Ángel Alsina Pastells, Antonio Miguel Oller Marcén, Carlos de Castro Hernández, Joaquín Giménez Rodríguez, Jordi Deulofeu Piquet, Núria Gorgorió Solà, and Teresa Fernández Blanco.

Cluster 2, identified in Figure 3 in green, in the center right of the image: it is a large cluster, with 7 members: Antonio Estepa Castro, Carmen Batanero Bernabeu, Enrique de la Torre Fernández, José Miguel Contreras García, Juan Díaz Godino, Mª Asunción Estrada Roca, and Vicent Font Moll.

Cluster 3, which corresponds to dark blue in the center of the image: it is a cluster that is smaller in number, but with more centrality in the network, that is, better communication. It is made up of 5 members or items: Ceneida Fernández Verdú, Gloria María Sánchez Matamoros García, Josep María Fortuny Aymemich, Philippe R. Richard, and Mª Mercé Edo Basté.

Cluster 4, represented in yellow on the left of the image: it is a smaller cluster with less centrality, made up of 5 members: Cesáreo Paulo Lameiras Almeida, Clara Jiménez Gestal, José María Chamoso Sánchez, Lorenzo J. Blanco Nieto, and Luis Manuel Casas García.

Cluster 5 is the furthest from the center of this main network. It is represented in purple, to the right of the image. It has three members: José María Gavilán Izquierdo, Mª Luz Callejo de la Vega, and Mª Teresa González Astudillo.

Cluster 6, formed by Francisco Javier García García, Luis Puig Espinosa, Nuria Planas Raig, and Pablo Flores, which is identified in light blue in Figure 3 .

3.2 Analysis of the exemplary nature of doctoral theses

This second analysis evaluates the quality of the doctoral theses analyzed. Specifically, the scientific impact of the doctoral theses in mathematics education in Spain was evaluated using the indicators defined at the beginning of this work. The implementation and analysis of their relevance will make it possible to establish an additional evaluation protocol to those used so far by research centers and universities when evaluating the quality of the doctoral theses defended ( DE-MIGUEL, 2010 ; GONZÁLEZ-PÉREZ; PERALTA-GONZÁLEZ; MENESES-PLACERES, 2019).

3.2.1 Resulting publications

In accordance with the above, all publications generated from the analyzed doctoral theses have been counted as resulting publications. For this, an analysis of the research topics has been carried out (which must be similar to those of the doctoral thesis) and the methodological data (sample, information collection instruments etc.)

According to the results obtained after the search carried out – in the Dialnet and Google academic databases – 102 doctoral theses have not had any publications resulting from this doctoral work.

According to Table 4 , it is observed that only two doctoral theses have had a considerable number of publications; specifically, seventeen publications, while the remaining theses register between one and seven publications.

Table 4
– Number of publications resulting from doctoral thesis

Number of theses	Number of resulting publications
102	0
39	1
30	2
11	3
6	4
5	5
2	6
2	7
2	17
Total	199

Source: prepared by the authors

These two doctoral theses are defended by J. García School failure from the perspective of educational exclusion. The prescribed curriculum of the PDC and the PCPI in the Valencian community and by C.M. León Juan Cortázar and his contribution to Spanish mathematical training in the 19th century .

These results show the scarce dissemination that is made of the investigations carried out from the doctoral works; apparently having, as the main objective, the achievement of the Doctor’s degree instead of the search for development and scientific advancement, which is an issue that requires broad and deep reflection.

3.2.2 Citations (direct / indirect)

Regarding the citations received, both direct citations (made to the doctoral thesis itself) and indirect citations (resulting publications) counted in academic Google have been counted. The use of this resource is due to the fact that most of the publications resulting from doctoral theses, in addition to the doctoral thesis itself, are not indexed in databases that provide citation data.

Of the 199 doctoral theses analyzed, only 64 obtained a citation. Of these 64 theses, a small number (seven) group a considerable number of citations, and a much larger number of theses (sixty) received few citations (between one and ten citations). Specifically, the four doctoral theses with the most citations accumulated 71.05% of them, the sixty theses that received between one and ten citations accounted for 65.35%, while the remaining 124 theses did not register any citations.

Focusing on the theses that received the highest number of citations, it is worth highlighting, first, the doctoral thesis entitled: Academic performance of compulsory secondary school students and the relationship with mental aptitudes and attitudes towards study by L.O. Solano, defended in 2015 at the National Distance Education University, with 94 citations received. Second to be highlighted is the thesis entitled: A possible ‘Reason for being’ of elemental differential calculus in the field of functional modeling by C. Oliveira from the University of Vigo (2015) with 28 citations. Third, with a total of nine citations is the thesis entitled: Competence in didactic analysis in the initial training of teachers of basic general education with a mention in mathematics by M. J. Seckel from the University of Barcelona (2016). And, fourthly, the doctoral thesis entitled: Emotions in the teaching and learning of science in secondary education by A.B. Borrachero from the University of Extremadura (2015), which received a total of six citations during the period studied.

3.2.3 Latency time

When the latency time analysis was performed; that is, the time elapsed from the defense of the thesis and the first publication of the doctoral thesis results, it can be observed ( Table 5 ) that 102 doctoral theses did not have publications after the defense of the doctoral thesis (as indicated above). Of the 97 doctoral theses that did result in publications, in 59.79% of the cases (58 theses) less than a year elapsed from the defense of the thesis and the first publication of the results. 27.83% of the theses published a year after the defense. Only ten theses took up to two years to publish their results and two theses did so in three years.

Table 5
– Latency time per doctoral thesis

Number of theses	Latency time (in years)
58	0
27	1
10	2
2	3
102	Not published
Total: 199

Source: prepared by the authors

This finding is considerably higher than that evidenced in Miyahira’s (2015) study on doctoral theses in the field of medicine in Peru. His study concludes that only 5% of medical theses are published and that in the case of Geriatrics theses, this percentage rises to 11.2%. Similar results are found in the study by Ortiz-Torres et al . (2010) where he analyzes doctoral theses in pedagogical sciences in Cuba.

Regarding the latency time, the results found suggest that publications derived from doctoral theses maintain a similar publication pattern (in this respect) to the rest of the research reports.

3.2.4 Validity of the doctoral thesis

In relation to the term of the theses (time elapsed between the first publication and the last publication), the data show ( Table 6 ) the existence of a short term, which ranges between 0 and 1 years in 60.82% of doctoral theses. Only 7 doctoral theses (7.21%) published their results up to 4 years after their first publication of the thesis.

Table 6
– Validity of doctoral theses

Number of theses	Years of validity	Percentage
102	Not published	-
34	1	35.05%
25	0	25.77%
16	2	16.49%
15	3	15.46%
7	4	7.21%
Total: 199

Source: prepared by the authors

It should be considered that this percentage could be somewhat higher for those doctoral theses that were defended in 2020, taking into account the aforementioned validity result.

Finally, it should be noted that almost 32% of doctoral theses would publish their main findings over a period of two-three years.

From these results, it is inferred that the obsolescence of the publications resulting from doctoral theses is four years (maximum); or using the terminology of Price (1986) , that doctoral theses die in the four years following their defense.

4 Conclusions

In relation to the findings related to the field of mathematics education, the consolidation of this field has been verified through the production of doctoral theses and two large research centers: the University of Granada and the Autonomous University of Barcelona. Likewise, the network analysis confirms the existence of these two nerves centers and makes it possible to represent the existing connections between the members of the evaluation committees from these university institutions.

With the findings presented in this study, it is evident that the investigations carried out by means of doctoral theses can be evaluated following criteria and parameters like those used in other formats of research reports, such as journal articles. Likewise, it is revealed that there are still no criteria agreed upon by the scientific community that allow the analysis and quantification of the research impact carried out through doctoral theses.

This article can be a starting point to widen this debate. We consider that doctoral theses should be evaluated from a double perspective: on the one hand, evaluating the relevance and quality of the study for granting the doctoral degree (academic merit). And, on the other hand, evaluating the impact of their research findings on the field of study. The conjunction of these two optics or goals of doctoral theses would allow the incorporation of important advances to the university field (Academia) and to the scientific field (Science).

This duality of objectives would allow the creation of a protocol for the evaluation of the quality of doctoral theses applicable by any university to doctoral theses or other research reports (e.g. Master’s theses) developed in that institution, as suggested by De-Miguel (2010) , González-Pérez; Peralta-González; Meneses-Placeres (2019) y Xie; Li; Li (2020), thus ensuring compliance with the levels of rigor and requirements established by that institution.

References

• AJANOVIC, E.; ÇIZEL, B. How interdisciplinarity helps knowledge production: Reflections on a doctoral dissertation. Journal of Hospitality, Leisure, Sport & Tourism Education , New York, v. 28, [s.n.], p. 100310, Jun. 2021. DOI 10.1016/j.jhlste.2021.100310. Available at: https://linkinghub.elsevier.com/retrieve/pii/S1473837621000113 Accessed on: 6 Jul. 2021.
  » https://doi.org/10.1016/j.jhlste.2021.100310.» https://linkinghub.elsevier.com/retrieve/pii/S1473837621000113
• ANDERSON, T.; ALEXANDER, I.; SAUNDERS, G. An examination of education-based dissertation macrostructures. Journal of English for Academic Purposes , London v. 45, [s.n.], p. 100845, May 2020. DOI 10.1016/j.jeap.2020.100845. Available at: https://linkinghub.elsevier.com/retrieve/pii/S1475158519303509 Accessed on: 6 Jul. 2021
  » https://doi.org/10.1016/j.jeap.2020.100845» https://linkinghub.elsevier.com/retrieve/pii/S1475158519303509
• CLEARY, R. E. The Public Administration Doctoral Dissertation Reexamined: An Evaluation of the Dissertations of 1998. Public Administration Review , Florida, v. 60, n. 5, p. 446-455, Sep. 2000. DOI 10.1111/0033-3352.00107. Available at: https://onlinelibrary.wiley.com/doi/10.1111/0033-3352.00107 Accessed on: 6 Jul. 2021.
  » https://doi.org/10.1111/0033-3352.00107» https://onlinelibrary.wiley.com/doi/10.1111/0033-3352.00107
• CASASEMPERE-SATORRES, A.; VERCHER-FERRÁNDIZ, M. L. Análisis documental bibliográfico. Obteniendo el máximo rendimiento a la revisión de la literatura en investigaciones cualitativas. New Trends in Qualitative Research , Aveiro, v. 4, [s.n.], p. 247-257, 2020.
• COOPER, H.; HEDGES, L. V.; VALENTINE, J.C. The handbook of research synthesis and meta-analysis . Nueva York: Russell Sage Foundation, 2009.
• CRANE, D. Invisible Colleges. Diffusion of knowledge in scientific communities. Chicago: The Chicago University Press, 1972.
• DE-MIGUEL, M. La evaluación de tesis doctorales. Propuesta de un modelo. RELIEVE. Revista Electrónica de Investigación y Evaluación Educativa, Granada, v. 16, n. 1, p. 1-18, 2010.
• FERNÁNDEZ-CANO, A.; TORRALBO, M.; VALLEJO, M. Time series of scientific growth in Spanish doctoral theses (1848–2009). Scientometrics, Budapest,v. 91, [s.n.], p. 15–36.
• ESPAÑA. Real Decreto 99, de 28 de enero de 2011, por el que se regulan las enseñanzas oficiales de doctorado. BOE n. 35, Madrid, 10 de febr. 2011. p.2541. Disponible en: https://www.boe.es/buscar/act.php?id=BOE-A-2011-2541 Acceso en: 24 mayo 2023.
  » https://www.boe.es/buscar/act.php?id=BOE-A-2011-2541
• FERNÁNDEZ-BAUTISTA, A. F. Análisis diacrónico y prospectivo de las tesis doctorales españolas en educación (1840-2020) . 2018, p.248. Universidad de Granada, Granada, 2018.
• FERNÁNDEZ-CANO, A.; TORRALBO, M.; VALLEJO, M. Time series of scientific growth in Spanish doctoral theses (1848–2009). Scientometrics , Budapest, v. 91, [s.n.], p.15–36, 2012.
• GONZÁLEZ-PÉREZ, L.; PERALTA-GONZÁLEZ, M. J.; MENESES-PLACERES, G. Undergraduate research: Evaluation of its quality through theses. Bibliotecas, Anales de Investigación , La Habana, v. 15, n. 1, p. 51-67, 2019.
• KANT, I. Critique of pure reason. Modern Classical Philosophers . Cambridge: Houghton Mifflin, 1781.
• LÓPEZ-LÓPEZ, P. Introducción a la bibliometría . Valencia: Promolibro, 1996.
• LOTKA, A. F. The frequency distribution of scientific productivity. J. Wash. Acad. Sci. , Washington , v. 16, n.19, p. 317-323, 1926.
• MIYAHIRA, J. ¿Por qué la tasa de publicación de las tesis en Medicina es baja? Posibles explicaciones . Rev Medica Hered, Lima , v. 26, n. 4, p. 207, 2015.
• MORALEJO-ÁLVAREZ, M. R. Las tesis doctorales de las Universidades españolas. Control bibliográfico y acceso. Revista general de información y documentación , Madrid, v. 10, n. 1, p. 235-243, 2000.
• ORTIZ-TORRES, E.; GONZÁLEZ-GUITIÁN, M. V.; INFANTE PÉREZ, I.; VIAMONTES- GARRIDO, Y. Evaluación del impacto científico de las tesis doctorales en Ciencias Pedagógicas mediante indicadores cienciométricos. Revista Española de Documentación Científica , Madrid, v. 33, n. 2, p. 279-286, 2010. DOI 10.3989/redc.2010.2.728. Available at: http://redc.revistas.csic.es/index.php/redc/article/view/555/629 Acceso en: 6 de julio 2021
  » https://doi.org/10.3989/redc.2010.2.728» http://redc.revistas.csic.es/index.php/redc/article/view/555/629
• PAPPAS, C.; WILLIAMS, I. Grey literature: its emerging importance. Journal of Hospital Librarianship , London, v. 11, n. 3, p. 228-234, 2011.
• PRICE, D. J. Little science, big science... and beyond . New York: Columbia University Press, 1986.
• SHARMINI, S.; SPRONKEN-SMITH, R.; GOLDING, C.; HARLAND, T. Assessing the doctoral thesis when it includes published work. Assessment & Evaluation in Higher Education , London, v. 40, n. 1, p. 89-102, 2015.
• UNNIKRISHNAN, S. An analysis of the quality of Ph.D theses: A peep through in-text citations and bibliography. Library Philosophy and Practice (e-journal) , Lincoln, v. 1, [s.n.], 2019. Available at: https://digitalcommons.unl.edu/libphilprac/2219/ Acceso en: 24 de mayo 2021
  » https://digitalcommons.unl.edu/libphilprac/2219/
• VALLEJO, M.; FERNÁNDEZ CANO, A.; TORRALBO, M.; MAZ, A. La investigación española en educación matemática desde el enfoque conceptual inserto en sus tesis doctorales. Enseñanza las Ciencias , Barcelona, v. 25, [s.n.], p. 259–266, 2007.
• VALLEJO, M.; TORRALBO, M.; FERNÁNDEZ-CANO, A. Gender Bias in Higher Education. Journal of Hispanic Higher Education, California, v. 15, [s.n.], p. 205–220, 2016.
• XIE, Z.; LI, Y.; LI, Z. Assessing and predicting the quality of research master’s theses: an application of scientometrics. Scientometrics , Budapest, v. 124, n. 2, p. 953-972, 2020.

Publication Dates

• Publication in this collection
  01 Sept 2023
• Date of issue
  May-Aug 2023

History

• Received
  28 Oct 2021
• Accepted
  16 Dec 2022