Research Evaluation: Mapping the Field Structure

Pinho, Isabel; Pinho, Cláudia; Rosa, Maria J.

doi:10.1590/S1414-40772020000300003

Isabel Pinho

University of Aveiro | DCSPT | Aveiro | Portugal. Contato: isabelpinho@ua.pt

http://orcid.org/0000-0003-1714-8979

Cláudia Pinho

University of Aveiro | DCSPT | Aveiro | Portugal.. Contato: claudiapinho@ua.pt

http://orcid.org/0000-0001-6249-5019

Maria J. Rosa

University of Aveiro | DEGEIT | Aveiro | Portugal. Contato: m.joao@ua.pt

http://orcid.org/0000-0003-2956-6801

Cluster	Number of publications ^a	Number of publications with 10 or more citations	Number of citations
1	1083	221	6385
2	409	38	409
3	78	3	78
4	69	4	69
5	68	2	68
6	28	1	61

		Author (year)	Number of citations^a	Title
Subcluster 1.1	Seminal	Garfield (1955GARFIELD, E. Citation indexes for science: a new dimension in documentation through association of ideas. Science, UK, v. 122, p. 108-111, 1955. )	28	Citation indexes for science: a new dimension in documentation through association of ideas.
		Garfield (1972GARFIELD, E. Citation Analysis As a Tool in Journal Evaluation - Journals Can be Ranked by Frequency and Impact of Citations for Science Policy Studies. Science, UK, v. 178, p. 471-479, 1972. )	17	Citation Analysis As a Tool in Journal Evaluation - Journals Can be Ranked by Frequency and Impact of Citations for Science Policy Studies.
		Merton (1967MERTON, R. On theoretical sociology. Five essays, old and new. New York: The Free Press, 1967.)	22	On Theoretical Sociology. Five Essays, Old and New.
	Highest citation	Hirsch (2005HIRSCH, J. An index to quantify an individual’s scientiﬁc research output. Proceedings of the National Academy of Sciences of the United States of America, USA, v. 102, n. 46, p. 16569-16572, 2005. )	134	An index to quantify an individual’s scientiﬁc research output.
		Moed (2005MOED, H. F. Citation analysis in research evaluation. Dordrecht: Springer Verlag, 2005.)	73	Citation Analysis in Research Evaluation.
		Van Raan (2005)	43	Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods.
		Hicks (2012HICKS, D. Performance-based university research funding systems. Research Policy, Brighton, UK, v. 41, n. 2, p. 251-261, 2012. )	34	Performance-based university research funding systems.
	Seminal	Macroberts and Macroberts (1989MACROBERTS, M. H.; MACROBERTS, B. R. Problems of citation analysis: A critical review. Journal of the American Society for information Science, USA, v. 40, n. 5, p. 342, 1989. )	15	Problems of citation analysis: A critical review.
	Highest citation	*Nederhof et al.* (1993**NEDERHOF, A. J. et al. Research performance indicators for university departments: A study of an agricultural university. Scientometrics, Belgium,v. 27, n. 2, p. 157-178, 1993.)	41	Research performance indicators for university departments: A study of an agricultural university.
		Meho and Yang (2007MEHO, L. I.; YANG, K. Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar. Journal of the American Society for Information Science and Technology, USA, v. 58, n. 13, p. 2105-2125, 2007. )	38	Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus Scopus and Google scholar.
Subcl. 1.3	Seminal	Garfield (1986GARFIELD, E. Current Contents. Mapping cholera research and the impact of Sambhu Nath De of Calcutta. Essays of an Information Scientist, USA, v. 9, n. 15, p. 112, 1986. )	6	Mapping cholera research and the impact of Sambhu Nath De of Calcutta
	Highest citation	Baird and Oppenheim (1994BAIRD, L. M.; OPPENHEIM, C. Do citations matter? Journal of Information Science, Aberystwyth, UK, v. 20, n. 1, p. 2-15, 1994. )	17	Do citations matter?
		Leydesdorff (1998LEYDESDORFF, L. Theories of citation? Scientometrics, Belgium, v. 43, n. 1, p. 5-25, 1998. )	17	Theories of citation?
Subcl. 1.4	Seminal	Liebowitz and Palmer (1984LIEBOWITZ, S. J.; PALMER, J. P. Assessing the relative impacts of economics journals. Journal of Economic Literature, USA, v. 22, n. 1, p. 77-88, 1984.)	8	Assessing the relative impacts of economics journals.
	Highest citation	Oswald (2007OSWALD, A. J. An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision-Makers. Economica, USA,v. 74, n. 293, p. 21-31, 2007.)	16	An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision-Makers.
Subcl. 1.5	Seminal & Highest citation	Bordons et al. (2002)	12	Advantages and limitations in the use of impact factor measures for the assessment of research performance.

		Author (year)	Number of citations^a	Title
Subcluster 2.1	Seminal	Polanyi (1962POLANYI, M. The republic of science: Its political and economic theory. Minerva, Netherlands, v. 1, p. 54-73, 1962. )	7	The Republic of science: Its political and economic theory.
		Glaser and Strauss (1967GLASER, B. G.; STRAUSS, A. L. The Discovery of Grounded Theory: Strategies for Qualitative Research. Chicago: Aldine, 1967. )	5	The Discovery of Grounded Theory: Strategies for Qualitative Research.
		Kuhn (1962KUHN, T. S. The structure of scientific revolutions. Chicago, US: University of Chicago Press, 1962.)	5	The structure of scientific revolutions.
		Dimaggio and Powell (1983DIMAGGIO, P. J.; POWELL, W. W. The Iron Cage Revisited: Institutional Isomorphism and Collective Rationality in Organizational Fields. American Sociological Review, USA, v. 48, n. 2, p. 147-160, 1983. )	7	The Iron Cage Revisited: Institutional Isomorphism and Collective Rationality in Organizational Fields.
	Highest citation	*Gibbons et al.* (1994**GIBBONS, M. et al. The New Production of Knowledge: The dynamics of science and research in contemporary societies. London: SAGE Publications, 1994. )	23	The New Production of Knowledge: The Dynamics of Science and Research in Contemporary Societies
		Nowotny, Scott and Gibbons (2001NOWOTNY, H.; SCOTT, P.; GIBBONS, M. Re-thinking science: knowledge and the public in an age of uncertainty. Cambridge: Polity Press, 2001.)	15	Re-thinking science: Knowledge and the public in an age of uncertainty.
Subcluster 2.2	Seminal	Mahoney (1977MAHONEY, M. J. Publication prejudices: An experimental study of confirmatory bias in the peer review system. Cognitive Therapy and Research, Boston, USA, v. 1, n. 2, p. 161-175, 1977. )	4	Publication prejudices: An experimental study of confirmatory bias in the peer review system.
		Ceci and Peters (1982CECI, S. J.; PETERS, D. P. Peer Review: A Study of Reliability. Change: The Magazine of Higher Learning, Canada, v. 14, n. 6, p. 44-48, set. 1982.)	4	Peer Review: A Study of Reliability.
	Highest citation	Macdonald and Kam (2007)	16	Ring a ring o’roses: Quality journals and gamesmanship in management studies.
		Geary, Marriott and Rowlinson (2004GEARY, J.; MARRIOTT, L.; ROWLINSON, M. Journal Rankings in Business and Management and the 2001. Research Assessment Exercise in the UK. British Journal of Management, UK, v. 15, n. 2, p. 95-141, 2004.)	14	Journal Rankings in Business and Management and the 2001 Research Assessment Exercise in the UK.
		Starbuck (2005STARBUCK, W. H. How much better are the most-prestigious journals? The statistics of academic publication. Organization Science, USA, v. 16, n. 2, p. 180-200, 2005. )	14	How Much Better are the Most-Prestigious Journals? The Statistics of Academic Publication.
Subcluster 2.3	Seminal	WEISS (1979WEISS, C. H. The many meanings of research utilization. Public administration review, USA, v. 39, n. 5, p. 426-431, 1979.)	6	The many meanings of research utilization.
	Highest citation	Martin (2011MARTIN, B. R. The Research Excellence Framework and the 'impact agenda': are we creating a Frankenstein monster? Research Evaluation, Oxford, UK, v. 20, n. 3, p. 247-254, Sept. 2011.)	14	The Research Excellence Framework and the ‘impact agenda’: are we creating a Frankenstein monster?
		Buxton and Hanney (1996BUXTON, M.; HANNEY, S. How Can Payback from Health Services Research Be Assessed? Journal of Health Services Research, UK, v. 1, n. 1, p. 35-43, 1996. )	11	How Can Payback from Health Services Research Be Assessed?
		Donovan (2007DONOVAN, C The qualitative future of research evaluation. Science and Public Policy, Oxford, UK, v. 34, n. 8, p. 585-597, 2007.)	10	The qualitative future of research evaluation.
Subcluster 2.4	Seminal	Levin and Stephan (1991)	2	Research productivity over the life cycle: Evidence for academic scientists.
	Highest citation	Dietz and Bozeman (2005)	7	Academic careers, patents, and productivity: industry experience as scientific and technical human capital.
		Lee and Bozeman (2005LEE, S.; BOZEMAN, B. The impact of research collaboration on scientific productivity. Social Studies of Science, Canada, v. 35, n. 5, p. 673-702, 2005. )	7	The Impact of Research Collaboration on Scientific Productivity.
Subcl. 2.5	Seminal & Highest citation	Lincoln and Guba (1985)	6	Naturalistic Inquiry.
		Miles et al. (2013)	6	Qualitative Data Analysis.
Subcl. 2.6	Seminal & Highest citation	Charnes et al. (1978)	5	Measuring the efficiency of decision making units.

[TFN7] ^a
number of citations of the publication within the citation network being analyzed and obtained from CitNetExplorer

[TFN8] Source: Authors

Brasil

Brasil

Research Evaluation: Mapping the Field Structure

Avaliação da Pesquisa: Mapeando a Estrutura do Campo

		Author (year)	Number of citations^a	Title
Cluster 3	Seminal	*Woolf et al.* (1999**WOOLF, S. H. et al. Clinical guidelines: potential beneﬁts limitations and harms of clinical guidelines. BMJ, USA, v. 318, p. 527-530, 1999. )	7	Clinical guidelines: potential beneﬁts limitations and harms of clinical guidelines.
	Highest citation	*Brouwers et al.* (2010**BROUWERS, M. C. et al. AGREE II: advancing guideline development, reporting and evaluation in health care. Canadian Medical Association Journal, Canada, v. 182, n. 18, p. E839-E842, dec. 2010. )	18	AGREE II: advancing guideline development, reporting and evaluation in health care.
		Cluzeau (2003CLUZEAU, F. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Quality & Safety in Health Care, UK, v. 12, n. 1, p. 18-23, 2003. )	16	Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project.
Cluster 4	Seminal	Kuhn (1962KUHN, T. S. The structure of scientific revolutions. Chicago, US: University of Chicago Press, 1962.)	9	The structure of scientific revolutions.
		*Beck et al.* (1961**BECK, A. T. et al. An inventory for measuring depression. Archives of General Psychiatry, USA, v. 4, p. 561-571, 1961. )	7	An inventory for measuring depression.
	Highest citation	APA (1994APA. Diagnostic and Statistical Manual of Mental Disorders (DSM-IV). 4. Arlington: American Psychiatric Association, 1994. )	24	Diagnostic and Statistical Manual of Mental Disorders (DSM-IV)
		Cohen (1988COHEN, J. Statistical Power Analysis for the Behavioral Sciences. 2. Ed.Hillsdale, NJ: Lawrence Erlbaum Associates, 1988.)	12	Statistical Power Analysis for the Behavioral Sciences.
		Landis and Koch (1977LANDIS, J. R.; KOCH, G. G. The measurement of observer agreement for categorical data. Biometrics, Washington, USA, v. 33, n. 1, p. 159-174, 1977. )	10	The Measurement of Observer Agreement for Categorical Data.
Cluster 5	Seminal	Freeman (1979FREEMAN, L. C. Centrality in Social Networks Conceptual Clariﬁcation. Social Networks, UK, v. 1, n. 3, p. 215-239, 1979. )	6	Centrality in Social Networks Conceptual Clariﬁcation.
		Anderson (1998ANDERSON, G. Fundamentals of educational research. London: Routledge, 1998.)	6	Structural equation modeling in practice: A review and recommended two-step approach.
		Cronbach and Meehl (1955CRONBACH, L. J.; MEEHL, P. E. Construct validity in psychological tests. Psychological Bulletin, USA, v. 52, n. 4, p. 281-302, 1955. )	5	Construct Validity in Psychological Tests.
	Highest citation	*House et al.* (2004**HOUSE, R. J. et al. Culture, leadership, and organizations: the GLOBE study of 62 societies. Thousand Oaks, CA: Sage, 2004.)	13	Culture, leadership, and organizations: The GLOBE study of 62 societies.
		Hofstede (2001HOFSTEDE, G. Culture's consequences: comparing values, behaviors, institutions and organizations across nations. 2. ed. Thousand Oaks: SAGE Publications, 2001. )	11	Culture's Consequences: Comparing Values, Behaviors, Institutions and Organizations Across Nations.
		Hofstede (1980HOFSTEDE, G. Culture’s consequences: International differences in work-related value. Beverly Hills, CA: Sage, 1980.)	9	Culture’s consequences: International differences in work-related value.
Cluster 6	Seminal	Callaham, Wears and Weber (2002CALLAHAM, M.; WEARS, R. L.; WEBER, E. Journal prestige, publication bias, and other characteristics associated with citation of published studies in peer-reviewed journals. JAMA - Journal of the American Medical Association, USA, v. 287, n. 21, p. 2847-2850, 2002. )	6	Journal prestige, publication bias, and other characteristics associated with citation of published studies in peer-reviewed journals.
		Vinkler (2003VINKLER, P. Relations of relative scientometric indicators. Scientometrics, Belgium, v. 58, n. 3, p. 687-694, Nov. 2003.)	5	Relations of relative scientometric indicators.
	Highest citation	Young, Ioannidis and Al-Ubaydli (2008YOUNG, N. S.; IOANNIDIS, J. P. A.; AL-UBAYDLI, O. Why current publication practices may distort science. Plos Medicine, UK, v. 5, n. 10, p. e201, 2008)	10	Why Current Publication Practices May Distort Science.
		Patsopoulos, Analatos and Ioannidis (2005PATSOPOULOS, N. A.; ANALATOS, A. A.; IOANNIDIS, J. P. Relative citation impact of various study designs in the health sciences. JAMA - Journal of the American Medical Association, USA, v. 293, p. 2362-2366, 2005. )	8	Relative citation impact of various study designs in the health sciences.