Quality of higher education and the complex exercise of proposing indicators

ABSTRACT

This article provides reflections on the quality of higher education carried out on the Observatory of Higher Education Project, developed by the South Brazilian Network of Researchers in Higher Education – RIES/ CAPES/INEP. The methodology, involving different quanti-qualitative procedures, points to the complexity of proposing evaluation indicators which assume university’s contextualized and temporal character and take multireferentiality as a value. Data review is accompanied by the project’s general theoretical interpretation and by specific interpretations of axes relating the quality of higher education to themes of internationalization, administration, undergraduate teaching, innovation, professional teacher training and development. Thus, indicators are constructed aiding the perception of quality and contributions offered.

KEYWORDS
higher education; quality; university; indicators; Observatory of Higher Education CAPES/INEP

RESUMO

Este artigo sintetiza algumas reflexões sobre a qualidade da educação superior realizadas no Projeto Observatório da Educação Superior, desenvolvido pela Rede Sulbrasileira de Investigadores de Educação Superior – RIES/CAPES/INEP. A metodologia, com diferentes procedimentos quantiqualitativos, aponta a complexidade de serem propostos indicadores de avaliação que assumem o caráter contextualizado e temporal da universidade e a multirreferencialidade como valor. A análise dos dados é acompanhada de interpretação teórica geral do projeto e específica de eixos que relacionam qualidade da educação superior aos temas de internacionalização, gestão, ensino de graduação, inovação, formação e desenvolvimento profissional docente. Constroem-se, assim, indicadores que auxiliam a percepção sobre a qualidade e oferecem contribuições.

PALAVRAS-CHAVE
educação superior; qualidade; universidade; indicadores; Observatório de Educação CAPES/INEP

RESUMEN

Este artículo sintetiza algunas reflexiones acerca de la calidad de la educación superior realizadas por el Projeto Observatório da Educação Superior, desarrollado por la Rede Sulbrasileira de Investigadores de Educação Superior – RIES/CAPES/INEP. La metodología, con diferentes procedimientos cuanti-cualitativos, apunta la complejidad de proponerse indicadores de evaluación que asumen el carácter contextualizado y temporal de la universidad y la multirreferencialidad como valor. La análisis de los datos es acompañada de interpretación teórica general del proyecto y específica de ejes que relacionan la calidad de la educación superior a los temas de internacionalización, gestión, enseñanza de grado, innovación, formación y desarrollo profesional docente. De esta manera, se construyen indicadores que auxilian la percepción acerca de la calidad y ofrecen contribuciones.

PALABRAS CLAVE
educación superior; calidad; universidad; indicadores; Observatorio de Educação CAPES/INEP

To shed light on reflections about the quality of higher education and possible indicators that can guide its evaluation, we contemplate the phenomenon by contextualizing daily academic practices and discourses produced in this environment. The concept of quality is unquestionably affected by educational policies and regulations and by social representations, particularly those manifest on social communication media. It should be emphasized, however, that academic culture is also an important factor in determining understandings about the quality of higher education. The words of Milton Santos (2000, p. 76) reveal this condition:

The tyranny of information isn't just the media's, for it also includes our work at the university. I want to insist on this point, because our work as teachers is the foundation upon which generations are educated and reeducated. The freer our work, the more we educate towards citizenship. The more our work is chained, the more we will be producing feeble individualities. It is urgent that education recognize this situation, to delineate the deserved reaction, without which we run a great risk of becoming farther from the ideal pursuit of truth.

These considerations are quite stimulating because, in the sphere of reflections about higher education, we frequently hear the expressions "for a quality higher education" and "an educational institute of excellence". These expressions appear to encompass multiple understandings. When one speaks of quality education, a complement doesn't seem necessary, because the term "quality", as much as "excellence", points to the maximum, to the best.

However, the concept of quality is multidimensional, which makes defining its meaning a complex process. The understanding of this concept incorporates an ethical and aesthetic dimension, and, especially, an axiological one. Rios (2001, p. 68) emphasizes that the term "quality" implies the idea of something good. Nonetheless, quality is not an adjective that refers to a universal construct, but a property that is found in beings, actions or objects. An earlier concept is found in these expressions, which assumes a valuative position, and that is linked to morality and the political condition of man. This signifies that quality is self-referential; it presupposes a subject or a community that accepts certain standards as desirable.

To define quality, we must first explain the sense of the action and the scope upon which its intentions are established. By indicating these ideas, we want to draw attention to the challenge of proposing quality indicators in the broader realm of the educational system. Even recognizing this and considering the limits of this exercise, we will attempt to raise dimensions that are being legitimized in the fields of culture and educational politics.

The Law of Guidelines and Bases for National Education (LDBEN - Lei de Diretrizes e Bases da Educação Nacional), in its Section II - The Principles and Purposes of National Education, Article 3, paragraph IX, states that education should be guaranteed within a quality standard. However, it does not clearly define what this quality standard is, even though its measures point to certain concepts and values. A number of paragraphs of Chapter IV - On Higher Education of the law explain the understanding of quality by characterizing it by a set of obligations. (Brasil, 1996)

We can infer a position that favors the student's development of a power of cultural creation, critical spirit and reflexive thought. Despite the merits of this explanation, quality indicators are highly complex, making it difficult to guide support and evaluation policies that sustain the processes that are to be implemented. Subjective conditions are involved and the complexity of the educational process is assumed, recognizing the multiplicity of factors to be considered. However, they favor a definition of standards that is subject to the political and economic logic of the moment.

These were the assumptions that sustained the interinstitutional study that we describe below. As authors, we were provoked and stimulated by our participation in the project, led by the Southern-Brazilian Network of Higher Education Researchers (Rede Sulbrasileira de Investigadores de Educação Superior - RIES) in the Education Observatory (Observatório da Educação) program, launched by a partnership between the Brazilian educational support agency, CAPES, the Coordination for Continuing Development of Higher Education Personnel (Coordenação de Aperfeiçoamento de Pessoal de Nível Superior) and the Anísio Teixeira National Institute of Educational Research and Studies (Inep) [Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira]. RIES, which is recognized as a Center of Excellence in Science, Technology and Innovation (Pronex/CNPq/Fapergs)¹ is dedicated to shaping higher education as a field for the production of knowledge.

This article presents a summary of the final stage of the Project "Indicadores de Qualidade para a Educação Superior" (Quality Indicators for Higher Education), which included researchers and research groups (RGs) from graduate courses in education from four universities: the Pontifícia Universidade Católica do Rio Grande do Sul (PUCRS), the Universidade Federal do Rio Grande do Sul (UFRGS), the Universidade Federal de Santa Maria (USFM) and the Universidade do Vale do Rio dos Sinos (Unisinos).

The team has worked on this topic since 2006, conducting studies, investigations and academic practices. Its operations were expanded to other universities by involving masters and doctoral students and by promoting post-doctoral internships. The project's initiatives have included the publication of Qualidade da Educação Superior - Observatório da Educação (Quality of Higher Education - Education Observatory), a book series that originated in seminars carried out by RIES.

We emphasize that the methodology for constructing the project was broadly participatory, while preserving the autonomy of each researcher and research group. The project's processes and results were also evaluated by international peers. To facilitate understanding of the studies conducted, we will present the focus of each research group.

QUALITY OF HIGHER EDUCATION AND INTERNATIONALIZATION

The internationalization of academic institutions is a factor of legitimation in the circulation of knowledge and the education of human resources. However, is the direct relationship between university quality and internationalization a myth (Knight, 2011)? Can internationalization help to establish equity without excluding fundamental standards? Or, in other words: can we combine the notion of isomorphic quality with the quality of equity and diversity (Morosini, 2009)?

In the effort to establish quality indicators for internationalization (Morosini, 2012) we adopted a methodology based on states of knowledge, understood as the production of qualified books and scientific articles available online, that have been reviewed by international experts. A first finding is the presence of indicators of internationalization of higher education in the international literature of different developed regions such as the European Union (EU), the United States of America (USA), Canada and Japan. After the selection of the corpus to be analyzed, the annotated bibliography was categorized considering indicators of transnational, national (governmental) and institutional scope.

The transnational scope requires complex negotiations concerning the sovereignty of states. Therefore, the broadest quality indicators for higher education we identify are regional ones, such as those of the EU and South America's Southern Common Market (Mercosur). In the realm of the state, (national), performance indicators that allow comparing quality among higher education institutions are emphasized. These are related to quality assurance and accreditation processes, and to indicators focused on the percentage of foreign students and professors present in international programs (Sarrico, 2010). Among the state indicators, tension is decreasing between those that offer accreditation and those that seek improvement (evaluation). "There is now a reasonable consensus that evaluation and accreditation are two inseparable processes - constituting two sides of the same coin " (Santos, 2010, p. 3).

The institutional scope involves two subdimensions: the networks (international) and the institution itself. The indicators for networks, which are considered to be emergent, give priority to research. The indicators for institutions, considered traditional, prevalent in foreign literature, also involve teaching and particularly management, a reflection of the university modernization paradigm.

The institutional indicators are divided between those that evaluate internationalization of the university in the institution as a whole - conducting a total view - and indicators that evaluate a portion of the university institution - offering a specific view. The examination of these indicators can also lead us to the model of peripheral internationalization or central internationalization, that is, internationalization of the entire institution of higher education (Wit, 2002).

The indicators of the total institutional vision (Ching, Chin, 2012) evaluate the inter-relationship of indicators at different levels: indicators that promote internationalization (core); indicators that articulate internationalization; and ultra-peripheral indicators (frontline). They include: institutional commitments, strategic planning, financing, institutional policies and guidelines, organizational infrastructure and resources, academic and curricular offers, Internet presence, faculty and staff development, foreign students and grantees, study abroad, life on campus and performance evaluation and accountability.

There are also institutional indicators (Reeb-Gruber, 2009), which compose a checklist, evaluating: networks and accreditation, leadership, mobility and exchange, corporate relations and interventions, faculty, students, research and transfer of knowledge, content, curricular pedagogy and facilities and international development and expansion. This methodology also includes internationalization indicators of the university functions (Madera, 2006): teaching, research, extension and organizational development.

Robson (2011, p. 619) considers the concept of a transformative internationalization, which requires a holistic approach in which universities become communities with an international spirit, with the probability of undergoing a "transformation of consciousness" when research presents itself as "reflective, interactive and constructive, distributed and transformative to help students and academic personnel to develop the skills, values and dispositions needed to adopt an internationalist perspective" (idem, p. 614).

There are indicators that regard the Latin-American and Caribbean realities in a perspective of solidarity, as pointed out by Didriksson (1998) about horizontal international cooperation. This outlook highlights:

política de diálogo e intercambio de información para crear un ambiente de colaboración y responsabilidades compartidas; trabajo en redes; aportes de los organismos multilaterales de la región; solidaridad internacional y el mutuo reconocimiento; movilidad académica y estudiantil, en la óptica de la confianza mutua y las equivalencias de calidad; colaboración en todos los niveles para mejorar las condiciones de enseñanza e investigación; capacitación para una cooperación horizontal compartida; reuniones para articular acciones académicas comunes; y participación en foros para garantizar la vigencia del servicio educativo como responsabilidad de toda la sociedad. (idem, p. 23)

In this proposal, internationalization has the potential to aid the establishment of a local identity and social-economic development. It is important to consider a typology that contemplates this local reality. Based on the considerations mentioned, we can utilize some more traditional and standard indicators, but along with these, some others must be included.

The quantitative indicators focus on the functions of the institute of higher education: teaching - which encompasses students, professors, didactic organization (especially the curriculum), regulations, both in undergraduate and graduate courses; research - which includes networks, financing, members, academic production; extension - other functions to be specified according to the mission of the institution; and management - local, international, evaluation, financing.

The qualitative indicators analyze the principles, actors, strategies, relations and the focus of internationalization, cover both South-North internationalization, as well as South-South. They are indicators concerned with the training of the actors (students, faculty and staff) and the construction of networks, based on a policy of dialog, extended towards institutes of higher education with democratic inclusion, participation of society and mutual trust, in the raising and sharing of funds.

It is important to emphasize that we can raise internationalization indicators relative to process and product. The first assess internationalization according to three perspectives of coverage:

1. in spatial terms, the prevalence of total or specific internationalization, occurring in isolation or concurrently on different levels (transnational, national, institutional);

2. in terms of scope, the intensity of internationalization of academic components, applied to aspects such as organization/management, curriculum, mobility, human resources, graduates; and

3. in terms of the system, internationalization promoted continuously (planned and clear procedures) or periodically (intermittent procedures).

Both process and product indicators are based on a vision of the university-society relationship. However, for the product indicators, this vision becomes a guiding principle, defining its ethos. Thus, product indicators point to the presence and intensity of the balance: (i) in spatial terms, between the South-North and South-South directions of internationalization, on different levels (transnational, national and institutional); and (ii) in terms of scope, between internationalization and the other components of higher education, at the level of strategic planning, educational policies, infrastructure, and the volume of academic opportunities at the different levels (transnational, national and institutional).

The proposal of indicators that evaluate the relationship between university internationalization and quality is not clear, and has become even more complex with the flexible and accelerated expansion of higher education in this century. It is necessary (Brandenburg, Wit, 2011, p. 15) to divorce dogmatic and idealist concepts of internationalization and globalization from the comprehension of these processes in their pure meanings, not as objectives themselves, but as means to an end. It is also necessary to reconsider the excessive concern with the instruments and means, and invest more time in questions of principles and results.

QUALITY IN HIGHER EDUCATION MANAGEMENT

Reflecting on higher education management involves addressing conflicts related to formats taken by the institutions that are supported by public policies, internationalization movements, social demands and scientific-technologic development. The question becomes more complex due to the values and logic that pervade the trinomial management-higher education-quality. This is what circumscribes, in this work, the "indicatives"/indicators and categories of quality in higher education management.

Knowledge is constructed in a route of collaboration with and between groups and in dialogs between different forms of knowledge. It draws upon productions and policies anchored in trajectories of scientific knowledge (Tight, 2003), and in Brazilian higher education policies and those for science and technology (S&T), including, because of their driving force, those related to the National System of Evaluation of Higher Education (SINAES). Studies with a metatheoretical perspective are emphasized, such as the states of knowledge, understood as analytic aggregators of academic productions about higher education.

The concept of management is expressive of the practices that, along with the policies, planning and evaluation, compose the management space of higher education. This regards the types of relationships assumed by institutes of higher education and by the Higher Education System regarding the conceptions and practices that express decision making processes, and their actions and logic. It encompasses the conception and purposes of the institutes of higher education, premises about research, teaching, extension and organizational principles. The idea of "indicatives" supplants the normative by making available a textual body through which institutions reveal themselves.

The configuration of quality indicatives and their transposition into indicators contemplated the actions of the research groups,² whose collective work, in successive stages, was submitted to peer evaluation. The indicatives were supported by various forums for conversations with specialists, including:

1. "Jornada Vozes da Comunidade: qualidade e gestão da Universidade" (Voices of the Community Workshop: University quality and management) (2008), Observatory Capes/RIES, promoted by the GEU Network (Group of University Studies - UFRGS/UFPel/UPF) and Grupo de Pesquisa Inovação e Avaliação na Universidade (Research Innovation and Evaluation Group) (InovAval/CNPq/UFRGS);

2. Workshop about "Quality in Higher Education Management" (Network GEU and InovAval Group, in 2010), in which about 30 professors and graduate students discussed the subject, which in turn substantiated the formulation of indicatives; and

3. the 9th International Seminar on Quality in Higher Education: Indicators and Challenges) (2010), with a panel on dimensions of quality in management.

The declarations constructed in this process were summarized by Franco, Afonso and Longhi (2011) in categories of higher education quality indicatives. Two types of principles pervaded the construction: those of methodological quality and those of institutional quality. The first emphasize contextualized knowledge and the second social commitment and the integration of research, teaching and extension. The indicatives guide the formatting of indicators, under the following procedures:

1. review of the indicatives, their categorical location and subcategorical convergences, by members of the team;

2. focal discussion, seeking adjustments, insertion and adaptation as indicators;

3. submission of the indicators to the judgment of the specialists, as part of the qualitative validation of content;

4. organization of the instrument in an ordinal scale; and

5. application of the "Quality Indicators in Higher Education Management Survey" to 60 professors and staff at different types of universities.

The instrument generated in the process combines 100 indicators linked to procedural and final activities, distributed in 7 categories and 31 subcategories. A first glance at the responses, anticipating statistical analyzes, indicates that the categories, in their indicators, mention established institutional policies, programs inductive of actions and some type of measurement of results. In addition, evaluative options and institutional definitions are also present, as well as the university's social commitments, international insertion, the presence of a political decision-making system articulated by the specificity of segments, units, purposes and processes, the construction of university promotional systems and the adoption of information technology. The protagonistic and inductive character of institutes of higher education is also indicated, the participation of the academic and social community, democratic management, space for partnerships, and environmental and institutional sustainability.Chart 1 describes the categories and identifies the subcategories and the number of indicators.

Chart 1
Quality Indicators in higher education management: thematic categories and subcategories

Thematic categories	Subcategories and Number of indicators
1. Quality in Project Management and Institutional Planning (5) – Concerns the identity, mission and commitments of the institution and actions dedicated to institutional sustainability and improvement.(23 indicators and 7 subcategories)	• Institutional project (5)
	• Democratic Management (4)
	• Institutional Sustainability (3), Planning (3),
	Physical Structure (2), Staff and Faculty (4)
	• Informational Structure (2)
2. Financial Management Quality – Concerns sustainability and autonomy in financial-budgetary procedures. (6 indicators and 2 subcategories)	• Financial Sustainability (3)
	• Autonomy in Financial Procedures (3)
3. Quality in Management of education and teaching– Encompasses decision-making in the strategies of education and articulation between levels,knowledges, qualification of teaching staff, career and employability of graduates and their impacts.(15 indicators and 6 subcategories)	• Political Decision-Making
	System about education (6)
	• Articulation of Levels and Knowledge (2)
	• Education/Teaching Strategies (2)
	• Teaching Staff Qualification (2)
	• Career Construction/Employability (2)
	• Economic and Social Impact (1)
4. Research Management Quality – Involves the articulation of knowledge and local and global transformations, the development of technologies that meet demands, the approximation of university and society and the improvement of the quality of life. (19 indicators and 4 subcategories)	• Political Decision-Making System (5)
	• Research Development (5)
	• Research Sustainability (4)
	• Socialization of Knowledge (5)
5. Quality in Management of University Extension and Services – this is realized by establishing compatibility between academic quality and social commitment, and in cultural, technical and scientific development towards a more just society.(20 indicators and 5 subcategories)	• Political Decision-Making System (9)
	• Social Responsibility (1)
	• Social Commitment (3)
	• Inclusionary Policies and Practices (5)
	• Cooperation and Partnerships (2)
6. Quality in Management of Student Services – Encompasses the factors that allow students to enter, remain in and graduate from the university.(7 indicators and 4 subcategories)	• Access/Permanence of the Student in the institute of higher education (3)
	• Student Mobility Programs (1)
	• Inclusionary Policies/Programs (2)
	• Continuing Education Policies – Graduates (1)
7. Quality in th Management of the Evaluation of the University – Concerning the evaluation processes of action of the institute and its political decisionmaking system, linked to the university's evaluation (internal and external, and self-evaluation). (10 indicators and 3 subcategories)	• Institutional Political Decision- Making System of Evaluation (4)
	• Internal/External Institutional Evaluation (5)
	• University Self-Evaluation (1)
Total: 7 categories	31 subcategories and 100 indicators

• Source: Franco, 2011
• Prepared by the author

These actions made it clear that, the direction for the search for a quality university can be found through commitment and collaboration. The interfering tensions show the importance of establishing spaces for sharing the understandings of the academy and the community about what is a university with quality. Quality requires a "permanent exercise of self-reflection" that attends to theoretic, ethic and political references guided towards an evaluative process in which the entire institution can be engaged.

UNDERGRADUATE TEACHING QUALITY

In general, from a universal understanding, the nature of university education has been explained by the concept of the inseparability of teaching, research and extension programs. There is an awareness that this condition is the main element of quality. However, the international literature indicates (Barnett, 2008) that there are few efforts and investigations dedicated to understanding how this inseparability is comprehended by the epistemic communities that form the university.

In an effort to advance the understanding of this phenomenon, we took this subject as the theme of an investigation (Cunha, 2009) that, had eight foci, and sought to consult the literature and academic interlocutors about the concept of inseparability and how it is manifest in academic practice. The foci were: the literature and intellectuals who write about the university; university administrators, senior researchers, good professors (in the opinion of peers and students); students; beginning university professors; those responsible for distance education; and representatives of society who receive graduates of higher education.

The results of the study are polysemous and multiple, but some consistencies are observed. The most evident is that the concept of inseparability of teaching, research and extension is not given deep consideration in the academic environment, confirming the stance taken by international literature. The second is that there are, at least, four understandings of the term: the epistemological view, which focuses on academic capacities; the institutional view focusing on distribution of knowledge; the methodological view that emphasizes the forms of production of knowledge; and the political view that focuses on social impacts. From each one of these is derived an understanding of how this concept influences the quality of higher education. This conceptual imprecision certainly affects the management and institutional policies, and can generate different quality indicators.

From the data of this study, and instigated by the goals of the Observatory Project/RIES, we constructed an instrument using the Likert scale with five variations to identify with an increasing objectivity, the indicators of quality of undergraduate education in the opinion of university professors. The quantitative analysis of this instrument indicated two large matrices that result in undergraduate teaching quality indicators: one based on products and another based on processes. Chart 2 identifies some indicators.

Chart 2
Undergraduate teaching quality indicators: the product perspective

Categories	Quality indicators
1. Institution Quality	• infrastructure involving adequate facilities
	• installed and functional labs for students
	• up-to-date library and with unrestricted access
	• the assumption of representative processes in university administration
	• institutional guide of academic action project
2. Teaching Faculty Quality	• compatible issuance of degrees, including master's and doctorates
	• continuing education and professional development programs for professors in the field of pedagogy
	• structured career and progression
	• work regime focused on teaching, research and extension
3. Student Body	• conditions of student support and permanence (food, housing, transport assistance)
	• programs for insertion into academic life, including offers of compensatory studies
	• national and international exchange programs
	• opportunity to participate in "scientific initiation" programs
	• social insertion programs, including encouragement for participation in solidarity projects
	• cultural action programs promoting student curiosity and valuing their general development

• Source: Cunha, 2011.
• Prepared by the author

When quality is related to processes, it refers to indicators that point to questions of a pedagogic and academic nature, backed by an epistemological concept compatible with the contemporary paradigmatic transition. These indicators are presented in Chart 3.

Chart 3
Undergraduate teaching quality indicators: the process perspective

Categories	Quality indicators
1. Curriculum Quality	• innovative curricula that go beyond the traditional organization of knowledge in a rigid and disciplinary structure
	• explanation of the theory-practice axis in curricular proposals in an articulate and meaningful manner (a presumption of the guidelines)
	• articulation of teaching with research, understanding doubt as the basic principle for the processes of teaching and learning
	• offer of optional curricular activities that broaden the cultural base of education
2. Pedagogical practices	• presence of participatory practices that stimulate student autonomy
	• valorization of autonomous activities that stimulate students' capacity for self-regulation
	• familiarization with technological languages and their impact on the forms of student thought production
	• flexibilization of times/spaces of education, stimulating contact with the world of labor and with culture
	• encouragement of integrated scientific production, through meaningful final course projects that are valued as a production that is integrated to the education
3. Evaluation	• stimulation of comprehensive evaluation, focused on the objectives and dynamic of academic work
	• use of different processes and multiple timeframes in the measurement of learning emphasis on complex learning over memorization
	• appreciation of student authorship and autonomy in the realization of learning

• Source: Cunha, 2011.
• Prepared by the author

The exercise of analyzing the undergraduate teaching quality indicators conducted in the context of the Observatory Project, was a challenge. The decision to not propose universal indicators was intentional and based on a multireferenced understanding of the concept of quality. This condition, however, does not mean that indicators were not formulated. It means that they must be legitimized by specific cultural and political conditions, either by each institution, or by a group of institutions that have common affiliations.

If the formulation of indicators is essential for guiding institutional actions focused on quality, this exercise must consider the educational proposal, the objective conditions and the cultural context of each institution of higher education as a referent. It is a condition that considers universal dimensions and local conditions. Above all, it preserves the freedom that allows education for citizenship, as encouraged by Boaventura de Sousa Santos (2000). This was the intent of the study and it is in this direction that we intend to proceed.

QUALITY OF HIGHER EDUCATION AND PEDAGOGIC INNOVATION

Various moments and distinct works characterize the search for indicators of the relationship between university and society. Multiple relationships require multiple perspectives. Among these perspectives, the best known usually are from within the academy towards the outside. There has thus been a concern for reverting the direction of the process and paying attention to the voices of people outside the academy to understand what they expect of this institution.

Questions regarding the higher education system, graduates, ethics, citizenship, democracy, participation and the pedagogy of the university are a part of the search for knowledge that was the basis for the development of indicators (Bertolin, Leite, 2008; Schaedler, 2010). We were concerned that the indicators would not be directly committed to what is already established as the university's official evaluative parameter, that is, we wanted them to be new indicators or markers.

Thinking about the quality of higher education and the evaluative indicators that could demonstrate it, we thought of the commitment of the university to society, and to the world of work, and the "world of life" in a way that the educational event within four walls, in the teaching and learning spaces, as well as outside the four walls, in other settings and contexts, could be translated into a language marked by pedagogical innovation.

In our perspective, the indicators of innovative pedagogical processes are among those related to the accountability of the university towards society. They express equations that affirm: "what takes place in the classroom relations in this university, students will conduct in their future classes"; "the values presented here are related to the values that tomorrow will be in the circuit of the professional practice of the graduates of this institution". Evaluating teaching, research and extension requires evaluating the degree of innovation of their pedagogy from a non-traditional measure, that breaks from conservative reproduction. In this manner, we focused on pedagogical innovation (Leite; Fernandes, 2011, 2012), and elected to prioritize this theme to carry out the goals of the RIES Project - Stage V - Validation of indicators.

The principal dimensions or categories, their main analyzers, referents, evidences and markers that were researched and validated, are listed in Chart 4.

Chart 4
Higher education quality indicators of pedagogical innovation

Dimensions	Quality indicators
1. Educational MemoryWeb of relationships that involve knowledge as a foundational category of the teaching and learning process	Accounts of experiences brought by individuals representation of their realities and meanings accounts reinterpreted in the university dialectic, knowledge and life.
	Trajectory of professors and students explained and brought for the construction of a common territory marks of differences, web of relationships constructed with stories of students and professors.
	Meaningful bonds among students, and between students and professors, as part of the experience they carry within themselves and life openly faced conflicts.
2.2. ProtagonismConscious participation and autonomy of students and professors in the education process	Exercise of authorship in classroom decisions, elaboration of work, reorganization of groups, studies, text writing.
	Development of argumentative capacity
	Development of decision-making capacity in an independent and justified manner.
	Shared decisions in the pedagogical process of choices at the individual and collective level
	Teaching-learning experiences as non-finished appropriation of reality.
3. TerritorialityOccupation, circulation and appropriation of different formal and non-formal spaces of academic life	Different configurations woven by bonds constructed in and by classroom work
	Decisions and actions of professors not limited to the classroom territory nor by the walls of the University classroom as an intentional teaching and learning space.
	Professors and students expand frontiers social-cultural relations with knowledge and day-to-day life,beyond the physical boundary of the classroom.
4. RuptureEpistemological ruptures with static content of knowledge	Different epistemes for comprehension of knowledge, science and the world.
	Different rationalities beyond the cognitive-instrumental.
	Overcoming knowledge as a static content – "cadaver of information – dead body of knowledge" (Freire,Shor, 1987)
	Forms of teaching-learning that overcome the positivist reproductive model.
	Overcoming individualism and understanding the social construction of knowledge.
5. Historicity of KnowledgeModes of production of knowledge and their relations with socio-cultural, political space-time and political structures of society	Manifestation of the break with the mythical belief of the superiority of scientific knowledge
	Manifestation of the reception of different interpretations of reality
	Recognition of the intentions and interests that forge the history of knowledge.
	Recognition of values involved in the production of knowledge,in different times, circumstances and spaces of social praxis
	Reflective understanding of the dated and situated knowledge individual and collective construction of humanity
6. PedagogicalDemocracyShared pedagogical relations and democratic contract for teaching learning decisions	Relationship of trust built on respectful attitudes, reception within the bounds of possible human relations
	Relations permeated by affection and availability for dialog
	Condition established in conjunction by professors and students-students
	Condition built on teaching-learning processes that move between the individual and the social

• Source: Leite; Fernandes, 2011.
• Prepared by the author

We carried out a pilot study to validate the dimensions and markers using an instrument based on this chart. The instrument was sent to 130 professors who are familiar with and scholars within the field of pedagogical innovation. Of these, 86 replied to the invitation. The respondents were selected in a deliberate manner, they are professors and researchers from 20 universities in Brazil and abroad. The instrument was prepared with support of the Google Docs tool.

The answers indicated that the respondents both confirmed and denied some of the evidences and markers that were presented in the validation instrument. Significance tests are yet to be conducted. At the time this article was written, the percentage of items with which the respondents fully agreed, which had a value five on the Likert scale of the instrument, or fully rejected, which had a value one of the scale, were recorded.

The Educational Pedagogical Memory dimension was accepted by 70% of the repliers according to the referent: "the experiences brought by professors and their students to the teaching-learning processes." The Protagonism dimension appears with a score dispersed among the five items of the scale, with a complete disagreement of 10-14% of the repliers. The Territoriality dimension was checked by 77.35% of the repliers in the evidence or marker that suggests "in the Territory, professors and students expand the borders of knowledge (...)". The Rupture dimension had answers checked as agree completely by the repliers. The rates ranged between 77.25% and 87.15% and confirmed all of the evidence or markers suggested by the instrument for this dimension. The Historicity of Knowledge was checked by 80.75% of the repliers by the analyzer proposed; its evidence or marker was accepted by 77.75% of the repliers as "The Historicity of Knowledge reveals the values implied in the knowledge production processes at different times, circumstances and spaces of social praxis". The Pedagogical Democracy dimension was checked by 69.75% of repliers by the analyzers proposed in the instrument and by 79.1% of repliers by the principal evidence or marker " Pedagogical Democracy is an ethical condition between the personal and the social constructed in conjunction by professors and students-students ".

The validation of the pedagogical innovation markers reveals certain trends. One of the issues raised by the validation is the rejection of the dimension of Protagonism as a part of pedagogical innovation. On the other hand, the respondents confirmed the understanding that the dimensions of Memory, Rupture and Pedagogical Democracy are categories that help understand pedagogical innovation.

Our conviction is that by evaluating the use of indicators or markers, we are also going beyond the borders of the unconventional with the sole purpose of better organizing the evaluative processes that reveal the quality of what takes place in our universities.

QUALITY OF THE TRAINING AND DEVELOPMENT OF PROFESSORS

In the search for indicators of professional development and education of professors, we contemplated the following investigative question: what is the understanding of professors in higher education about the reality of professional education and development in their institutions, having quality indicators as references?

We carried out a descriptive-explanatory study of a quantitative and-qualitative nature with 100 active higher education professors, who agreed to contribute to this study. The instrument used was an educational survey, utilizing a Likert-type scale. The data management and analysis was supported by the scientific application software SPSS. The following categories were identified: teaching in higher education, institutional/teaching ambience, learning and interaction of the professor, diversity and inclusion (Chart 5). The dialog with the results was based on a comparison between the theoretical perspective and the analytical categories that originated in the study.

Chart 5
Higher education quality indicators in professional education and development

Categories	Quality indicators
1. University Teaching	The professors engaged in the educational process for which they are responsible meet the demands of the academic and professional fields, in which they perform.
	The professors, aware of teaching as a profession, accept the challenges of new ways of educating in an ever-changing world.
	The professors display genuine understanding of the knowledge, skills and competencies, related to their respective professional areas.
	The professors from various areas of education and their students recombine experiences and knowledges for an autonomous professional performance.
	University teaching involves spaces and times of reflection and reconstruction of personal and professional trajectories.
	The way in which the teaching trajectory is experienced determines personal, group and institutional purposes, reflecting on the quality of the formative process.
2. Institutional/ Teaching Ambience	The institution invests in the construction of university pedagogies,contemplating individual and collective reflection processes.
	The institution promotes policies and spaces for the qualification of professors,favoring access to master's and doctoral degrees in the specific field.
	The institution promotes policies and spaces of education and development for the teaching professional, favoring well-being and professional achievement.
	The institutional organization determines how the teaching trajectory is experienced, reflecting on the quality of the educational process.
	The creation of interactive networks favors dialog among peers, encouraging innovative pedagogical practices.
3. Professors' Learning and Interactivity	Professors learn through collaboration and reflection among peers, with repercussions for professional autonomy.
	The generative professors develop themselves and their group of students, in dialog with others from the academic world.
	The professors give priority to interpersonal relations with their students as components of the teaching and learning processes.
	The professors prioritize interpersonal relations as components of personal and professional education and development.
	The processes of individual and group reflection favor awareness about the educational practice promoting professional development of professors.
4. Inclusion and Diversity	Institutional policies for the professional development of professors consider the requirements of the diversity of students, contexts and the world of labor.
	The higher education institutions are prepared to meet the demands of inclusion of diversity, accessibility and permanence of students in the academic context.
	Professors are prepared to meet the demands of inclusion and diversity,accessibilit , and the permanence of students in the academic context.

• Source: Isaia, 2011
• Prepared by the author

The category university teaching involves a complex and unique activity, which encompasses a multiplicity of knowledges, skills and attitudes; it is an interactive job; which requires the mastery of a specialized content and is oriented towards education of a profession; it includes the interaction of teaching, research, extension and management. The findings show that the participating professors partially agreed that professors engaged in the educational process for which they are responsible meet the demands of the academic and professional fields in which they act. They also value the genuine understanding of the knowledge, know-how, skills and competencies related to their respective professional areas. In terms of items in which they were in full agreement, they indicated that they recognize teaching as a profession, accepting the challenges of new ways of educating in an ever-changing world.

The category institutional/teaching ambience is characterized as an internal/external dialectic unit that expresses the impact of the conditions in which teaching is exercised, involving experiences that relate to a perception of the academic environment as favorable or limiting. It also highlights the levels of involvement in the teaching profession, from entry into higher education to the prospection of the coming years. (Maciel, 2009; Maciel; Isaia; Bolzan, 2009). The professors fully agree that the institution needs to invest in the construction of university pedagogies that consider processes of individual and collective reflection. Furthermore, they partially agree that the institution should promote policies and spaces for the training of professors, favoring access to master's and doctoral programs in the specific fields of teaching, and considering professional well-being and achievement. In this same sense, they reinforce the idea that the institutional organization determines the way in which the professor's trajectory is experienced. It reflects the quality of their educational process, thus determining personal, group and institutional purposes.

The category learning and interaction of the professor implies the interpersonal/intrapersonal process, involving the appropriation of knowledge, know-how and productions specific to higher education, which are linked to the teaching activity and undertaken in different professional domains. It is understood from a perspective of systematic inter and intrapersonal reflection, constituting a component intrinsic to the process of teaching, learning, education and, consequently, professional development. (Bolzan; Isaia, 2008, 2010; Isaia, 2008).

Most of the respondents agreed that the professor's learning is made effective through collaboration and reflection among peers, leading to professional autonomy. Another fully accepted assertion is that generative professors develop themselves and their group of students in dialog with others from the academic world. Not only do they execute teaching activities, but they feel committed to carrying them out in the best possible way (Isaia, 2006). In agreeing partially, professors demonstrate that they give priority to interpersonal relations as indispensable components important to teaching and learning and to professional and personal education and development. It is therefore understood that interaction permeates the teaching and learning processes, giving them the necessary foundation for their constitution.

The category diversity and inclusion encompasses: clear inclusionary policies; an institutional environment favorable to the academic inclusion of socially discriminated groups and the importance of that professors are prepared to interact with diversity. The professors who participated in the study partially agree that the institutional policies for professional development of professors consider the needs of the diversity of students, contexts and of the world of labor and that the institutions of higher education are prepared to meet the demands of inclusion of diversity, accessibility and the permanence of students in the academic context. They also indicate the need for professors to be prepared to meet these demands. It can be seen that, in partially agreeing, they point to the fact that in their institutions these questions are still in a process of acquisition/construction, with greater investment necessary for them to be effectively accomplished.

The study was characterized by the initial exploration of our findings, indicating the need for institutions of higher education to recognize professors as the main protagonists in the process of pedagogic innovation. It identified the importance of supporting initiatives for learning by professors, giving priority to physical, material and human conditions so that diversity and inclusion become an indispensable part of pedagogical projects and educational practices.

SUMMARIZING...

The task of producing indicators led to developments on two interconnected planes: that of conceptual problematizations and that of methodologies. On the conceptual plane, it is undeniable that the indicators maintain objectivity, for they are anchored in concrete institutional situations and in their regulatory frameworks, resulting in the inclusion of indicators constructed in long and careful study processes about aspects of higher education materialized in documents and other products. However, they also have a subjective facet, that results from the values and experiences present in the movement between the global, local and the individual. They thus lead to new possibilities for configurations of quality for higher education, around their many specificities, thus serving the Brazilian reality.

In the plane of methodology, we can affirm that during the study, we learned to do by doing. We listened to colleagues from other countries and universities, in eleven international seminars and produced theses and dissertations, books and articles. The question of quality was exhaustively reviewed. The publications from the research, some online, recognize that the concepts involved in this debate are multidimensional and complex and, therefore require investments and constant reflection. In the understanding of quality, in addition to the production and reflection nurtured by the group research, the hearing of external demandsfrom the globalized world was enhanced by establishing dialog with research groups from Brazil and those from the United States, Mexico, England, Portugal, Spain, Argentina and Uruguay who were present in the seminars and brought their analytical views.

In both planes, we have concluded that dialog encourages and strengthens the production of quality, understanding that it was the problematizing consequences of dialog that favored the opportunity to produce syntheses. Paraphrasing Santos (2000, p. 76), we reaffirm our learning that dialog is the foundation upon which generations are educated and reeducated. The freer the work is and the more it is shared, the more we will educate towards citizenship.

We study indicators of university internationalization, management of higher education, undergraduate teaching, pedagogical innovation and professional education and development of professors, in relation to the understanding of its quality.

We reaffirm that internationalization is at the core of the university, and is a factor of legitimation of the circulation of knowledge and the education of human resources. The proposal of indicators that evaluate the relation between university internationalization and quality is not clear, and becomes more complex with the flexible and accelerated expansion of transnational economic negotiations to consider higher education as a service. In the plane of the state, indicators are still rare, while the process of internationalization begins within the country. It is on the plane of institutions of higher education that the indicators have greater presence, and are promoted as symbols of quality.

Regarding the management of higher education, two types of principles pervaded the construction of the indicators: those of methodological quality and those of institutional quality. The first group emphasized contextualized knowledge and the second social commitment and the integration of research, teaching and extension. Also present are evaluative options and institutional definitions, social commitments, international insertion, the protagonistic and inductive character of institutes of higher education, the participation of the social and academic community, democratic management, partnerships and environmental and institutional sustainability.

Two large matrixes organize the indicators of the quality of undergraduate education: one based on products and another on processes, revealing that objective and subjective dimensions are present in teaching and learning. Both dimensions need to be articulated to attend the quality standard.

The validation of pedagogical innovation markers revealed the rejection of the dimension of student protagonism as a referent of innovation and confirmed the dimensions, memory, rupture and pedagogical democracy as categories that help evaluate pedagogical innovation. This condition raised tensions with the institutional management perspective, which understands protagonism as a value and an indicator of management quality.

The dimension of professional development of professors revealed the importance of supporting initiatives for learning by professors, prioritizing the physical, material and human conditions so that diversity and inclusion become an indispensable part of the pedagogical projects of courses and educational practices.

We consider that the indicators presented in Chart 6 are an alternative to what is proposed as measurements by the known international rankings. Different from these, their conception originated in everyday university practices. This condition has rarely been approached in the literature that organizes indicators, especially because they are subjective issues, whose restricted objectivity makes them difficult to quantify.

Chart 6
General higher education quality indicator

Categories	Subcategories
Internationalization	Transnational	Process (range)
	National	Product (balance)
	Institutional
Management	Institutional Project and Planning
	Financial
	Education and Teaching
	Research
	Extension and Services
	Student Service
	Evaluation
Teaching	Institution
	Faculty Products
	Student Body
	Curriculum
	Pedagogical Practices Processes
	Evaluation
Pedagogical Innovation	Educational Memory
	Protagonism
	Territoriality
	Rupture
	Historicity of Knowledge
	Pedagogical Democracy
	University Teaching
Faculty Professional Development	Institutional/Teaching AmbienceFaculty
	Learning and Interactivity
	Inclusion and Diversity

• Source: The authors, 2013.
• Prepared by the author

Universal indicators were intentionally not proposed, based on a multi-referenced understanding of the concept of quality. This condition, however, does not signify the inexistence of the formulation of indicators. It seeks to denote that they must be legitimized by the specific cultural and political conditions, be it from each institution, or a group with common affiliations. If the formulation of indicators becomes essential to guiding institutional actions aimed at quality, it is essential that the references for this exercise are the educational proposal, the objective conditions and the cultural context of each institution of higher education. This is a conjugation that considers global and local dimensions. It preserves, above all, the freedom that allows us to educate towards citizenship, as Boaventura de Sousa Santos (2000) encourages. This was the intent of this study and it was in this direction that we seek to advance.

• 1
  The Program of Support to Centers of Excellence (Pronex – Programa de Apoio a Núcleos de Excelência) results from a partnership between the National Council of Scientific and Technologic Development (CNPq – Conselho Nacional de Desenvolvimento Científico e Tecnológico) and Rio Grande do Sul’s Foundation for Research Support (Fapergs – Fundação de Amparo à Pesquisa do Estado do Rio Grande do Sul).
• 2
  Research Groups linked to the Graduate Program in Education/UFRGS, Federal University at Pelotas (UFPel) and University of Passo Fundo (UPF).

REFERÊNCIAS

• Barnett, R. (Ed.). Para una transformación de la universidad: nuevas relaciones entre investigación, saber y docencia. Barcelona: Octaedro, 2008.
• Bertolin, J. G.; Leite, D. Quality evaluation of the Brazilian higher education system: relevance, diversity, equity and effectiveness. Quality in Higher Education, London: Taylor & Francis, v. 14, n. 2, p. 121-133, jul. 2008.
• Bolzan, D. P. V.; Isaia, S. M. A. Aprendizagem colaborativa: processos formativos em construção. In: Seminário de Pesquisa em Educação da Região Sul, 7., 2008. Itajaí. Anais... Itajaí: Univali-Itajaí, 2008.
• ______.; ______. Pedagogia universitária e aprendizagem docente: relações e novos sentidos da professoralidade. Revista Diálogo Educacional, Curitiba: PUC-PR, v. 10, n. 29, p. 13-26, 2010.
• Brandenburg, U.; Wit, H. The end of internationalization. International Higher Education, Boston: CIHE, n. 62, p. 15-16, 2011. Disponível em: <http://www.bc.edu/research/cihe/ihe/issues/2011.html>. Acesso em: 25 jan. 2012.
  » http://www.bc.edu/research/cihe/ihe/issues/2011.html
• Brasil. Lei n. 9.394, de 20 de dezembro de 1996. Lei de Diretrizes e Bases da Educação Nacional (LDBEN). Estabelece as Diretrizes e Bases da Educação Nacional. Diário Oficial [da] República Federativa do Brasil, Brasília, DF, 20 dez. 1996. Disponível em: <http://www.planalto.gov.br/ccivil_03/leis/L9394.htm>. Acesso em: 30 ago. 2012.
  » http://www.planalto.gov.br/ccivil_03/leis/L9394.htm
• Ching, G. S.; Chin, J. M. S. Managing higher education institution internationalization: contemporary efforts of a university in Taiwan. International Journal of Research Studies in Management, Malabon: Consortia Academia, v. 1, n. 1, p. 3-16, Apr. 2012.
• Cunha, M. I. (Coord.). Qualidade do ensino de graduação: a relação entre ensino, pesquisa e o desenvolvimento profissional docente. São Leopoldo: Conselho Nacional de Desenvolvimento Científico e Tecnológico; Universidade do Vale do Rio dos Sinos; Programa de Pós-Graduação em Educação, 2009. Projeto de Pesquisa.
• Didriksson, A. Reformulación de la cooperación internacional en la educación superior de América Latina y el Caribe. Caracas: IESALC; UNESCO, 1998.
• Franco, M. E.; Afonso, M. R.; Longhi, S. M. Indicativos de qualidade na gestão da universidade: dimensões e critérios em questão. In: Franco, M. E. D. P.; Morosini, M. C. (Orgs.). Qualidade na educação superior: dimensões e indicadores. Porto Alegre: EDIPUCRS, 2011. p. 292-318. (Série Qualidade da Educação Superior, v. 4). Disponível em: <http://ebooks.pucrs.br/EDIPUCRS/qualidadedaeducacaosuperior4.pdf>. Acesso em: 30 ago. 2012.
  » http://ebooks.pucrs.br/EDIPUCRS/qualidadedaeducacaosuperior4.pdf
• Freire, P.; Shor, I. Medo e ousadia: o cotidiano do professor. Rio de Janeiro: Paz e Terra, 1987.
• Isaia, S. M. A. Verbetes. In: Cunha, M. I.; Isaia, S. M. A. Professor da educação superior. In: Morosini, M. (Ed.). Enciclopédia de pedagogia universitária: glossário. Brasília: INEP; MEC, 2006. v. 2, p. 375-377.
• ______. Aprendizagem docente: sua compreensão a partir das narrativas de professores. In: Traversini, C.; Eggert, E.; Peres, E.; Bonin, I. (Orgs.). Trajetórias e processos de ensinar e aprender: práticas e didáticas. Porto Alegre: EDIPUCRS, 2008. p. 618-635.
• Knight, J. Five myths about internationalization. International Higher Education, Boston: CIHE, n. 62, p. 14-15, 2011. Disponível em: <http://www.bc.edu/research/cihe/ihe/issues/2011.html>. Acesso em: 25 jan. 2012.
  » http://www.bc.edu/research/cihe/ihe/issues/2011.html
• Leite, D.; Fernandes, C. M. B. Indicators of pedagogical innovation in the university/Indicadores de inovação pedagógica na universidade. In: Morosini, M. C. (Org.). Quality in higher education: reflections and investigative practices/Qualidade na educação superior: reflexões e práticas investigativas. Porto Alegre: EDIPUCRS, 2011. p. 303-312. (Série Qualidade da Educação Superior, v. 3). Disponível em: <http://ebooks.pucrs.br/EDIPUCRS/qualidadedaeducacaosuperior3.pdf>. Acesso em: 25 abr. 2012.
  » http://ebooks.pucrs.br/EDIPUCRS/qualidadedaeducacaosuperior3.pdf
• ______.; ______. (Orgs.). Qualidade da educação superior: avaliação e implicações para o futuro da universidade. Porto Alegre: EDIPUCRS, 2012. (Série Qualidade da Educação Superior, v. 6)
• Machado, G. R. Perfil do egresso da Universidade Federal do Rio Grande do Sul. 2010. 330f. Tese (Doutorado em Educação) - Universidade Federal do Rio Grande do Sul, Porto Alegre, 2010.
• Maciel, A. M. R. O processo formativo do professor no ensino superior: em busca de uma ambiência (trans)formativa. In: Isaia, S. M. A.; Bolzan, D. P. V.; Maciel, A. M. R. (Orgs.). Pedagogia universitária: tecendo redes sobre a educação superior. Santa Maria: Editora UFSM, 2009. p. 63-77.
• ______.; Isaia, S. M. A.; Bolzan, D. P. V. Trajetórias formativas de professores uni versitários: repercussões da ambiência no desenvolvimento profissional docente. In: Reunião Anual da ANPEd, 32., 2009. Caxambu. Anais... Rio de Janeiro: ANPEd, 2009.
• Madera, I. Un sistema de gestión de la internacionalización y la cooperación en la universidad APEC como eje transversal de la dinámica institucional. In: Congreso Internacional de Educación Superior: Universidad 2006, 5., 2006; Simposio de Internacionalización de la Educación Superior, 2., 2006. Anales.. Havana. Havana: MES; Universidades de la República de Cuba; UNESCO, 2006.
• Morosini, M. C. Qualidade na educação superior: tendências do século. Estudos em Avaliação Educacional, São Paulo: Fundação Carlos Chagas, v. 20, n. 43, p. 165-186, maio/ago. 2009.
• ______. Qualidade da educação superior e internacionalização: estado de conhecimento sobre indicadores. In: Cunha, M. I.; Broilo, C. (Orgs.). Qualidade na educação superior: grupos investigativos internacionais em diálogo. São Paulo: Junqueira & Marin, 2012.
• Reeb-Gruber, S. Checklists program internationalization Haarlem: Inholland University, 2009. Disponível em: <http://www.nuffic.nl/international-organizations/docs/quality-assurance-and-internationalization/checklistis-program-internationalization-english-21-07-2009.pdf>. Acesso em: 1 fev. 2012.
  » http://www.nuffic.nl/international-organizations/docs/quality-assurance-and-internationalization/checklistis-program-internationalization-english-21-07-2009.pdf
• Rios, T. A. Compreender e ensinar. Por uma docência da melhor qualidade. São Paulo: Cortez, 2001.
• Robson, S. Internationalization: a transformative agenda for higher education? Teachers and Teaching, London: Taylor & Francis, v. 17, n. 6, p. 619-630, 2011. Disponível em: <http://dx.doi.org/10.1080/13540602.2011.625116>. Acesso em: 23 jan. 2012.
  » http://dx.doi.org/10.1080/13540602.2011.625116
• Santos, B. S. Crítica da razão indolente: contra o desperdício da experiência. São Paulo: Cortez, 2000.
• Santos, M. Por uma outra globalização: do pensamento único à consciência universal. Rio de Janeiro; São Paulo: Record, 2000.
• Santos, S. M. Análise comparativa dos processos europeus para a avaliação e certificação de sistemas internos de garantia da qualidade. Lisboa: A3ES, 2010. Disponível em: <http://si.esgt.ipsantarem.pt/esgt_si/web_gessi_docs.downloadfile?p_name=F277086999/AnáliseComparativadosProcessosEuropeusparaaAvaliaçãoeCertificação.pdf>. Acesso em: 24 jan. 2012.
  » http://si.esgt.ipsantarem.pt/esgt_si/web_gessi_docs.downloadfile?p_name=F277086999/AnáliseComparativadosProcessosEuropeusparaaAvaliaçãoeCertificação.pdf
• Sarrico, C. Indicadores de desempenho para apoiar os processos de avaliação e acreditação de cursos Lisboa: GEA; A3ES, 2010. Disponível em: <http://si.esgt.ipsantarem.pt/esgt_si/web_gessi_docs.download_file?p_name=F326257361/Estudo_IndicadoresDesempenho_CláudiaSarrico.pdf>. Acesso em: 25 jan. 2012.
  » http://si.esgt.ipsantarem.pt/esgt_si/web_gessi_docs.download_file?p_name=F326257361/Estudo_IndicadoresDesempenho_CláudiaSarrico.pdf
• Schaedler, L. I. Por um plano estético da avaliação nas residências multiprofissionais: construindo abordagens avaliativas SUS-implicadas. 2010. 184f. Tese (Doutorado em Educação) - Universidade Federal do Rio Grande do Sul, Porto Alegre, 2010.
• Tight, M. Researching higher education. London; Berkshire: Society for Research into Higher Education; Open University Press; McGraw - Hill Education, 2003.
• Wit, H. Internationalization of higher education in the United States of America and Europe. A historical, comparative and conceptual analysis. Westport: Greenwood Press, 2002.

Publication Dates

• Publication in this collection
  Jan-Mar 2016

History

• Received
  Apr 2013
• Accepted
  May 2014