The effect of policies for filling school principal positions on school management in Brazil

Abstract

Leadership by a school principal is one of the key factors for ensuring an educational environment conducive to the teaching-learning process. Nevertheless, only a handful of studies have analyzed the determinants of principals’ performance. This study evaluated the potential effect of principal selection policies on teacher perceptions of principal leadership and school climate. The factors were created based on responses to the National Basic Education Assessment System (Sistema Nacional de Avaliação da Educação Básica - Saeb) contextual questionnaires applied in 2013 and 2015. The legislation for filling school principal positions in the 26 states and the Federal District was also analyzed. The results showed a statistically significant difference between the group of schools that carried out “civil service examination only” and “selection process and election” compared to the group that adopted the “appointment only” modality. However, the high variation observed in the climate and leadership coefficients between the federative entities reveals that the local context affects teacher perceptions of these variables. This demonstrates that in addition to the selection process, it is important to consider each locality’s characteristics to select professionals with competencies and skills appropriate to the position’s challenges, which may contribute to better educational outcomes for students.

Keywords:
educational policies; school management; filling school principal positions; leadership; school climate

Resumo

A liderança do diretor escolar constitui um dos fatores centrais para garantir um ambiente educacional adequado ao processo de ensino-aprendizagem. No entanto, ainda há poucos estudos que analisam os determinantes da atuação do diretor. O presente estudo avaliou o potencial efeito das políticas de seleção de diretores sobre a percepção do corpo docente em relação à liderança e ao clima escolar. Os fatores foram criados a partir das respostas aos questionários de contexto aplicados do Sistema Nacional de Avaliação da Educação Básica (Saeb) de 2013 e de 2015. Também foram analisadas as legislações de provimento ao cargo de diretor nos 26 estados e no Distrito Federal. Os resultados apontaram uma diferença estatisticamente significante entre o grupo de escolas que realizou o “concurso público apenas” e o “processo seletivo e eleição”, em relação ao grupo que adotou a modalidade de “indicação apenas”. Todavia, a alta variação observada nos coeficientes de clima e de liderança entre os entes federados evidencia que o contexto local afeta a percepção do corpo docente sobre estas variáveis. Isso demonstra que, além do processo seletivo, é importante considerar as características de cada localidade para a seleção dos profissionais com as competências e as habilidades apropriadas aos desafios do cargo, que possam vir a contribuir com melhores resultados educacionais dos estudantes.

Palavras-chave:
políticas educacionais; gestão escolar; provimento ao cargo de diretor; liderança; clima escolar

Resumen

El liderazgo del director escolar es uno de los factores centrales para asegurar un ambiente educativo apropiado al proceso de enseñanza-aprendizaje. Sin embargo, aún son pocos los estudios que analizan los determinantes del desempeño del director. El presente estudio evaluó el efecto potencial de las políticas de selección de directores sobre la percepción del profesorado en relación con el liderazgo y el clima escolar. Los factores se crearon a partir de las respuestas a los cuestionarios contextuales aplicados por el Sistema Nacional de Evaluación de la Educación Básica (Saeb) de 2013 y 2015. También se analizó la legislación que reglamenta el cargo de director en los 26 estados brasileños y el Distrito Federal. Los resultados mostraron una diferencia estadísticamente significativa entre el grupo de escuelas que realizó el “concurso público” y el “proceso de selección y elección”, en relación con el grupo que adoptó la modalidad de “nominación política”. Sin embargo, la alta variación observada en los coeficientes de clima y liderazgo entre las entidades federativas evidencia que el contexto local afecta la percepción del cuerpo docente sobre estas variables. Por lo tanto, además del proceso de selección, es importante considerar las características de cada localidad para la selección de profesionales con las competencias y habilidades adecuadas a los desafíos del puesto, que pueden contribuir a mejores resultados educativos para los estudiantes.

Palabras clave:
políticas educativas; gestión escolar; selección de directores; liderazgo; clima escolar

1. INTRODUCTION

The objective of this study is to analyze whether the different principal selection modalities at state public basic education schools affect teacher perceptions of principal leadership and the school climate where they work. Both are considered key factors of effective schools, in order to ensure an educational environment conducive to cooperation and pedagogical goals (Abrucio, 2010; Alves & Franco, 2008; Teddlie & Reynolds, 2000). The selection process was chosen as the object of analysis because it makes it possible — when it is well designed — to identify whether candidates have the competencies and skills necessary to perform such a complex role. Microdata was obtained from National Basic Education Assessment System (Sistema Nacional de Avaliação da Educação Básica - Saeb) contextual questionnaires applied in 2013 and 2015, and the legislation for filling principal positions in the 26 states and the Federal District was also consulted.

The method used to create the principal leadership and school climate factors was Samejima’s (1969) Item Response Theory (IRT), adapting the constructs developed by Oliveira and Waldhelm (2016). The related questions on the Saeb were consolidated into one indicator for leadership and another for school climate, which represent the average teacher perception per school. Next, a multiple linear regression model (Wooldridge, 2011) was applied to determine whether “the method for filling principal positions is associated with teacher perceptions of principal leadership and school climate”. The effect observed in the two models confirmed the study’s initial hypothesis: there is a difference in perceptions of leadership and climate between schools that adopted different principal selection modalities.

In the analysis of the legal frameworks for filling principal positions, it was possible to observe which modalities are regulated by each Federated Unit (FU) as well as their correspondence with the spontaneous responses on the Saeb questionnaires. A great disparity was found between the formulation (evaluated through legislation) and implementation processes (evaluated through microdata) of these policies in each locality, particularly due to the high incidence of the “appointment only” criterion, even though other selection methods have been regulated. Furthermore, the two analysis models suggest that adopting technical and/or participatory criteria for school principal selection enhances the perception of leadership and climate, consistent with the provisions of the National Education Plan of Brazil (Plano Nacional de Educação - PNE) (PNE, 2014). In the following section, the theoretical framework will be presented, with an emphasis on the conceptions of school management, leadership, climate, principal selection, and the relationship between those variables. Next, the methodological procedures of the document research and the two models developed for principal leadership and school climate will be described, followed by an analysis of the results and conclusions about the object of study.

2. THEORETICAL FRAMEWORK

Principals play a significant role in both learning and the educational environment (Abrucio, 2010). Although academic performance is the central criterion of school quality, it is vital that the management model acts to improve the organizational process as a whole, creating favorable conditions for all the actors in this system. The British and American literature use the concept of School Effectiveness Research to identify the factors associated with better academic outcomes for students, including internal school policies and practices (Soares, 2004). To this end, according to Alves and Franco (2008), it is necessary to control for the external influence of the socioeconomic and cultural level of their families, including the educational path (age/grade mismatch) and the type of school in which they studied (public or private).

The importance of principal background to achieving good learning outcomes has been evident since the earliest studies on school effects (Teddlie & Reynolds, 2000). As mentioned in the introduction, we consider two important aspects of that background in our analysis in this article: leadership and its influence on school climate. Accordingly, we will now discuss studies that analyze the effect of the principal selection process on learning outcomes and, subsequently, the relationship between the selection modality chosen and teacher perceptions of leadership and school climate, with the latter as the focus of our study.

2.1 The relationship between the principal selection process and school performance

To justify the relevance of this study, we will begin the theoretical discussion by presenting a number of articles that have analyzed the relationship between the methods for filling principal positions and school performance. Oliveira and Carvalho (2015) argue that the high incidence of the “appointment only” principal selection criterion is part of the Brazilian patrimonial tradition and runs the risk of disregarding educational or management skills. In their analyses, this modality accounted for approximately 20% of the negative change in the mathematics scores of 5^th year students on three editions of the Prova Brasil. Pereda, Lucchesi, Mendes and Bresolin (2015), in turn, observed higher mathematics scores from 5^th and 9^th year students on the 2007 and 2011 Prova Brasil, in schools that carried out “selection process and election,” “civil service examination only,” and “selection process only,” compared to those with “appointment only.”

Lück (2011) corroborates the critical view of selecting principals through “appointment only,” to the detriment of other competencies essential to student education and learning, consistent with the literature. According to the Brazilian Institute of Geography and Statistics (Instituto Brasileiro de Geografia e Estatística [IBGE], 2015), this modality is the least recommended for advancing with the PNE (2014), due to common clientelistic practices (individual privileges in exchange for votes), lacking any consultation with the school community or assurance of the candidates’ technical qualifications. Both authors reported similar problems with the “election only” criterion, whose risks are similar to the electoral process. However, the proponents of this modality argue that it is possible to involve different segments in the pursuit of more effective school leadership, a change in power, an improved organizational climate and transparent relationships (Maia & Manfio, 2010).

“Civil service examination only” is considered more technical and transparent according to Lück (2011), but it limits the possibility of replacements for the position, if necessary, as well as the participation of the school community. “Selection process only” and “selection process and election,” in turn, enable a better verification of the candidates’ competencies and skills through interviews, a certification examination, proof of knowledge, proof of qualifications, a letter of intent, and management plans for the evaluation of the school community. The latter is common in countries with high-performing education systems and allows for greater accountability in management. The advantages of the “selection process and election” mixed modality are also indicated by Ghanem (1995) and Libâneo (2001).

Although there is an apparent consensus on the negative consequences of “appointment only,” Gremaud, Pazello and Maluf (2015) did not observe a relevant impact of this method on student educational performance, despite the potential effect of that option on principals’ managerial skills. The models analyzed showed little difference in the average mathematics scores between schools that changed their selection modalities, based on microdata from the 2007 to 2011 Prova Brasil.

Pereda et al. (2015) concluded that the selection process impacts learning and/or academic performance based on the background of the principal selected. The principal’s time in the position at the same school for between 05 and 15 years and the promotion of professional development for teachers had more consistent results in different analyses. Accordingly, most of the studies cited support the argument that in the Brazilian case, the principal selection method influences school performance.

2.2 The relationship between perceptions of leadership and school climate and the school principal selection process

In the school management literature, principal leadership and school climate—as a cooperative environment focused on the teaching-learning process—are prominent among the main characteristics for differentiating student performance (Levine & Lezotte, 1990; Rutter, Maughan, Mortimore, Ouston & Smith, 2008; Sammons, 2008; Teddlie & Reynolds, 2000).

Leadership is synonymous with an effective school, and a good principal is capable of exercising leadership in a firm, objective, stable, appropriate and academic way, in collaboration with his or her staff (Mortimore, Sammons & Ecob, 1988; Rutter et al., 2008; Sammons, 2008; Teddlie & Stringfield, 1993). Leadership accounted for 25% of the total variance in student learning, when intraschool factors were analyzed. When compared with interschool differences, the percentage was 3% to 5% (Leithwood & Jantzi, 2009). In Brazil, principal leadership is also one of the common aspects associated with good school performance (Abrucio, 2010; Alves et al., 2015; Alves & Franco, 2008; Lück, 2009).

Alves et al. (2015) used three leadership constructs: administrative, pedagogical, and participative. The results exhibited high consistency in relation to school performance. The first emphasizes financial resources for the operation and maintenance of the school, as well as problems that affect its routine. The second refers to the principal’s ability to design and implement shared goals and projects to improve education, encourage teacher development and legitimize the pedagogical team. The third includes the principal selection method, the composition of the school council, the frequency of class council meetings and the way in which the political pedagogical project is developed.

According to Oliveira (2015), discussions about school leadership typologies are frequent, although there is no consensus. In the United States, transformational leadership (human and organizational development) and instructional leadership (interaction with teachers to guide curriculum compliance and learning) are particularly prominent. Marks and Printy (2003) proposed to aggregate the characteristics of the principal’s pedagogical and relational management, i.e., integrated leadership. This concept was corroborated by Oliveira and Carvalho (2015) and Urick and Bowers (2014), who also suggested considering interaction with the context and other school agents. This is because principals tend to adapt their actions based on the needs of teachers and students as well as local accountability policies and regulations, combining different types of leadership when performing their role.

School climate, in turn, is considered by Abrucio (2010) to be the greatest explanatory factor differentiating schools with better performance. The author emphasizes the importance of teamwork, cohesion, commitment, participation, co-responsibility, and motivation, also indicated by Lück (2009). For Sammons (2008), a school’s ethos is determined by the way its professionals work together, common values and goals, and the learning environment among students. Teddlie and Reynolds (2000) add a shared mission and vision, a sense of community, and consistency in curricular practices and the application of rules and disciplinary punishments. In short, when norms and goals are agreed upon and everyone’s participation is clearly defined, there is a positive school climate.

In the view of some authors, school climate is the result of more effective leadership (Abrucio, 2010; Lück, 2009), such that the two variables are interconnected. A study by Oliveira (2015) identified significant associations between leadership types and the principal’s managerial performance, particularly in the construction and maintenance of an organizational climate conducive to teaching and learning. The combination of these factors is in line with the institutional effect of Rutter et al. (2008), with greater power of action. It is therefore necessary to reflect on the concepts of management, leadership, and school climate in an integrated way, rather than in isolation, considering that it is a social organization with people in constant interaction. The author also indicates that few studies in the country have focused on understanding teacher perceptions of situations involving the work of the school principal or relationships in the school environment.

However, some of the empirical studies examined treat these factors differently. Analyzing inequalities in learning in Brazilian schools, the United Nations Educational, Scientific and Cultural Organization (Organização das Nações Unidas para a Educação, a Ciência e a Cultura [UNESCO], 2017) constructed four aggregate factors: leadership, climate, teaching and teacher characteristics, and school infrastructure. With regard to leadership, administrative, pedagogical and participatory factors were analyzed, as well as those related to human resources and the principal’s education and experience; for school climate, factors such as the cohesion of the pedagogical team, the school’s operating conditions, interventions for improvement (access, failure and dropout) and school violence were considered. It was concluded that in schools where leadership and climate are favorable, there are fewer chances of students experiencing exclusion and greater chances of them being at the appropriate learning level.

The school climate and principal leadership characteristics were also grouped into two different factors by Oliveira and Waldhelm (2016), in order to establish a potential relationship between intraschool factors and the mathematics performance of 5^th year students on the 2013 Prova Brasil. In this study, the authors used questions involving personal relationships between teachers and principals and the pedagogical and administrative performance of principals in schools, resulting in the creation of the Average Principal Leadership Index. Thus, the variables analyzed, which identify different leadership backgrounds, were grouped into a single factor in order to measure overall teacher perceptions of the subject. Notably, however, not all of the school leadership dimensions are included in this index.

In the same study, questions related to teacher perceptions of factors that affect the construction of a school environment conducive to student learning were added, in order to create the Average School Climate/Teacher Collaboration Index, including information on cohesion among staff members, generally related to the principal’s performance. However, it also does not consider all of the elements of school climate, such as violence and discipline. The model developed was able to explain 15.9% of the variation in student performance in Rio de Janeiro, reinforcing the importance attributed to intraschool factors in the literature.

The indicators discussed above enabled a differentiated analysis of the impacts of the method for filling principal positions in relation to perceptions of leadership and school climate, which guided the creation of the dependent variables in the empirical model adopted in this article. In the next section, we will present our hypotheses and discuss the methodology adopted to test them.

3. HYPOTHESES AND METHODOLOGICAL PROCEDURES

Based on this initial theoretical analysis, we present the two main hypotheses of this paper:

1. There is a difference between the average teacher perceptions of principal leadership in schools that adopt different principal selection modalities; and

2. There is a difference between the average teacher perceptions of school climate in schools that adopt different principal selection modalities.

To prove these hypotheses, the following methodological procedures were developed.

The first stage of this study’s empirical analysis involved a survey of the regulations on state school principal selection policies in Brazil’s 26 FUs and the Federal District, carried out in December 2016, using the official internet channels of the legislative and executive branches. A rich body of laws and rules was identified for analysis; these laws and rules were classified into three groups: State Education Plans (Planos Estaduais de Educação - PEE); career plans and statutes for teachers or education professionals; and state education systems, democratic management and specific laws on the subject. The selection modalities chosen reproduce the same responses as those on the Saeb questionnaires, to ensure comparison of the formulation and implementation stages of these policies. The summary of the laws analyzed is available in Appendix.

Next, microdata from the 2013 and 2015 Saeb contextual questionnaires was analyzed, answered by principals and teachers at state public schools that taught mathematics or the Portuguese language to 5^th year and 9^th year primary school classes, as well as 3^rd year secondary school classes. Among the 40,792 eligible schools, 39,098 principals responded to the specific question about their access to the position (96%). To ensure a representative sample, only data from teachers who responded to the ten selected questions on leadership (191,073) or the four on school climate (192,108) were included, representing an average of close to five teachers per school. Incomplete observations were maintained for all the variables, for the purposes of descriptive analysis. A final sample of 39,456 schools was thus obtained.

In this study, IRT was used to construct the climate and leadership indicators, in order to obtain measurements of latent constructs based on dichotomous or ordinal factors (Alves et al., 2015). These models are often adopted in educational studies due to the need to evaluate test items and estimate students’ abilities as well as to allow the use of questions with incomplete data, in order to estimate the value of the score of interest, which is quite common in these questionnaires. For this reason, they offer an advantage over conventional methods.

Then, based on the set of variables selected from the contextual questionnaires, the teachers’ responses were synthesized into two different indicators: principal leadership and school climate. Subsequently, these measurements were grouped at the school level by their average. As these variables are ordinal, Samejima’s (1969) model was employed, which assumes the unidimensionality of the data to produce a latent construct. The analysis of the eigenvalues and eigenvectors synthesizes the polychoric correlation matrix, more appropriate for ordinal variables: “If the first eigenvalue accounts for most of the variability present in the correlation matrix, this assumption is considered valid, which justifies the synthesis of the variables into a single factor” (Alves et al., 2015, p. 61). Accordingly, the parameters of the items and the distribution of responses were first estimated, in order to subsequently define the factor scores.

Next, a multiple linear regression model (Wooldridge, 2011) was applied, in order to measure variations (in percentage terms) in teacher perceptions of the principal leadership and school climate indicators when the principal selection method changes and the other factors remain constant. Effects were estimated for the following modalities¹: “civil service examination only”; “election only”; “appointment only”; “selection process only”; “selection process and election”; “selection process and appointment”; and “another method.” Each modality was transformed into a binary variable and analyzed simultaneously in the leadership and climate models.

For purposes of comparison, “selection process and appointment” was chosen as the standard modality, as the coefficient values in the schools that adopted it are closer to the national average for leadership and school climate. The results of the other modalities analyzed should thus consider this reference. The “another method” responses were discarded due to their difficulty of interpretation, as it is not possible to identify the evaluation mechanisms addressed in the response.

The selection of the dependent variables was made based on their relevance in the literature in terms of positive effects on learning. Specifically, the Principal Leadership and School Climate/Teacher Collaboration constructs developed by Oliveira and Waldhelm (2016) were adapted, which proved to be suitable in the test of association between variables. However, in the initial group of five questions related to climate, the decision was made to exclude the first, as it exhibited a different behavior in terms of polarity when compared to the others. This indicates that they may not be associated with a single construct. The ten leadership variables were maintained according to the authors’ model. Another difference, mentioned above, was the use of IRT rather than factor analysis, as it allows incomplete profiles to be included in the estimation of these constructs, i.e., variables with missing data are considered rather than excluded from the analysis. The results obtained are provided in Appendix.

Box 1contains the questions used to construct the principal leadership and school climate factors. In the former, the response options indicate the frequency per action and have 5 ordinal levels, which were categorized into values from 0 to 4. In the latter, the questions have 4 ordinal levels, with values from 0 to 3. After estimating the latent trait, using IRT, which returns the indicator on a normal distribution scale with an average of zero and a variance of 1 (ranging from -3 to 3 points), the two scales were transformed into a scale with an average of 50 and a standard deviation of 10, to facilitate data interpretation.

Box 1
School Climate and Principal Leadership Indicators on the 2013 and 2015 Saeb

The independent control variables selected refer to the characteristics of principal and management background, such as education, time in the position at the same school, gross salary and gender. The year of the Saeb and the school’s FU were also included (transformed into binary variables and inserted simultaneously into the models analyzed). In this case, the state of Acre was used as a baseline for comparison, as the climate and leadership indicators in this state’s schools are close to the national average.

In addition, four context indicators from the “Anísio Teixeira” National Institute for Educational Studies and Research (Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira - Inep) were inserted: average socioeconomic level of students in basic education schools²; school management complexity³; teacher retention⁴; and adequacy of teacher training⁵. Some of the variables were grouped into different categories than the questionnaires, in order to follow the findings in the literature or because they were a better fit in the models. For example, graduate studies consolidated four responses into a binary variable, while time in the position adopted the categories of Pereda et al. (2015).

The following equations are used in the models to be tested:

$SCHOLL\_CLIMATE={\beta }_0+{\beta }_1(\mathrm{F}\mathrm{I}\mathrm{L}\mathrm{L}\_\mathrm{P}\mathrm{O}\mathrm{S}\mathrm{I}\mathrm{T}\mathrm{I}\mathrm{O}\mathrm{N})+{\beta }_2\left(\mathrm{S}\mathrm{A}\mathrm{L}\mathrm{A}\mathrm{R}\mathrm{Y}\right)+{\beta }_3\left(\mathrm{W}\mathrm{O}\mathrm{M}\mathrm{A}\mathrm{N}\right)+{\beta }_4\left(\mathrm{G}\mathrm{R}\mathrm{A}\mathrm{D}\right)\mathrm{}+{\beta }_5(\mathrm{R}\mathrm{E}\mathrm{C}\_\mathrm{T}\mathrm{R}\mathrm{A}\mathrm{I}\mathrm{N}\mathrm{I}\mathrm{N}\mathrm{G})+{\beta }_6(\mathrm{O}\mathrm{R}\mathrm{G}\_\mathrm{T}\mathrm{R}\mathrm{A}\mathrm{I}\mathrm{N}\mathrm{I}\mathrm{N}\mathrm{G})+{\beta }_7(\mathrm{E}\mathrm{X}\mathrm{T}\mathrm{E}\mathrm{R}\mathrm{N}\mathrm{A}\mathrm{L}\_\mathrm{A}\mathrm{C}\mathrm{T}\mathrm{I}\mathrm{V})+{\beta }_8(\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{A}\mathrm{N}\mathrm{C}\mathrm{I}\mathrm{A}\mathrm{L}\_\mathrm{D}\mathrm{I}\mathrm{F})\mathrm{}+{\beta }_9(\mathrm{Y}\mathrm{E}\mathrm{A}\mathrm{R}\mathrm{S}\_\mathrm{P}\mathrm{O}\mathrm{S}\mathrm{I}\mathrm{T}\mathrm{I}\mathrm{O}\mathrm{N})+{\beta }_{10}(\mathrm{M}\mathrm{A}\mathrm{N}\mathrm{A}\mathrm{G}\mathrm{E}\mathrm{M}\mathrm{E}\mathrm{N}\mathrm{T}\_\mathrm{C}\mathrm{O}\mathrm{M}\mathrm{P})+{\beta }_{11}(\mathrm{S}\mathrm{E}\mathrm{L}\_\mathrm{S}\mathrm{T}\mathrm{U}\mathrm{D}\mathrm{E}\mathrm{N}\mathrm{T}\mathrm{S})\mathrm{}+{\beta }_{12}(\mathrm{T}\mathrm{E}\mathrm{A}\mathrm{C}\mathrm{H}\mathrm{E}\mathrm{R}\_\mathrm{R}\mathrm{E}\mathrm{T})+{\beta }_{13}(\mathrm{T}\mathrm{E}\mathrm{A}\mathrm{C}\mathrm{H}\mathrm{E}\mathrm{R}\_\mathrm{T}\mathrm{R}\mathrm{A}\mathrm{I}\mathrm{N}+{\beta }_{14}(\mathrm{F}\mathrm{U})+{\beta }_{15}(\mathrm{Y}\mathrm{E}\mathrm{A}\mathrm{R})+\mathrm{\mu }$

$\mathrm{P}\mathrm{R}\mathrm{I}\mathrm{N}\mathrm{C}\mathrm{I}\mathrm{P}\mathrm{A}\mathrm{L}\_\mathrm{L}\mathrm{E}\mathrm{A}\mathrm{D}\mathrm{E}\mathrm{R}\mathrm{S}\mathrm{H}\mathrm{I}\mathrm{P}={\beta }_0+{\beta }_1(\mathrm{F}\mathrm{I}\mathrm{L}\mathrm{L}\_\mathrm{P}\mathrm{O}\mathrm{S}\mathrm{I}\mathrm{T}\mathrm{I}\mathrm{O}\mathrm{N})+{\beta }_2\left(\mathrm{S}\mathrm{A}\mathrm{L}\mathrm{A}\mathrm{R}\mathrm{Y}\right)+{\beta }_3\left(\mathrm{W}\mathrm{O}\mathrm{M}\mathrm{A}\mathrm{N}\right)+{\beta }_4\left(\mathrm{G}\mathrm{R}\mathrm{A}\mathrm{D}\right)+{\beta }_5(\mathrm{R}\mathrm{E}\mathrm{C}\_\mathrm{T}\mathrm{R}\mathrm{A}\mathrm{I}\mathrm{N}\mathrm{I}\mathrm{N}\mathrm{G})+{\beta }_6(\mathrm{O}\mathrm{R}\mathrm{G}\_\mathrm{T}\mathrm{R}\mathrm{A}\mathrm{I}\mathrm{N}\mathrm{I}\mathrm{N}\mathrm{G})+{\beta }_7(\mathrm{E}\mathrm{X}\mathrm{T}\mathrm{E}\mathrm{R}\mathrm{N}\mathrm{A}\mathrm{L}\_\mathrm{A}\mathrm{C}\mathrm{T}\mathrm{I}\mathrm{V})+{\beta }_8(\mathrm{F}\mathrm{I}\mathrm{N}\mathrm{A}\mathrm{N}\mathrm{C}\mathrm{I}\mathrm{A}\mathrm{L}\_\mathrm{D}\mathrm{I}\mathrm{F})+{\beta }_9(\mathrm{Y}\mathrm{E}\mathrm{A}\mathrm{R}\mathrm{S}\_\mathrm{P}\mathrm{O}\mathrm{S}\mathrm{I}\mathrm{T}\mathrm{I}\mathrm{O}\mathrm{N})+{\beta }_{10}(\mathrm{M}\mathrm{A}\mathrm{N}\mathrm{A}\mathrm{G}\mathrm{E}\mathrm{M}\mathrm{E}\mathrm{N}\mathrm{T}\_\mathrm{C}\mathrm{O}\mathrm{M}\mathrm{P})+{\beta }_{11}(\mathrm{S}\mathrm{E}\mathrm{L}\_\mathrm{S}\mathrm{T}\mathrm{U}\mathrm{D}\mathrm{E}\mathrm{N}\mathrm{T}\mathrm{S})+{\beta }_{12}(\mathrm{T}\mathrm{E}\mathrm{A}\mathrm{C}\mathrm{H}\mathrm{E}\mathrm{R}\_\mathrm{R}\mathrm{E}\mathrm{T})+{\beta }_{13}(\mathrm{T}\mathrm{E}\mathrm{A}\mathrm{C}\mathrm{H}\mathrm{E}\mathrm{R}\_\mathrm{T}\mathrm{R}\mathrm{A}\mathrm{I}\mathrm{N}+{\beta }_{14}(\mathrm{F}\mathrm{U})+{\beta }_{15}(\mathrm{Y}\mathrm{E}\mathrm{A}\mathrm{R})+\mathrm{\mu }$

The complete table with the variables used is shown in Table 1.

Table 1
Descriptive statistics of the variables used on the 2013 and 2015 Saeb

Variable	No. of Schools	Average	Standard Deviation	Minimum	Maximum	NA
School_climate	38,805	57.3921	9.7386	23.98866	79.12826	651
Principal_leadership	38,854	59.1359	9.5945	14.72490	78.58443	602
Civil_service_examination_only	39,098	0.1016	0.3022	0	1	358
Election_only	39,098	0.2630	0.4402	0	1	358
Nomination_only	39,098	0.2355	0.4243	0	1	358
Selection_process_only	39,098	0.0494	0.2168	0	1	358
Selection_process _and_election	39,098	0.2033	0.4024	0	1	358
Selection_process _and_nomination	39,098	0.0592	0.2360	0	1	358
Principal_gross_salary	38,936	7.2863	1.9074	0	10	520
Principal_woman	39,252	0.7479	0.4341	0	1	204
Graduate_studies	38,966	0.8505	0.3564	0	1	490
Received_impact_training	39,178	0.6815	0.4658	0	1	278
Organized_continued_training	38,892	0.7404	0.4384	0	1	564
External_activity	39,131	0.2533	0.4349	0	1	325
School_financial_difficulty	39,120	1.1928	1.0798	0	3	336
3_to_5_years_in_position	39,165	0.4923	0.4999	0	1	291
6_to_10_years_in_position	39,165	0.1849	0.3882	0	1	291
11_to_15_years_in_position	39,165	0.0911	0.2878	0	1	291
More_than_15_years_in_position	39,165	0.0397	0.1954	0	1	291
School_management_complexity	37,651	4.0144	1.2385	1	6	1,805
Student_socioeconomic_level	37,797	4.4363	0.9792	1	7	1,659
Teacher_retention	37,407	3.0310	0.5614	0	5	2,049
Adequacy_teacher_training	37,663	3.9500	10.445	0	100	1,823
FU_(27)	39,456	-	-	0	1	0
Year_[T.2015]	39,456	-	-	0	1	0

• Note: NA = missing responses Source: Elaborated by the authors.

Additionally, an analysis of the legal frameworks for school principal selection in the FUs and the Federal District revealed a very heterogeneous configuration in the scope of the specific or democratic management legislation that includes the subject, compared to the guidelines established in the PEEs. These were approved between 2014 and 2016, except in the states of Minas Gerais and Rio de Janeiro, where they were still pending in the legislative bodies until December 2016. The continuity of the modalities across the years was also evaluated. In some cases, a transition to “selection process and election” is perceived, particularly in the most recent legal provisions. Others maintained their initial orientation over different periods.

The PEEs indicate the need to develop specific laws to regulate the subject within one or two years. They thus represent an important step toward institutionalizing a new policy, although they do not guarantee its implementation in practice. For this reason, the decision was made to disregard those plans as a criterion for classifying the selection method adopted in each FU, except for those that subsequently approved specific laws. This facilitated the comparison with the microdata from the 2013 and 2015 editions of the Saeb. Notably, the state policies do not always coincide with the principals’ responses to the questionnaires, as will be seen below.

4. RESULTS AND ANALYSES

In this section, we will present the results of the analyses performed using the legal texts from the FUs and the Federal District, the descriptive statistics based on the Saeb microdata, and the econometric tests performed using that same database, in order to prove the hypotheses enumerated above.

4.1. Analysis of the current rules for filling school principal positions

The “election only” criterion is the most common modality and was observed in fourteen FUs and the Federal District (56%). Their geographical distribution is quite comprehensive, with four located in the Center-West region, four in the Northeast, three in the North, two in the South and two in the Southeast. Seven of them appear to be in a stage of transition from the current policy to “selection process and election,” as recommended by the PNE, in Guideline 19.1. There is also a broad spectrum of evaluation mechanisms used in each of these systems. A detailed description of these rules and criteria per FU can be found in Appendix.

Currently, only São Paulo regulates “civil service examination only,” while “selection process only” was located only in Amazonas. Roraima was the only state classified as “appointment only,” due to the absence of legal frameworks on the subject prior to the PEE⁶. However, it is also important to note the high incidence of this modality in many states in the microdata analysis, despite the divergence with the legislation.

Furthermore, according to an opinion of the Supreme Federal Court of Brazil (Supremo Tribunal Federal - STF)⁷, the Chief Executive Officer is considered to have the exclusive authority to fill civil service positions for public school principals, who should be freely appointed by the local government. The STF considers unconstitutional any rules that provide for direct elections—with community participation—for principals in schools administered by the government. Although it is mentioned in legal provisions of subnational governments in Brazil and on the Saeb, it is not a standard electoral process, as it does not ensure that principals will remain in their positions for the entire term. For this reason, the term public consultation with the school community is used in the PNE.

“Selection process and election,” in turn, encompasses nine FUs (33%) and is the second most popular modality in the country. Its geographical distribution is predominantly in the Northeast region, with five occurrences, followed by two in the North, one in the South and one in the Southeast. This category excludes the seven FUs whose PEEs indicate a movement toward the mixed method but that have not yet been consolidated by other legal instruments.

In the analysis of the legislation and rules, it was not possible to identify any FU that established the “selection process and appointment” criterion or the “another method” modality. Nevertheless, there was a significant percentage of responses from principals in these categories when the microdata were analyzed, which will take place in the following section.

4.2 Descriptive analysis of methods for filling school principal positions per FU based on Saeb microdata

After analyzing the legislation, the descriptive statistics on filling principal positions were analyzed based on the principals’ responses to the 2013 and 2015 Saeb questionnaires. Once again, “election only” was the most popular modality among the schools, followed by “appointment only” and “selection process and election.” Conversely, “selection process only” had the lowest incidence among the FUs, followed by “selection process and appointment.”

The distribution percentages per modality are shown in Graph 1.

Graph 1
Filling Principal Positions per Modality on the 2013 and 2015 Saeb

From a territorial standpoint, there is great heterogeneity in terms of the distribution of the modalities⁸. “Civil service examination only” is concentrated in São Paulo. “Selection process only” is most common in Espírito Santo and Rio de Janeiro (2015). “Election only” predominates in the Federal District and in seven FUs: Alagoas, Mato Grosso, Paraná, Piauí (2013), Rio Grande do Norte, Rio Grande do Sul and Rondônia. Finally, “appointment only” predominates in eleven FUs: Amapá, Amazonas, Maranhão, Pará, Paraíba, Piauí (2015), Rio de Janeiro (2013), Roraima, Santa Catarina, Sergipe and Tocantins.

Graph 2shows the distribution of these percentages per FU among the single selection modalities. There is a greater dispersion in the “election only” and “appointment only” modalities, i.e., where the majority of the FUs are concentrated. In turn, the dispersion is low for “civil service examination only” and “selection process only,” in which all the FUs are below 50% and the great majority are less than 10%.

Graph 2
Filling the Principal Position per Single Modality on the 2013 and 2015 Saeb

Among the mixed modalities, only “selection process and election” is noteworthy, with a high dispersion and a greater concentration in seven FUs: Acre, Bahia, Ceará, Goiás, Mato Grosso do Sul, Minas Gerais and Pernambuco. “Selection process and appointment” and “another method” are not predominant in any state, which is evident based on the low dispersion of these modalities.

Graph 3shows the same percentage distribution per FU among the mixed selection modalities.

Graph 3
Filling the Principal Position per Mixed Modality on the 2013 and 2015 Saeb

4.3 Tests for Model 1: Effects of the method for filling positions on school climate

Following the legal and empirical analysis of the methods for filling principal positions, econometric tests were conducted to assess whether the school principal selection method influences perceptions of school climate, based on the construct adapted from Oliveira and Waldhelm (2016) and the equations described in the methodological section. Before proceeding to the tests, the internal consistency of the questionnaire applied for school climate was analyzed.

The internal consistency reliability of this construct was estimated using the Cronbach’s alpha coefficient, calculated based on the variance in the individual items and the covariances between them. For the school climate indicator, the four items range from 0.77 to 0.81, and together, 0.83. This indicates its reliability, as a value of a ³ 0.70 is considered satisfactory. In the polychoric correlation matrix, the values are positive and have a coefficient greater than 0.45, an indication that all these items can be associated with a single construct. The eigenvalues are 2.61 for the first item, almost five times higher than the second (0.56), and the third and fourth are even lower. The assumption of unidimensionality is therefore considered valid.

Graph 4shows the distribution of the average values of the school climate indicator per school, ranging from 23.99 to 79.13. The parameters of the indicator were adjusted to an average of 50, with a standard deviation of 10. Of the 39,456 observations, 7,565 were discarded because they did not exhibit any response in the variables used.

Graph 4
Distribution of the Average Values for the School Climate Indicator per School

By applying the multiple linear regression method, the estimation of the results of the equations described in the methodological section identified that the principal selection variable was statistically significant for explaining the behavior of the average school climate variable. This is because the p-value of the hypothesis test, related to the selection modality coefficient, was estimated to be less than 2.2e-16. The interpretation of this result indicates the rejection of the null hypothesis (H0: Beta = 0), concluding that the adjusted coefficient for the selection modality is significantly different from zero.

“Civil service examination only” obtained the highest ceteris paribus elasticity on average school climate, with a coefficient of 1.36, indicating that there is greater collaboration among teachers in day-to-day pedagogical activities. It is followed by “selection process and election” (0.80). Both were statistically significant at a level of 0.1%. The “selection process only” (0.59), “election only” (0.48) and “appointment only” (0.43) single modalities were significant at a level of 5%. Importantly, the “selection process and appointment” variable was used as a baseline for comparison. The effects can be observed in Table 2, in the section with the joint analysis of the results for the leadership and school climate variables.

Although the selection modality contributes to explaining the variation in the average school climate, it is necessary to consider other explanatory variables included in the model, which proved to be relevant. With regard to principal and management background, the following have positive effects: between 11 and 15 years in the position at the same school (0.72), organizing training activities (0.98), being a woman (0.91) and graduate studies (0.57). In the opposite direction are principals who carry out external activities (-0.43). Among Inep’s school context indicators, school management complexity showed the highest statistical relevance (-1.37). Consequently, the greater the size, number of classes and stages offered, the lower the average school climate. Teacher retention, however, which measures the continuity of teachers in schools in the last five years, has a positive effect (1.19).

In general, the FU variable exhibited the greatest change among the effects in this model, as it adds other unobservable or unexplained factors, which exercise a controlling role to isolate the effect of the selection modality on school climate. This corroborates the argument of Urick and Bowers (2014), as a principal’s managerial performance and the climate are influenced by the school context. Accordingly, the institutional characteristics of each Secretariat of Education should be partially captured in this indicator, pointing to the need for future analysis on the state capacity of the agencies and the processes to control and supervise teaching in schools, among other political and economic factors.

The lowest values were observed in the states of Pará, Santa Catarina and Sergipe, with coefficients above nine and a negative polarity. Although Pará was classified as “election only” in the analysis of the legal frameworks and the others as “selection process and election,” the microdata analysis indicates the predominance of “appointment only.” Ranging from -6.2 to -7.2 are Alagoas, Paraná, Piauí and Rio Grande do Norte, which were categorized as “election only” (in transition to “selection process and election”), in line with what we observed in the microdata analysis. Just below are Maranhão and Rio de Janeiro (-5.86). The former regulated “selection process and election,” but “appointment only” is predominant. The latter consolidated “election only” in specific legislation in 2016, but microdata analysis suggests a tendency to replace “appointment only” with “selection process only.”

The FUs that obtained the highest positive coefficients are Espírito Santo, Minas Gerais and São Paulo, as well as the Federal District ranging from 2.96 to 3.71. In no case is “appointment only” predominant. In the latter, there is a higher concentration of “election only” when observing the microdata, which converges with the legislation. In São Paulo, there is convergence with “civil service examination only” as well as a higher percentage of “another method” (26%) responses. Espírito Santo has one of the earliest regulations on “selection process and election,” but principals indicate a higher concentration of “selection process only.” Minas Gerais was classified as “election only,” although the microdata analysis indicates the predominance of “selection process and election.” Ceará, Bahia and Tocantins, in turn, also presented positive values, but with statistical relevance only at a significance level of 10%.

It can be concluded, in general, that the highest averages for the school climate indicator were found in schools where “civil service examination only” occurred, followed by “selection process and election,” according to the responses on the questionnaires. Conversely, “appointment only” had the smallest effect, close to the “election only” and “selection process only” modalities. When analyzed by FUs, the highest coefficients were seen in the federal capital and in three states in the Southeast region, where technical and/or participatory criteria are predominant in both the laws and in the microdata. “Appointment only” is predominant in the three FUs with the lowest coefficients, although their laws indicate other modalities. Among the other factors, those associated with the school context are particularly noteworthy: school management complexity (with a negative effect) and teacher retention (with a positive effect).

4.4 Tests for Model 2: Effects of the method for filling principal positions on principal leadership

In this section, we present the results of the econometric tests that made it possible to evaluate whether the school principal selection method influences perceptions of principal leadership, also based on the construct adapted from Oliveira and Waldhelm (2016) and the equations described in the methodological section. Once again, before proceeding to the tests, the internal consistency of the questionnaire applied for principal leadership was analyzed.

The Cronbach’s alpha of the principal leadership construct had a coefficient of 0.947 for all the items together, the lowest of which was 0.938, indicating the strong internal consistency of the questionnaire and the reliability of the model (a ³ 0.70). The polychoric correlation matrix showed positive values for the items, ranging from 0.48 to 0.87, indicating that they can be associated with a single construct. In the analysis of the eigenvalues and eigenvectors, the former had a coefficient of 6.95, much higher than the others, which were below 0.8. The assumption of unidimensionality is therefore considered valid.

Graph 5shows the distribution of the average values of the principal leadership indicator per school, with a range of 14.72 to 78.58. The parameters were adjusted for an average of 50, with a standard deviation of 10. Of the 39,456 observations, 7,540 were discarded because they did not exhibit any response in the variables used.

Graph 5
Distribution of the Average Values for the Principal Leadership Indicator per School

As in the previous model, by applying the multiple linear regression method, it was identified that the principal selection variable was statistically significant to explain the behavior of the average principal leadership variable. This can be asserted because the p-value of the hypothesis test, related to the selection method coefficient, was estimated to be less than 2.2e-16. The interpretation of this result indicates the rejection of the null hypothesis (H0: Beta = 0), as the adjusted coefficient for the selection modality is significantly different from zero.

In this model, “civil service examination only” also had the highest ceteris paribus elasticity with regard to average principal leadership (1.65), at a significance level of 0.1%. “Selection process and election” (0.77) presented a coefficient close to “election only” (0.74), both with a significance level of 1%. The “appointment only” and “selection process only” modalities, however, did not prove to be statistically significant. Similarly, the “selection process and appointment” variable was used as a baseline for comparison. The effects are provided in Table 2, in the section with the joint analysis of results for the leadership and school climate variables.

Among the other explanatory variables associated with principal and management background, included in the model tested and previously presented in the methodological section, the following are particularly noteworthy: organizing teacher training (1.35), graduate studies (1.32), being a woman (1.19), between 11 and 15 years in the position at the same school (1.11) and principals trained during the last two years (0.81). External activities (-0.46) and school financial difficulties (-0.21) contributed the least to increasing leadership and had negative polarity. Among Inep’s context indicators, school management complexity (-0.86) and teacher retention (0.78) were the most relevant, followed by the average socioeconomic level of the students (-0.31).

Once again, the control variables of the FUs indicate that they contribute to the greatest increase or decrease in the average of the school leadership indicator, particularly in Goiás (3.50) and Pernambuco (3.04). According to the descriptive analysis based on the microdata, “selection process and election” is predominant in both. This mixed modality is convergent with the legal frameworks in Pernambuco, while Goiás was classified as “election only.” The states of Amazonas, Minas Gerais, Paraíba and Rio de Janeiro also presented significant coefficients, ranging from 2.19 to 2.45. In the first, “selection process only” is regulated by law, but there was a higher concentration of “appointment only.” The same occurs in Paraíba and Rio de Janeiro (2013), where “appointment only” is predominant, despite “election only” legal frameworks. In Minas Gerais, “selection process and election” has a higher incidence, although its laws indicate “election only.”

Pará (-5.86), Sergipe (-5.09), Amapá (-3.19), and Roraima (-3.18) all had negative polarity and a high impact on average perceptions of principal leadership. There was a higher incidence of “appointment only” in these states, based on the microdata analysis, but with different regulatory categories: “election only” in Pará and Amapá; “selection process and election” in Sergipe; and “appointment only” in Roraima. Slightly lower is Rio Grande do Norte (-2.89), with a synergy between the legislation and the microdata analyzed for “election only” but seemingly in transition to “selection process and election.”

It can be concluded, in general, that the highest average values for the principal leadership indicator occur in schools with “civil service examination only” to fill this position, followed by “selection process and election” and “election only” (with a reduced effect), considering the responses on the questionnaires. Conversely, “appointment only” and “selection process only” were not statistically significant in this model. From a territorial standpoint, the two FUs with the highest average leadership values had higher percentages of “selection process and election,” one in convergence with the legislation and the other classified as “election only.” In turn, “appointment only” is more common in the four FUs with the lowest average values, despite their legislation regulating other criteria. The factors associated with principal and management background were also notable: graduate studies, being a woman, between 11 and 15 years in the position at the same school, and principals who organized or received training in the last two years.

4.5 Joint analysis of the results for the leadership and school climate variables

There are thus significant similarities between the results obtained in the two models. The modalities with technical criteria (civil service examination only) or the participation of the school community (selection process and election) exhibited the most significant averages in the school climate and principal leadership coefficients, to the detriment of the results for “appointment only.” However, the variation between the school climate and leadership coefficients in the FUs and the Federal District were even higher, demonstrating that regional characteristics are important determinants of teacher perceptions. Finally, it is suggested that factors linked to the school context (Inep) have the greatest effect on the average school climate, while those related to principal and management background (e.g., graduate studies and time in the position at the same school) are more associated with average perceptions of leadership.

Table 2 contains the scores obtained for the two models analyzed.

Table 2
Multiple Linear Regression for Principal Leadership and School Climate

DV	PRINCIPAL LEADERSHIP		SCHOOL CLIMATE
Independent variables	Estimate	Std. Error	Estimate	Std. Error
(Intercept)	55.687315***	(0.799605)	56.306334***	(0.717069)
Civil_service_examination_only	1.646511***	(0.298004)	1.360552***	(0.267461)
Election_only	0.738387**	(0.253610)	0.476487*	(0.227682)
Nomination_only	0.227849	(0.238804)	0.426758*	(0.214452)
Selection_process_only	0.462445	(0.314750)	0.593090*	(0.282365)
Selection_process _and_election	0.772306**	(0.243553)	0.795453***	(0.218507)
3_to_5_years_in_position	-0.140970	(0.131714)	-0.055220	(0.118149)
6_to_10_years_in_position	0.178734	(0.152210)	0.217331	(0.136530)
11_to_15_years_in_position	1.108839***	(0.196165)	0.718451***	(0.175911)
More_than_15_years_in_position	0.428728	(0.275392)	0.323784	(0.246927)
Principal_woman	1.185838***	(0.122417)	0.911323***	(0.109805)
Graduate_studies	1.320912***	(0.154930)	0.568827***	(0.138998)
Principal_gross_salary	0.206206***	(0.036652)	0.137138***	(0.032857)
External_activity	-0.457719***	(0.127994)	-0.427450***	(0.114842)
Received_impact_training	0.806481***	(0.114163)	0.203486*	(0.102401)
Organized_continued_training	1.345734***	(0.128323)	0.977657***	(0.115103)
School_financial_difficulty	-0.205385***	(0.050728)	-0.090582*	(0.045502)
Student_socioeconomic_level	-0.309474***	(0.070324)	0.072454	(0.063094)
School_management_complexity	-0.862131***	(0.044738)	-1.368843***	(0.040102)
Teacher_retention	0.782851***	(0.115911)	1.189774***	(0.104019)
Adequacy_teacher_training	-0.009864.	(0.005483)	0.018936***	(0.004925)
RO	-1.789910**	(0.693807)	-4.476555***	(0.621773)
AM	2.448696***	(0.644756)	-1.666005**	(0.579317)
RR	-3.183769***	(0.952001)	-4.847660***	(0.853545)
PA	-5.862145***	(0.642764)	-10.626578***	(0.577213)
AP	-3.189508***	(0.798474)	-3.401664***	(0.715227)
TO	1.139944.	(0.682579)	1.035055.	(0.614960)
MA	-1.816243*	(0.797837)	-5.859191***	(0.713269)
PI	-1.778065*	(0.695840)	-6.750230***	(0.623865)
CE	1.615394*	(0.759876)	1.150122.	(0.686450)
RN	-2.885094***	(0.659799)	-6.555051***	(0.591575)
PB	2.193275**	(0.690064)	-0.389019	(0.621202)
PE	3.041047***	(0.607218)	-2.078997	(0.544403)
AL	-0.948738	(0.756474)	-7.219218***	(0.678236)
SE	-5.093671***	(0.696007)	-9.263094***	(0.623206)
BA	-0.341517	(0.611446)	1.057532.	(0.547973)
MG	2.432379***	(0.549039)	2.960575***	(0.492232)
ES	1.316954.	(0.702545)	3.663726***	(0.629661)
RJ	2.428588***	(0.609854)	-5.868012***	(0.546665)
SP	-0.156459	(0.576711)	3.575618***	(0.517080)
PR	1.457083*	(0.573466)	-6.250644***	(0.514111)
SC	-0.965873	(0.606219)	-9.029309***	(0.543571)
RS	1.893707**	(0.601664)	-1.679507**	(0.539444)
MT	-1.379145*	(0.628782)	-0.120073	(0.563676)
MS	1.080223	(0.665212)	-3.780827***	(0.596646)
GO	3.499895***	(0.582215)	-2.240160***	(0.521973)
DF	-1.629740**	(0.631616)	3.706162***	(0.566323)
Year_[T.2015]	0.013826	(0.107426)	0.029994	(0.096361)

• Note: Scale of significance codes ‘***’ (0.001); ‘**’ (0.01); ‘*’ (0.05); ‘.’ (0.1); ‘’ (1). Source: Elaborated by the authors.

Box 2consolidates the results of the analysis of the legal frameworks and the Saeb microdata for each federative entity. The qualitative classification adopted for the linear regression coefficients aims to facilitate comparability across scales, as well as a systemic view of the formulation (legislation) and implementation (microdata) aspects of principal selection policies. Its purpose is specific for use in these models and should not be replicated at random. The substantive judgment of cut-off points by experts should reflect quality gains in schools according to attributes measured by the variables analyzed and their respective categories (UNESCO, 2019).

Box 2
Legal Frameworks and SAEB Microdata for School Climate and Principal Leadership

Despite the great differences between policies for filling principal positions in each FU, their implementation in schools is a slow process, either as a result of difficulties in the transition to new modalities or due to appointments that occurred prior to the approval of the legal frameworks. There is still a high proportion of schools with principals who were appointed politically, although no law regulates this. Additionally, a number of states have been incorporating technical criteria of merit and performance and public consultation with the school community, established in the PNE, which seems to have a positive impact on the school climate indicator.

5. CONCLUSION

The present study analyzed whether different policies for filling principal positions affect teacher perceptions of principal leadership and the school climate where they work. Schools where principals were selected by “civil service examination only” exhibited the highest average coefficients for these two factors, followed by “selection process and election,” confirming the initial hypothesis. Conversely, “appointment only” exhibited the lowest average coefficients among the selection modalities. Subsequently, responses on the 2013 and 2015 Saeb contextual questionnaires were compared to policies for filling principal positions in the 26 states and the Federal District.

Once again, the highest averages for school climate were observed in four FUs with a predominance of technical and/or participatory criteria for principal selection, whether in the legislation or the microdata analysis (“civil service examination only,” “selection process and election” or “election only”). “Appointment only” is predominant in the three states that recorded the lowest climate values, although the legislation provides for other modalities. A similar pattern was observed in the analysis of principal leadership, where the highest coefficients occurred in the two states with a predominance of “selection process and election,” one in convergence with the legislation and the other classified as “election only.” “Appointment only” was also the most common modality in the four states with the lowest leadership coefficients, although their laws have regulated other selection methods.

Other variables analyzed were also relevant for explaining the behavior of the average school climate and principal leadership coefficients that resulted from the econometric tests. In the case of the Inep context factors, teacher retention and school management complexity were particularly noteworthy, with positive and negative polarities, respectively. The socioeconomic level of the students had a low effect on leadership and none on climate. Among the factors associated with principal and management background, graduate studies, being a woman, between 11 and 15 years in the position at the same school and principals who have organized or received training in the last 2 years are notable. In general, contextual factors had the greatest impact on climate, including the relationships established among teachers, with a focus on academic work. In turn, principal and management background had a greater effect on perceptions of leadership, including pedagogical or administrative performance and personal relationships with teachers.

The results thus contribute to understanding how policies for filling positions—when they are well designed and executed—make it possible to select principals with the most appropriate competencies and skills for addressing day-to-day school challenges. Nevertheless, there is a need to conduct further research on the subject and identify other explanatory variables that affect school management, as the effects of selection modalities on principal leadership and school climate are limited.

Significant variation was observed between the results for each FU, indicating that local specificities influence teacher perceptions of climate and leadership. This occurs because the control variables for each FU add other factors that are unobservable or not explained in this work, thus affecting the results. It is recommended that future studies seek to identify factors related to state capacity and mechanisms to control and supervise teaching by Secretariats of Education that have a potential effect on principal leadership and school climate. Other political and economic factors in the states also deserve further investigation.

It is also important to understand which evaluation mechanisms have been applied in the selection processes of the 26 states and the Federal District, as similar nomenclature may have different prerequisites and procedures for analyzing candidates. This makes it difficult to compare the selection modalities in a standardized way, whether in terms of the categorization of the legislation or the classification adopted in the Saeb microdata, representing another limitation of this study. Finally, it should be noted that the approval of the PNE is still recent. It is advisable to conduct a longitudinal study to measure the aggregate effect on schools that have changed their principal selection modalities, thus providing greater evidence for managerial decision-making processes.

Considering the results obtained, which require further in-depth studies, the sample analyzed seems to indicate to public administrators that “civil service examination only” and “selection process and election” have positive effects on the leadership and school climate indicators, involving fundamental aspects of democratic management.

ACKNOWLEDGMENTS

We are grateful for comments from Joyce Soares Rodrigues Petrus, Paula Baptista Jorge Louzano and Pedro Feliú Ribeiro, which were essential to the development of this article.

APPENDIX

State Legislation for Filling Principal Positions

The classification of the state and Federal District legislation was based on the Saeb response categories, in order to compare the principal selection modality adopted, i.e., whether there is a difference between the formulation and implementation processes of these policies. An evaluation was also made as to whether the modality appears to be consolidated in the period under analysis, until December 2016, or in a phase of transition to meet the criteria established in the PNE (selection process and election).

Box 3
States with ‘Election Only’ for Principal Positions (Consolidated)

Box 4
States with ‘Election Only’ for Principal Positions (Transition)

Box 5
States with ‘Civil Service Examination Only’ for Principal Positions

Box 6
states with ‘appointment only’ for principal positions

Box 7
States with ‘Selection Process Only’ for Principal Positions

Box 8
States with ‘Selection Process and Election’ for Principal Positions (Consolidated)

Box 9
States with ‘Selection Process and Election’ for Principal Positions (Transition)

DESCRIPTIVE STATISTICS AND IRT MODELS

Below are the tables and graphs used in this study, enabling a deeper analysis of the data used to construct the IRT model and apply the linear regression model as well as to analyze the state legislation on filling principal positions. They also include the percent distribution of the selection modalities per FU in 2013 and 2015.

Table 3
Filling Principal Positions per FU on the 2013 and 2015 SAEB

FU	Examination		Election		Appointment		Sel. process		Election_proc		Appoint_proc		Another_meth.
	2013	2015	2013	2015	2013	2015	2013	2015	2013	2015	2013	2015	2013	2015
AC	1.0%	0.0%	9.4%	8.0%	5.4%	9.5%	0.0%	1.5%	81.3%	75.9%	1.0%	3.0%	2.0%	2.0%
AL	0.0%	0.0%	81.2%	77.4%	7.9%	13.6%	0.0%	1.1%	7.4%	2.8%	2.6%	3.4%	0.9%	1.7%
AM	0.8%	0.4%	1.8%	2.9%	45.8%	51.4%	30.5%	19.9%	1.8%	1.0%	14.1%	21.1%	5.3%	3.3%
AP	0.5%	0.6%	1.6%	1.2%	91.3%	92.2%	0.5%	1.2%	0.5%	0.6%	1.6%	1.8%	3.8%	2.4%
BA	1.0%	1.4%	19.7%	11.6%	21.6%	30.9%	0.6%	1.4%	44.5%	43.8%	9.1%	7.9%	3.5%	3.0%
CE	4.9%	3.7%	1.2%	2.1%	2.0%	2.1%	6.1%	4.3%	67.2%	75.0%	16.6%	11.7%	2.0%	1.1%
DF	0.2%	0.2%	64.0%	72.3%	3.8%	3.9%	0.2%	0.8%	28.5%	18.8%	1.0%	1.9%	2.3%	2.1%
ES	1.8%	0.7%	8.9%	5.8%	18.8%	25.8%	42.9%	36.9%	5.3%	5.4%	10.6%	15.3%	11.7%	10.2%
GO	0.1%	0.4%	22.6%	28.3%	10.1%	11.9%	0.4%	0.3%	59.5%	53.3%	1.8%	3.4%	5.5%	2.5%
MA	0.4%	1.6%	2.8%	4.8%	79.2%	77.5%	2.1%	0.0%	0.7%	0.5%	7.1%	9.1%	7.8%	6.4%
MG	1.1%	0.8%	13.8%	16.5%	6.6%	14.2%	0.4%	0.5%	64.2%	52.7%	11.2%	12.1%	2.6%	3.2%
MS	0.3%	1.0%	10.8%	9.4%	1.7%	8.1%	0.3%	0.7%	75.7%	65.4%	6.6%	9.7%	4.5%	5.7%
MT	0.4%	1.0%	64.1%	67.9%	11.8%	11.5%	0.6%	0.0%	12.7%	12.8%	3.0%	1.7%	7.4%	5.0%
PA	3.7%	2.4%	22.7%	23.3%	48.3%	53.2%	0.4%	0.8%	9.2%	6.1%	9.2%	8.9%	6.6%	5.3%
PB	0.2%	0.5%	30.5%	26.8%	57.4%	61.0%	0.2%	0.5%	5.7%	6.2%	4.2%	3.1%	1.7%	1.9%
PE	0.3%	0.4%	2.3%	1.3%	2.9%	5.4%	16.4%	12.1%	66.7%	62.5%	9.0%	16.1%	2.4%	2.2%
PI	1.6%	0.7%	54.5%	33.6%	19.1%	48.3%	7.6%	3.1%	9.9%	6.6%	4.8%	5.9%	2.5%	1.7%
PR	0.4%	0.8%	81.6%	78.3%	6.3%	10.6%	0.2%	0.1%	8.1%	5.7%	1.0%	1.1%	2.3%	3.4%
RJ	1.4%	2.3%	26.5%	21.3%	42.3%	27.5%	11.9%	32.2%	7.3%	4.6%	5.5%	7.1%	5.2%	5.0%
RN	0.5%	1.0%	78.6%	80.1%	8.8%	8.6%	0.2%	0.0%	5.0%	5.8%	1.7%	1.4%	5.2%	3.1%
RO	0.8%	0.4%	75.4%	82.1%	9.0%	8.6%	0.4%	0.0%	9.8%	7.1%	1.2%	0.4%	3.3%	1.5%
RR	0.0%	0.0%	4.8%	2.0%	76.2%	75.8%	1.9%	0.0%	0.0%	0.0%	4.8%	7.1%	12.4%	15.2%
RS	0.5%	0.6%	85.4%	77.3%	3.9%	10.1%	0.6%	0.3%	7.4%	5.3%	1.1%	1.9%	1.2%	4.4%
SC	0.5%	0.6%	1.0%	7.4%	92.7%	71.5%	0.0%	1.6%	0.5%	2.5%	3.1%	9.3%	2.3%	7.1%
SE	0.4%	1.6%	1.5%	6.1%	94.8%	55.7%	0.4%	0.8%	1.1%	28.0%	0.7%	6.5%	1.1%	1.2%
SP	45.0%	38.1%	1.0%	0.8%	17.0%	19.7%	6.3%	10.3%	1.1%	0.6%	4.1%	4.5%	25.5%	25.9%
TO	0.5%	0.3%	1.9%	0.6%	74.7%	88.3%	5.1%	0.3%	1.6%	0.0%	5.6%	3.5%	10.5%	7.0%
∑	10.8%	9.6%	26.3%	26.3%	22.9%	24.2%	4.4%	5.5%	21.6%	19.0%	5.4%	6.5%	8.6%	9.0%

• Note: The percentages are defined per year and per FU, totaling 100%. Source: Elaborated by the authors.

Table 4
Cronbach’s Alpha for the School Climate Indicator

Items	Cronbach’s alpha
No TX_RESP_Q054	0.8166
No TX_RESP_Q055	0.7844
No TX_RESP_Q056	0.7767
No TX_RESP_Q057	0.8194
All items	0.8341

• Source: Elaborated by the authors.

Table 5
Polychoric Correlation of the Items in the School Climate Indicator

Items	TX_RESP_Q054	TX_RESP_Q055	TX_RESP_Q056	TX_RESP_Q057
TX_RESP_Q054	1.0000	0.5625	0.4920	0.4675
TX_RESP_Q055	0.5625	1.0000	0.6315	0.4997
TX_RESP_Q056	0.4920	0.6315	1.0000	0.5732
TX_RESP_Q057	0.4675	0.4997	0.5732	1.0000

• Source: Elaborated by the authors.

Graph 6
Eigenvalues for the School Climate Indicator

Table 6
Parameter of the Items in the School Climate Indicator

Items	a	b1	b2	b3	b4
TX_RESP_Q054	1.6120	-1.9880	-1.4960	-0.4870	0.4370
TX_RESP_Q055	2.3400	-1.7410	-1.2770	-0.3070	0.5040
TX_RESP_Q056	2.5060	-2.1820	-1.6810	-0.4620	0.4890
TX_RESP_Q057	1.7000	-2.0140	-1.0680	0.3620	1.5170

• Source: Elaborated by the authors.

Graph 7
Characteristic Curves for Items - School Climate Indicator

Graph 8
Information Curves for Items - School Climate Indicator

Graph 9
Information Curve for the test - School Climate Indicator

Table 7
Information Curve for the Test - School Climate Indicator

Items	Cronbach’s alpha
No TX_RESP_Q058	0.9446
No TX_RESP_Q059	0.9431
No TX_RESP_Q060	0.9454
No TX_RESP_Q061	0.9388
No TX_RESP_Q062	0.9423
No TX_RESP_Q063	0.9425
No TX_RESP_Q064	0.9403
No TX_RESP_Q065	0.9407
No TX_RESP_Q066	0.9443
No TX_RESP_Q067	0.9414
All Items	0.9476

• Source: Elaborated by the authors.

Table 8
Polychoric Correlation of the Items in the School Leadership Indicator

Items	TX_RESP_Q058	TX_RESP_Q059	TX_RESP_Q060	TX_RESP_Q061	TX_RESP_Q062
TX_RESP_Q058	1.00000	0.75349	0.67630	0.73450	0.57447
TX_RESP_Q059	0.75349	1.00000	0.67483	0.73897	0.63186
TX_RESP_Q060	0.67630	0.67483	1.00000	0.72424	0.60808
TX_RESP_Q061	0.73450	0.73897	0.72424	1.00000	0.73018
TX_RESP_Q062	0.57447	0.63186	0.60808	0.73018	1.00000
TX_RESP_Q063	0.55931	0.60355	0.58450	0.69172	0.82456
TX_RESP_Q064	0.68184	0.64150	0.66834	0.75952	0.62780
TX_RESP_Q065	0.68919	0.64182	0.67645	0.75946	0.62141
TX_RESP_Q066	0.48683	0.53564	0.49699	0.61313	0.57907
TX_RESP_Q067	0.57391	0.59846	0.56447	0.70018	0.64360
Items	TX_RESP_Q063	TX_RESP_Q064	TX_RESP_Q065	TX_RESP_Q066	TX_RESP_Q067
TX_RESP_Q058	0.55931	0.68184	0.68919	0.48683	0.57391
TX_RESP_Q059	0.60355	0.64150	0.64182	0.53564	0.59846
TX_RESP_Q060	0.58450	0.66834	0.67645	0.49699	0.56447
TX_RESP_Q061	0.69172	0.75952	0.75946	0.61313	0.70018
TX_RESP_Q062	0.82456	0.62780	0.62141	0.57907	0.64360
TX_RESP_Q063	1.00000	0.65375	0.63862	0.59968	0.66572
TX_RESP_Q064	0.65375	1.00000	0.87104	0.68315	0.73895
TX_RESP_Q065	0.63862	0.87104	1.00000	0.64551	0.70938
TX_RESP_Q066	0.59968	0.68315	0.64551	1.00000	0.84357
TX_RESP_Q067	0.66572	0.73895	0.70938	0.84357	1.00000

• Source: Elaborated by the authors.

Graph 10
Eigenvalues for the School Leadership Indicator

Table 9
Parameters of the Items in the School Leadership Indicator

Items	a	b1	b2	b3
TX_RESP_Q058	2.3570	-1.9000	-0.3140	0.9670
TX_RESP_Q059	2.4120	-2.3400	-0.9080	0.5260
TX_RESP_Q060	2.2850	-1.7170	-0.4270	0.7180
TX_RESP_Q061	3.6420	-1.8150	-0.6260	0.5110
TX_RESP_Q062	2.4400	-2.6300	-1.1820	0.3450
TX_RESP_Q063	2.3860	-2.6730	-1.1910	0.3130
TX_RESP_Q064	3.5080	-1.4390	-0.3630	0.5940
TX_RESP_Q065	3.3720	-1.4950	-0.3610	0.6050
TX_RESP_Q066	2.0770	-2.6280	-1.3510	-0.0420
TX_RESP_Q067	2.6920	-2.2360	-1.0960	0.0150

• Source: Elaborated by the authors.

Graph 11
Characteristic Curves for Items - School Leadership Indicator

Graph 11
continuation

Graph 12
Information Curves for Items - School Leadership Indicator

Graph 13
Information Curve for the Test - School Leadership Indicator

Graph 14
Analysis of Multiple Linear Regression Residuals for School Climate

Graph 15
Analysis of Multiple Linear Regression Residuals for Principal Leadership

REFERÊNCIAS

• Abrucio, F. L. (2010). Gestão escolar e qualidade da Educação: um estudo sobre dez escolas paulistas. In P. J. B. Louzano(Ed.), Estudos & Pesquisas Educacionais (pp. 241-274). São Paulo, SP: Fundação Vitor Civita.
• Alves, M. T. G., & Franco, C. (2008). A pesquisa em eficácia escolar no Brasil: evidências sobre o efeito das escolas e fatores associados à eficácia escolar. In N. Brooke, & J. F. Soares(Orgs.), Pesquisa em eficácia escolar: origens e trajetórias(pp. 482-500). Belo Horizonte, MG: UFMG.
• Alves, M. T. G., Xavier, F. P., Barbosa, L. E., Caldeira, B. F., Silva, C. A. S., & Soares, J. F. (2015, agosto). Fatores contextuais das escolas da educação básica: dados, métodos e aplicações. In Anais de Reuniões da ABAVE 8 (No. 2. pp. 57-76), Florianópolis, SC.
• Ghanem, E. G. G.Jr. (1995). Participação popular na gestão escolar: bibliografia São Paulo, SP: Ação Educativa.
• Gremaud, A. P., Pazello, E. T., & Maluf, B. T(2016, agosto). O impacto da indicação política do diretor no desempenho educacional. In Anais de Reuniões da ABAVE 8 (No. 2. pp. 313-336), Florianópolis, SC.
• Instituto Brasileiro de Geografia e Estatística. (2015). Perfil dos Estados e dos Municípios Brasileiros: cultura 2014 Rio de Janeiro, RJ: Autor.
• Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2015). Plano Nacional de Educação PNE 2014-2024: Linha de Base Brasília, DF: Autor.
• Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2017, março 18). Microdados Saeb (Aneb/Prova Brasil) de 2013 e de 2015 Brasília, DF: Autor. Recuperado de http://portal.inep.gov.br/microdados
  » http://portal.inep.gov.br/microdados
• Leithwood, K., & Jantzi, D. (2009). A review of empirical evidence about school size effects: A policy perspective. Review of educational research, 79(1), 464-490.
• Libâneo, J. C (2001). Organização e gestão da escola: teoria e prática Goiânia, GO: Editora Alternativa.
• Lück, H. (2009). Dimensões de gestão escolar e suas competências Curitiba, PR: Positivo.
• Lück, H. (2011). Mapeamento de práticas de seleção e capacitação de diretores escolares São Paulo, SP: Fundação Vitor Civita.
• Maia, G. Z. A., & Manfio, A. (2010). O provimento do cargo de gestor escolar e a qualidade de ensino: análise de publicações nacionais (1990-2005). RBPAE - Revista Brasileira de Política e Administração da Educação, 26(3), 477-494.
• Marks, H. M., & Printy, S. M. (2003). Principal leadership and school performance: An integration of transformational and instructional leadership. Educational administration quarterly, 39(3), 370-397.
• Mortimore, P., Sammons, P., & Ecob, R. (1988). Expressing the magnitude of school effects: A reply to Peter Preece. Research Papers in Education, 3(2), 99-101.
• Oliveira, A. C. P. (2015). As relações entre direção, liderança e clima escolar em escolas municipais do Rio de Janeiro (Tese de Doutorado). Pontifícia Universidade Católica do Rio de Janeiro, Rio de Janeiro, RJ.
• Oliveira, A. C. P., & Carvalho, C. P. (2015, novembro). Gestão escolar, liderança do diretor e resultados educacionais no Brasil. In Anais da 37º Reunião Nacional da ANPED, Florianópolis, SC.
• Oliveira, A. C. P., & Waldhelm, A. P. S. (2016). Liderança do diretor, clima escolar e o desempenho dos alunos: qual a relação. Ensaio: Avaliação e Políticas Públicas em Educação, 24(93), 824-844. Recuperado de http://www.scielo.br/pdf/ensaio/v24n93/1809-4465-ensaio-24-93-0824.pdf
  » http://www.scielo.br/pdf/ensaio/v24n93/1809-4465-ensaio-24-93-0824.pdf
• Organização das Nações Unidas para a Educação, a Ciência e a Cultura. (2019). Qualidade da infraestrutura das escolas públicas do ensino fundamental no Brasil Brasília, DF: Autor.
• Organização das Nações Unidas para a Educação, a Ciência e a Cultura. (2017). Desigualdades de aprendizado entre alunos das escolas públicas brasileiras. Evidências da Prova Brasil (2007 a 2013)Brasília, DF: Autor.
• Pereda, P., Lucchesi, A., Mendes, K., & Bresolin, A. (2015, dezembro). Avaliação do impacto do processo de seleção de diretores nas escolas públicas brasileiras. In Anais do 43º Encontro Nacional de Economia, Florianópolis, SC.
• Rutter, M., Maughan, B., Mortimore, P., Ouston, J., & Smith, A. (2008). Estudos anteriores. In N. Brooke, & J. F. Soares (Orgs.), Pesquisa em eficácia escolar: origem e trajetórias (pp. 142-152). Belo Horizonte, MG: UFMG.
• Samejima, F. (1969). Estimation of latent ability using a response pattern of graded responses (Psychometric Monograph, No. 17). Richmond, VA: Psychometric Society.
• Sammons, P. (2008). As características-chave das escolas eficazes. In: Brooke, N., & Soares, J. F. (Orgs.). Pesquisa em eficácia escolar: origens e trajetórias (pp. 335-387). Belo Horizonte, MG: UFMG.
• Soares, J. F. (2004). O Efeito da Escola no Desempenho Cognitivo de seus Alunos. REICE - Revista Electrónica Iberoamericana sobre Calidad, Eficacia y Cambio en Educación, 2(2), 83-104. Recuperado de https://www.redalyc.org/pdf/551/55120207.pdf
  » https://www.redalyc.org/pdf/551/55120207.pdf
• Teddlie, C., & Reynolds, D. (2000). The International handbook of school effectiveness research London, EN: Falmer Press.
• Teddlie, C., & Stringfield, S. (1993). Schools make a difference: lessons learned from a 10-year study of school effects New York, NY: Teachers College Press.
• Urick, A., & Bowers, A. J. (2014). What are the different types of principals across the United States? A latent class analysis of principal perception of leadership. Educational Administration Quarterly, 50(1) 96-134.
• Wooldridge, J. M. (2016). Introdução à econometria: uma abordagem moderna São Paulo, SP: Cengage Learning.

• 1
  Question no. 14 from the Principal contextual questionnaire on the 2013 and 2015 Saeb.
• 2
  Defined by family income, property ownership, employment of domestic help by the students’ family and the education level of their parents or guardians.
• 3
  Calculated by the size (number of students enrolled); number of work (teaching) shifts; complexity of the stages offered; and number of stages/modalities offered.
• 4
  Evaluates teacher retention in basic education schools in the last five years.
• 5
  Measures the number of teachers working in basic education whose teaching subject is in line with their academic training, according to the legal standards and provisions.
• 6
  This plan provides for the adoption of technical criteria of merit and performance but was disregarded due to the reasons explained above. During the document research, no specific laws or rules with such regulations were identified, only a civil society movement favoring “election only” over “appointment only.” It was therefore deduced that this method was a prevalent policy until 2016, which was confirmed by the microdata, as analyzed in the following section.
• 7
  Direct Action of Unconstitutionality (Ação Direta de Inconstitucionalidade - ADI) no. 2997, of August 12, 2009.
• 8
  Rio de Janeiro and Piauí were counted twice, as their most common selection modalities changed between 2013 and 2015.

• [Translated version] Note: All quotes in English translated by this article’s translator.

Publication Dates

• Publication in this collection
  27 Sept 2021
• Date of issue
  Jul-Aug 2021

History

• Received
  18 Dec 2019
• Accepted
  06 Feb 2021