Evaluating the impact of the selection process of principal in Brazilian public schools

Avaliação do impacto do processo de seleção de diretores em escolas públicas brasileiras

Abstract

Educational quality in Brazil has improved at a slow pace despite continued public policy efforts. The international literature has noted the importance of school management and the principal’s profile on students’ performance, but this discussion is recent in Brazil. In this study, we investigate the effects of the selection process of principals on students’ performance from public state schools in Brazil. We conclude that while the “selection process” channel is not clear, the “principal quality effect” indicates that the more democratic and technical selection methods tend to select most qualified principals in terms of leadership and managerial abilities. The principal who has time to implement changes, who adopts management practices to encourage teachers and has experience being the leader of the school present better results in terms of student’s performance. Therefore, it is important that the principal selection process choose professionals with the skills mentioned.

Keywords
basic education; principals’ selection; evaluation of Brazilian educational policy; differences in differences propensity score matching

Resumo

A qualidade da educação no Brasil ainda avança lentamente, apesar dos esforços de políticas públicas. Embora a literatura internacional já aponta para a importância da gestão escolar e perfil do diretor no desempenho dos alunos, essa discussão é ainda recente no Brasil. Nesse estudo nós investigamos o efeito do processo de seleção de diretores no desempenho dos alunos de escolas públicas estaduais brasileiras. Os resultados mostram que que, embora o efeito do processo de seleção não seja claro, o efeito das características do diretor indica que processos mais técnicos e democráticos de escolha do diretor tendem a selecionar diretores mais qualificados em termos de liderança e habilidades gerenciais. O diretor experiente, que tem tempo para implementar mudanças e que adota práticas gerenciais para estimular os professores apresentam melhores resultados, em termos de desempenho dos alunos. Nesse sentido, é importante que o processo seletivo escolha profissionais com as características mencionadas.

Palavras-chave
ensino fundamental; seleção de diretores; avaliação de políticas educacionais; diferenças em diferenças pareamento com escore de propensão

1 Introduction

In recent decades, several countries have advanced more in educational quality (academic achievement, students’ performance, etc.) than Brazil, despite continued efforts of public policies, such as promoting access to education (93.8% of 6-to-14-year-olds attend elementary school and 81.2% of 15-to-17-year-olds are enrolled in high school)¹ and high levels of public spending in terms of GDP, higher than many countries participating in the OECD Program for International Student Assessment (PISA)². In 2011 while Brazil spent 6.1% of GDP (or 19% of public expenditure), the OECD countries spent an average of 5.6% of GDP³ (Fernandes et al. 2014). However, the performance of Brazilian schools in the PISA standardized testing the country ranked: 57^th (among 65 countries) in mathematics, 55^th in reading (Portuguese) and 59^th in sciences, while Singapore was third and South Korea ranked seventh in the overall standings.

Thus, the major challenge now facing Brazilian education, especially in public schools, is to raise quality, particularly in basic education (preschool, elementary and secondary schools) due to its importance for the future performance of individuals. It is well known that several factors related to the characteristics of the family, student and school influence students’ performance. Besides that, a crucial factor affecting school success that has received little attention in the literature is the role played by the school principal and its methods of selection.

The school principal is the legal head of the educational unit and the person responsible for organizing and planning the school activities as well as creating conditions to enhance the educational process. Therefore, the importance of school principals is centered on their ability to create better working conditions for teachers (motivation and attendance of teachers, encouraging training, etc.), ensure the basic quality of infrastructure (ventilation, lighting, restrooms, security against violence, etc.), provide conditions for the academic life of students (motivation, disciplinary matters, etc.), organize and plan school activities, assist and enhance the educational/instructional process and encourage the participation of parents and the school community.

The most common methods to select principals are different types of appointment (technical, political or others) referral, election (including or not the community participation), examination ranking, public exam (for public schools)⁴, national tests, interview, resume submission, certification, or a combination thereof. The ideal selection process should choose the most qualified candidate, according to reasonable selection criteria, to meet the above-mentioned technical and managerial qualifications. Beyond that, the selection process per se can create favorable conditions (or not) for principals conduct their activities. Therefore, we can distinguish between two possible channels through which the different principal selection methods might impact school performance: the “selection process effect” - related to the environment for principals to develop their activities - and the “principal quality effect” - related to the principals’ personal/managerial characteristics. For instance, depending on the principal selection method, the principal can stay a longer or shorter length of time in the position, or can have greater or lesser access to funds, or even greater or lesser political articulation for creating partnerships, or dialoguing. In turn, the “quality” of the principal can impact students’ learning and/or achievement by means of his/her personal traits, such as academic background, experience in the position and management skills (leadership, attainment of goals, motivation of staff and students, etc.), because different processes select principals with different qualities/characteristics.

Within this context, we state the following assumptions concerning the “selection process effect”: a) usually appointment methods or election are characterized by higher principal turnover resulting in shorter time periods to implement the actions planned, while “public exam” method can generate an overly extended tenure of the same principal in the position hindering the implementation of more modern school administration practices, since in the case of public schools this method of selecting principals ensures stability in the position; b) it is possible that certain methods, such as “political appointment” or “election” work better to select principals who are more at ease with raising funds, due to greater political articulation. Related to the “principal quality effect”: c) “public exam”, “certification”, “national tests”, or “appointment by technical staff” methods are usually based on technical criteria (academic background, experience, etc.), while the “election” and “political appointment” methods tend to favor the candidate’s management project to the school, merit criteria and/or leadership characteristics.

To understand the potential effects of the principal selection process, it is worth scrutinizing the economic incentives behind such choices. Suppose that the principal is chosen by a central planner⁵. If the performance and characteristics of candidates for principal are known by planner (or can be inferred through observed performance) and if the objective of the planner is compatible with the well-being of society, the theoretical results show that the planner’s choice (political appointment) would be effective. Nevertheless, the legitimacy of any of these assumptions can be questioned, particularly with regard to a political appointment by local educational authorities which could work toward interests other than those aligned with efficient selection of the principal.

Another interesting example arises when employees and members of the community choose the principal by election. If the information available to voters (staff and community members) allows them to assess the ability of the candidates more effectively than the assessment made by a public planner and if the purpose of the voting group is compatible with that of the society, the economic results indicate that it is possible to choose the best possible principal for that community using this process. Again, these conditions can be violated if employees and teachers have self-interests that conflict with the interests of society. This process can also fail if the community is not well informed about the candidates’ performance.

For cases in which the principal is selected by means public exam or other testing, such incentives are more complex since the definition of subjects and test formats are prepared by specialists based on the expected performance of the candidates.

From these variations, a major discussion has developed on the best manner to select school principals. Therefore, in this study we aim to understand how the different ways of selecting principals influences the performance of public state school students in their 5^th and 9^th grades in Brazil. These grades have more impact on future educational outcomes and reflect intermediate (5^th) and final (9^th) evaluations from basic education. We examine the impact of both “selection process” and “principal quality” channels in Portuguese and mathematics scores, IDEB⁶ rate and dropout rate. We use panel data of state public schools⁷ and two different approaches to identify the effects: full sample panel analysis and a case study where the selection process of the principal changed during the period analyzed.

2 Principal selection in Brazil

In Brazil, there are different methods to select principals of public schools, such as appointment (technical, political or others), election (including or not community participation), state or municipal public exam, national tests, certification, or a combination thereof. The method by which a principal is chosen for the position is different in each state (or municipality) and has been changing in recent decades. In the past, the principal was traditionally selected based on political criteria (chosen by political appointment). With redemocratization in Brazil (from the 1980s forward), the demand for a more democratic and transparent process increased, and other methods were gradually introduced such as public exam, election (by staff, parents and/or students), election combined with public exam, or certification or appointment by technical staff on the basis of minimum requirements (educational level, experience in school administration, length of teaching practice). The selection methods have also changed with respect to the tenure and monitoring of principals’ performance. Moreover, such innovations in the processes of principal selection differ from state to state in the way they take place and the amount of time they take⁸ (Xavier et al., 1994). In many cases, different methods are combined to legitimize the selection process.

Besides that, it is not clear in the (scarce) process selection literature which method selects the “best” principal candidate in terms of improving school performance, either through the “selection process effect” or “principal quality effect” (or both). A survey conducted by Fundação Victor Civita (2011) with 24 (of 27) Brazilian state education authorities indicated the main advantages and disadvantages of each method of principal selection. Concerning the “political appointment” method, although it can follow technical criteria and occur according to the school demand and tailored characteristics, in many cases it is based exclusively on political issues and the person chosen may not be accepted by the school community. The “election” method, in turn, tends to select a principal that is accepted as legitimate by the school community since the school staff and parents usually vote in the election. The disadvantage of this type of selection is that it may favor clientelistic practices or antagonistic groups, generating fragmentation in the school. Concerning the “public exam” method, the main issue raised is the possibility of non-acceptance of the principal within the school community, although the selection was made according to criteria established by experts.

3 Literature review

We present the studies by grouping them as follows: a) studies related to the “selection process effect” (via turnover of principals, stability in the office or access to financial resources), and b) studies that focus on the “principal quality effect” (principals’ personal traits and school management characteristics). There are few studies on the impact of the principal selection method on school performance.

With regard to the “selection process effect”, Coelli and Green (2012) noted a negative effect of short principal tenure on the English score (among other performance measures) of high school students in British Columbia, Canada, from 1995 to 2004, while Béteille et al. (2012), using longitudinal data for Miami schools, showed that principal turnover is detrimental to school performance. Likewise, Miranda and Pazello (2014) analyzed data from Prova Brasil⁹ between 2005 and 2011 and used the political discontinuity in elections for the mayor as an instrument for the principal turnover variable in order to avoid potential problems of endogeneity. The authors concluded that the political appointment of principals impacts the IDEB negatively by causing an 11-point decline on average in the SAEB¹⁰ score, since on average the “political appointment” method was associated with a higher turnover.

In turn, Barros and Mendonça (1998) examined the changes made in the selection methods in the redemocratization period (from 1982 to 1993), using a state panel, which expanded that selection process including the “election” method The authors found a negative and statistically significant impact of the “election” method on students’ performance, though the impact was modest.

Concerning the evidence on the “principal quality effect”, Dobbie and Fryer (2013) analyzed a set of managerial practices adopted in high performing charter schools¹¹ in New York and concluded that the five best practices¹² - increased institutional time, more effective teachers and administrators, high dosage tutoring, data driven instructions and culture of high expectations - were strongly correlated with students’ achievement and explained approximately 45% of the variation in school effectiveness. Interestingly, they also found that the traditional input measures analyzed - class size, expenditure per student, teacher fraction with no certification and teacher fraction with advanced degree - were not correlated with school performance. In the same direction, Fryer (2014) used experimental methods and showed that, on average, the adoption of the five index managerial practices in low performing public schools in the U.S. significantly increased students’ achievement in mathematics in the treated schools by 0.15-0.18 standard deviation. In turn, Bloom et al. (2015) analyzed the quality of management in four broad areas - performance monitoring, operations management, target setting and talent management in day to day activities - in 1,800 schools in eight countries (including Brazil). They found that higher management quality is strongly positively correlated with students’ performance and had a greater effect than class size, competition or teacher quality. Therefore, they stated that the role of the principal should focus on school management either through standardized management procedures or through the principal’s existing characteristics as a manager. Another study that sought to identify good management practices, was conducted in several countries, including Brazil, in four dimensions within the school: operating and monitoring, setting goals to improve learning, personnel management, and leadership (World Management Survey Research, 2014). Based on data from interviews, the study showed that Brazil ranked penultimate (only better than India) under the general management practices indicator, with its worst dimension being personnel management.

Besides managerial practices and/or principals’ managerial skills, other empirical studies have tried to understand how the learning process is affected by principals’ personal traits, such as educational background, years of schooling, job experience, leadership characteristics, etc. In particular, leadership impact can work through a variety of mechanisms, such as supervision and retention of teachers and students, educational coordination of the courses, articulation of school vision and goals, resource allocation or openness to the community during tenure. In this context, Eberts and Stone (1988) and Béteille et al. (2012) examined American schools and found that the principal’s teaching experience had a positive effect on students’ performance. Béteille et al. (2012) noted the influence of the principal on recruiting and retaining the best teachers while dismissing the worst ones and promoting the development of teachers already hired. On the other hand, Eberts and Stone (1988) found evidence of a negative impact on students’ performance when dealing with a lack of consensus between the principal and the teachers and the exercise of leadership that is too forceful by the principal in situations of conflict resolution and guidance for teachers. In turn, Branch et al. (2012) and Clark et al. (2009) found a positive relationship between principals’ experience and students’ test scores.

Addressing Brazil, Marioni et al. (2014) used panel micro-data from a longitudinal study of the 2005 school generation (GERES) from 2005 to 2008 to evaluate different effects of principals on students’ performance. They found evidence of a positive effect for the following variables: teachers’ confidence in the principal, principals’ attention given to administrative rules, and principals’ encouragement of innovative activities.

4 Database

We use a panel of schools from Prova Brasil and the school censuses from 2007 and 2011¹³. Prova Brasil is a census-based evaluation conducted every two years encompassing students in the 5^th grade and 9^th grade at state schools¹⁴. The final sample, after excluding outliers and processing the data, was 28,906 state schools in 2007 and 26,973 state schools¹⁵ in 2011 across all Brazilian states.

The school performance indicators (proxies for student learning) were the IDEB score, mathematics and Portuguese scores and dropout rate. When it comes to the “selection of principal”¹⁶ variable, we investigate the following methods of principal selection: a) “public exam”), b) “election”, c) “public exam and election”, d) “appointment by technical staff”, d) “political appointment”, e) “appointment by others”, and f) “other methods¹⁷”.

With regard to school performance (Table 1), all the indicators (Portuguese and mathematics score, IDEB and dropout rate) showed an improvement in 2011 compared to 2007 (for both 5^th and 9^th grades) and the 5^th grade registered a higher increase in all the indicators (or in the case of the dropout rate, a higher decrease) in comparison to the 9^th grade, mainly due to policies oriented to literacy and early childhood education.

Table 1
State school performance scores - 2007 and 2011

Performance Indicator	5th grade		% variation (2011/07)	9th grade		% variation (2011/07)
	2007	2011	(%)	2007	2011	(%)
Score in Portuguese	173.5	189.2	9.1%	228.2	238.8	7.2%
Score in Mathematics	190.6	208.2	9.3%	240	245.7	7.0%
Ideb	4.2	5.0	19.8%	3.5	3.9	15.0%
Dropout Rate	2.8	1.5	-44.8%	7.1	4.5	-32.7%

Source: Elaborated by authors.

Figure 1 presents the breakdown of the different methods of principal selection adopted in state schools in 2007 and 2011. The most frequent methods were “election” (28% in 2007 and 30% in 2011), followed by “public exam and election” (22% in 2007 and 24% in 2011) and “public exam” (15% in 2007 and 14% in 2011). The incidences of “political appointment”, “appointment by technical staff” and “appointment by others” are more stable and less frequent: 10%, 6% and 10% respectively in 2011.

Figure 1
Principal selection method - 2007 and 2011

Considering the different geographic regions in Brazil, Table 2 presents the incidence of each principal selection method in state schools by Brazilian states in 2011. The incidence of principal selection through “political appointment” is higher in the northern and northeastern states and in the state of Santa Catarina in the south region (Table 2). On the other hand, the political appointment method is not used in any of the midwest states and in some the states of the southeast and is the lowest in the south and southeast regions There is predominance of “election” (8 states: PA, PI, RN, AL, RJ, PR, RS and MT) and “public exam and election” (8 states: AC, CE, PE, BA, MG, MS, GO and DF), the first being observed across all the regions in the country.

Table 2
Principal selection methods by state (in %) - 2011

State	public exam	election	public tender and election	app. by technical staff	political app.	app. by others	others
North region
Rondônia (RO)	2%	3%	3%	33%	21%	25%	7%
Acre (AC)	1%	15%	71%	3%	0%	4%	4%
Amazonas (AM)	32%	1%	3%	36%	6%	16%	6%
Roraima (RR)	5%	10%	5%	34%	15%	23%	4%
Pará (PA)	4%	29%	16%	8%	5%	16%	8%
Amapá (AP)	1%	1%	1%	13%	44%	26%	6%
Tocantins (TO)	8%	1%	1%	12%	47%	21%	8%
Northeast region
Maranhão (MA)	5%	3%	4%	24%	35%	18%	6%
Piauí (PI)	8%	63%	12%	4%	4%	4%	3%
Ceará (CE)	13%	5%	74%	1%	0%	2%	2%
Rio Grande do Norte (RN)	1%	77%	6%	3%	2%	5%	4%
Paraíba (PB)	2%	32%	9%	3%	42%	7%	2%
Pernambuco (PE)	4%	4%	53%	7%	2%	17%	10%
Alagoas (AL)	1%	83%	4%	7%	1%	2%	1%
Sergipe (SE)	0%	0%	0%	5%	77%	15%	1%
Bahia (BA)	3%	14%	41%	7%	17%	11%	3%
Southeast region
Minas Gerais (MG)	2%	17%	64%	3%	0%	7%	3%
Espírito Santo (ES)	34%	12%	9%	21%	4%	8%	6%
Rio de Janeiro (RJ)	9%	29%	20%	7%	2%	22%	8%
São Paulo (SP)	45%	0%	1%	2%	0%	8%	41%
South region
Paraná (PR)	1%	81%	8%	2%	0%	5%	1%
Santa Catarina (SC)	0%	1%	1%	1%	83%	9%	1%
Rio Grande do Sul (RS)	2%	78%	6%	1%	0%	4%	5%
Midwest region
Mato Grosso do Sul (MS)	2%	21%	63%	4%	0%	4%	4%
Mato Grosso (MT)	1%	57%	16%	5%	0%	9%	3%
Goiás (GO)	2%	20%	68%	2%	0%	4%	1%
Distrito Federal (DF)	6%	8%	67%	5%	0%	8%	5%

Source: Elaborated by authors.

The state schools in Sao Paulo and Espírito Santo use “public exam” (45% and 34%, respectively) and there is a predominance of “appointment by technical staff” in three states, all of them in the north region (RO, AM and RR).

Many state schools changed their method of selecting principals between 2007 and 2011. Table 3 shows the variation in the frequency (in percentage points) of each method by state: A lighter shade of gray indicates a positive variation in the method adopted in that state and a darker shade of gray means a negative variation. The cases that draw more attention are: a) the states which switched to the adoption of “public exam and election” - Goiás, Distrito Federal, Mato Grosso do Sul and Bahia; b) the states in which “political appointment” was abandoned¹⁸ - Espírito Santo, which switched to the “public exam” method, Roraima, which switched to the “election” and “appointment by others” processes, Amapá, which switched to the “appointment by technical staff” method; and c) Tocantins, which discontinued the use of the “public exam” and switched to “political appointment” and Pará, which switched from “technical appointment” to “election”. We use the changes introduced in some states as robustness tests of the results¹⁹.

Table 3
Principal selection methods - variation in percentage points, 2011-2007

State	public exam	election	public tender and election	app. by techincal staff	political app.	app. by others	Number of schools 2011	Number of schools 2007
RO	0.00	-0.06	-0.12	-0.01	0.06	0.11	271	254
AC	0.00	0.09	-0.08	0.00	0.00	-0.01	182	132
AM	-0.13	0.00	0.00	0.10	-0.01	0.03	489	431
RR	-0.01	0.09	0.05	-0.10	-0.11	0.07	108	83
PA	-0.02	0.24	0.09	-0.19	-0.03	-0.10	519	474
AP	-0.01	0.01	0.00	0.06	-0.08	0.01	171	113
TO	-0.42	0.00	-0.03	0.01	0.34	0.08	365	352
MA	-0.04	0.00	0.00	-0.04	0.08	-0.01	553	545
PI	0.07	0.01	-0.05	0.00	-0.07	0.01	380	472
CE	0.02	0.04	-0.07	0.00	0.00	0.02	265	431
RN	0.00	0.14	-0.11	0.02	-0.06	-0.01	442	469
PB	0.00	0.09	-0.01	-0.03	-0.06	0.01	530	583
PE	0.00	-0.01	-0.27	0.05	0.02	0.12	786	820
AL	-0.01	0.09	-0.09	0.01	0.00	0.00	256	265
SE	0.00	0.00	0.00	-0.01	-0.04	0.04	274	270
BA	-0.04	0.13	0.41	-0.02	-0.37	-0.09	808	1077
MG	0.00	0.01	-0.04	0.01	0.00	0.01	3006	3097
ES	0.31	-0.13	-0.01	0.00	-0.09	-0.05	321	282
RJ	0.08	-0.11	-0.10	0.02	0.00	0.09	937	974
SP	-0.05	0.00	0.00	0.00	0.00	0.03	4640	4606
PR	0.00	0.02	-0.04	0.01	0.00	0.01	1609	1351
SC	0.00	0.00	-0.01	-0.02	0.02	0.01	880	806
RS	0.01	-0.08	0.00	0.01	0.00	0.03	1530	1650
MS	0.01	-0.61	0.57	0.03	0.00	-0.02	310	316
MT	0.00	-0.03	0.04	0.03	-0.01	0.00	522	452
GO	0.01	-0.64	0.60	0.01	0.00	0.02	849	860
DF	-0.09	0.08	0.63	-0.17	-0.05	-0.34	460	385

Source: Elaborated by authors.

The principals’ characteristics (both personal and managerial) seems to differ according to the way the principal was selected. Table 4 indicates that the principals chosen by “public exam” and “public exam and election” show better managerial characteristics than those appointed by technical staff or politicians. The managerial skills that we examine include: principal participation in continuing education programs, development of tutoring education programs, exchange of information with other principals, promotion of continuing education programs for teachers, development of programs to reduce dropout and failure rates, and promoting meetings of the teaching advisory and deliberative council (conselho de classe) more than three times a year.

Table 4
Characteristics of principals according to the selection method (in %), 2007 and 2011

Selection process	Principal takes continuing education		Principal promotes continuing teachers' training program		3 or more meetings of school council		Program to reduce the dropout rate		Program to reduce failure rates		Principal develops a support program for students		Info is exchanged with other schools' principals
State schools	2007	2011	2007	2011	2007	2011	2007	2011	2007	2011	2007	2011	2007	2011
public exam	95.2	91.8	62.6	69.2	84.3	83.9	69.2	76.1	85.1	88.3	89.7	94.3	92.9	92.9
election	89.7	90.5	63.9	73.9	85.0	86.8	54.4	56.8	70.9	72	69.6	76.5	93.6	93.4
public tender and election	90.8	89.3	65.3	70.8	82.4	83.8	55.1	63	77.8	83.3	77.1	84.6	95.8	93.4
app. by tech staff	86.6	88.7	57.7	64.9	64.6	67.1	50.8	61.1	71.1	79.2	66.3	79.5	94.0	94.7
political app.	81.6	87.1	49.8	52.6	61.6	66	43.2	51.2	55.9	61.9	47.5	54.8	94.3	94.1
app. by others	86	84.2	54.7	61	70.3	71.1	45.7	55.7	67.2	74.5	66.6	80.1	93.0	94.5

Source: Elaborated by authors.

Finally, with respect to the principal’s term of office according to the method of selection, analysis of the 2011 data (Table 5) reveals that in state schools, the selection methods “appointment by technical staff”, “political appointment” and “appointment by others “ are associated with a higher turnover of principals: 50% of politically appointed principals, 48% selected by “appointment by others” and 47% appointed by “technical staff” remain in office less than two years, while 34% of the principals selected by “public exam”, 27% selected by “public exam and election” and 25% selected by “election” remain in office between 5 to 10 years and 16% selected by “public exam and election” stay in the position between 15 to 20 years.

Table 5
Principals' length of time in the same school (in %), 2007 and 2011.

Principal selection methods	How long have you worked as principal in this school?
	<2 years	2-4 years	5-10 years	11-15 years	+15 years	<2 years	2-5 years	5-10 years	10-15 years	+15 years
	2007					2011
public exam	23.3	35.5	30.2	4.7	5.5	26.9	21.9	32.9	11.4	6.5
election	36.3	30.8	24.4	5.5	2.3	25.9	33.8	24.7	9.7	5.0
public tender and election	22.9	30.0	36.0	7.2	3.2	17.8	33.2	26.5	16.2	5.4
appoitment by tech staff	48.1	23.3	18.9	5.3	3.6	46.5	30.2	12.9	5.3	4.6
political appoitment	49.8	20.8	19.7	4.7	3.9	50.1	26.7	15.0	4.3	3.3
appoitment by others	49.0	23.3	17.6	5.1	4.2	47.6	29.5	12.5	5.2	4.3

Source: Elaborated by authors.

5 Methodology

We aim to understand the impact of the selection process of public school principals on students’ learning and achievement. Possible confounders are unobserved variables of the principals or schools correlated with the selection process. For example, part of the negative relation observed between politically appointed principals and students’ learning can be explained by the higher frequency of this selection process in the northern and northeastern states, where students’ learning rates are lower, on average, than in the southern and southeastern states (Table 6).

Table 6
IDEB score by region - 2007 and 2011

Region	IDEB score
	2007	2011
North	3.6	4.0
Northeast	3.1	3.5
Midwest	4.0	4.6
Southeast	4.1	4.7
South	4.3	4.8

Source: Elaborated by authors.

To mitigate those problems, we test two approaches. The first considers that possible confounders are specific to the schools and remain constant in time (fixed effects), so panel data is used to identify these effects. In this case, unobserved variables, which are believed to be controlled by this approach, would be unobserved characteristics of students, parents, teachers and principals, as well as programs and actions effectively implemented in the school and not identified by the observed variables and the difference in raising funding, among others. In this first approach we use a sample from Brazilian state public schools. The equation we estimate is:

(1) $y_{\mathrm{it}}^d={\sum }_{j\in J}{S}_{\mathrm{it}}^j{\tau }^{j,d}+X_{\mathrm{it}}^A{\beta }^{A,d}+X_{\mathrm{it}}^E{\beta }^{E,d}+X_{\mathrm{it}}^D{\beta }^{D,d}+{\alpha }_i^d+\gamma {}_t{}^d+{\epsilon }_{\mathrm{it}}^d$

in which:

• $y_{\mathrm{it}}^d$ represents the performance measure d for school i in year t;

• d identifies the performance measure used (IDEB, scores in mathematics and Portuguese and dropout rate);

• $S_{\mathrm{it}}^j$ is a binary variable that identifies the selection process (j = “election”, “public exam”, “public exam and selection”, “political appointment”, “appointment by technical staff” and “appointment by others”, apart from “other methods”) in school i in year t;

• $X_{\mathrm{it}}^A,X_{\mathrm{it}}^E,X_{\mathrm{it}}^D$ denote the matrix of characteristics of the students (A), the school (E) and the principal (D) of school i in year t;

• $a_i^d,{\gamma }_t^d$ are the fixed effects of school and year, respectively, for each performance equation;

• τ, β are parameters to be estimated (the comparison of τ brings about the effect intended to estimate in this study); and,

• ε is the idiosyncratic error of the equation.

In this approach, the average effect of the selection process is representative for Brazil and is given by (being {-j} the set complementary to {j} of selection processes):

(2) $E\widehat{\left[y_{\mathrm{it}}^d\left|S^j,X^A,X^E,X^D,\alpha ,\gamma \right.\right]}-E\widehat{\left[y_{\mathrm{it}}^d|S^{-j},X^A,X^E,X^D,\alpha ,\gamma \right]}={\hat{\tau }}^{j,d}$

We estimate Equation (1) by fixed effect for linear regressions when the outcome variable is continuous, and by logit fixed effects when the outcome variable is binary.

The second approach bases its identification on case studies (improving the internal validity but impairing the external validity of the results). We propose a propensity score analysis with difference-in-differences approach, in which the treatment group is composed of public schools in states in which the principal selection method changed between 2007 and 2011 (from process A to process B, for example) and the control group encompasses state schools in states in which there was no change in the selection process in the same period (states that maintained process A).

The problem of endogeneity in this analysis (selection bias) emerges from the fact that schools from states that changed their selection process may have different characteristics from those that did not change and that are related to students’ performance/learning. Therefore, we need to control for those differences to compare “similar” schools within the states that are part of the control and treatment group. We analyze the following cases:

• a) Case 1: The effect of switching from “election” to “public exam and election” in the period between 2007 and 2011. The treatment group is state schools in Goiás²⁰ and Mato Grosso do Sul²¹ and the control group is state schools from Mato Grosso (which maintained the “election” selection process from 2007 to 2011).

• b) Case 2: The effect of switching from “public exam” to “political appointment” between 2007 and 2011. The treatment group is state schools in Tocantins²² and the control group is state schools in Ceará and Amazonas (whose selection process was the same: “public exam”).

• c) Case 3: The effect of switching from “political appointment” to “public exam and election”, with the treatment group of schools in Bahia²³ and a control group composed of schools in Sergipe, Paraíba and Maranhão (which maintained “political appointment” as the selection process).

We exclude from the sample schools whose principal was the same in the period for both comparison groups.

We use the propensity score estimation to weight the schools from the control group in order to improve the estimations, in a procedure called propensity score weighting. The propensity score is defined by a binarychoice model in which the dependent variable refers to the principal selection process in the state chosen.

(3) $P\left(S_i^p=1\left|Z^p\right.\right)=Z_i^p{\delta }^p+{\varepsilon }_i^p$

in which:

• $S_i^p$ refers to the binary variable that identifies the new selection process in the treatment state in schools i during the pretreatment period for the pair of states p (only the schools that employed the same selection process in the pretreatment period and the schools located in one of the states in which the change occurred and intervention took place);

• $Z_i^p$ denotes the matrix of observed variables that determine the selection process in the pretreatment period for schools i;

• δ is a vector of parameters to be estimated; and

• ε is the idiosyncratic error of the equation.

Therefore, the average effect of the change in the selection process can be written by comparing the performance of paired schools in both pretreatment (t₀) and post-treatment (t₁) periods:

(4) $\begin{array}{c}{\hat{\tau }}^{\mathrm{DID},\mathrm{PSW}}=\left(\widehat{E\left[y_{{\mathrm{it}}_1}^d|S=1,P\left(S|Z\right)\right]}-\widehat{E\left[y_{{\mathrm{it}}_1}^d|S=0,P\left(S|Z\right)\right]}\right)-\hfill \\ -\left(\widehat{E\left[y_{{\mathrm{it}}_0}^d|S=1,P\left(S|Z\right)\right]}-\widehat{E\left[y_{{\mathrm{it}}_0}^d|S=0,P\left(S|Z\right)\right]}\right)\hfill \end{array}$

6 Results

We divide this section into: a) estimation of the “selection process effect” using panel data methods for the full sample of schools (POLS and fixed effects) and for a selected sample of schools (propensity score analysis with difference-in-differences) and b) verification of the “principal quality effect” by using the panel data method of logit fixed effects²⁴.

6.1 Selection process effect

Table 7 presents the estimation results of the “selection process effect” on the IDEB rate, and Table 8 does the same for mathematics scores²⁵. Tables 5 and 6 show the results under the POLS²⁶ (columns 1, 3 and 5) and FE (columns 2, 4 and 6) methods as well as the results broken down as follows: a) state schools (columns 1 and 2), b) state schools - 5^th grade (columns 3 and 4), and c) state schools - 9^th grade (columns 5 and 6).

Table 7
Selection process effect - normalized IDEB (2007 and 2011)

Variables	State Schools		State - 5th year		State - 9th year
	POLS	FE	POLS	FE	POLS	FE
	(1)	(2)	(3)	(4)	(5)	(6)
Public Exam	0.0381***	-0.0270	-0.0489***	-0.00563	0.149***	-0.0198
	(0.0124)	(0.0222)	(0.0172)	(0.0309)	(0.0166)	(0.0289)
Election	-0.0203*	-0.0722***	0.00288	-0.0586*	-0.0233*	-0.0580*
	(0.0106)	(0.0234)	(0.0150)	(0.0335)	(0.0139)	(0.0300)
Public Tender and Election	0.150***	-0.0617***	0.190***	-0.0241	0.141***	-0.0676**
	(0.0112)	(0.0228)	(0.0156)	(0.0321)	(0.0147)	(0.0296)
Appoitment by Technical Staff	0.00197	-0.0714***	-0.0199	-0.0627*	0.0529***	-0.0458
	(0.0150)	(0.0243)	(0.0208)	(0.0327)	(0.0200)	(0.0323)
Appoitment by others	0.0273**	-0.0259	0.00655	-0.0237	0.0753***	0.00586
	(0.0131)	(0.0214)	(0.0182)	(0.0290)	(0.0173)	(0.0284)
Year Dummy	Yes	Yes	Yes	Yes	Yes	Yes
Student Controls[1]	Yes	Yes	Yes	Yes	Yes	Yes
Schools Controls [2]	Yes	Yes	Yes	Yes	Yes	Yes
Teacher Controls [3]	Yes	Yes	Yes	Yes	Yes	Yes
Principal Controls [4]	Yes	Yes	Yes	Yes	Yes	Yes
Observations	49,134	49,134	18,319	18,319	30,815	30,815

• ^***
  p<0.01,
• ^**
  p<0.05,
• ^*
  p<0.1
• ^[1]
  Students per class, students TDI, percentage of women, percentage of black, percentage of students who own computer and internet, live with their mother and/or father, parents' educational level, encouragement from the parents, percentage of students who work, percentage of students who started studying at preschool (2-6 yr olds), students' absenteeism.
• ^[2]
  Condition of classrooms and restrooms, safety conditions of school, inflow and outflow of students, signs of deliberate damage at school, existence of a library in the school, logarithm of the total employees, surveillance service in the school and time frame of classes (full-time, part-time and morning or evening classes).
• ^[3]
  Teachers characteristics: race, education, taken continuing education, teachers' salary, have other jobs (related or not to teaching practice), experience in teaching, experience in the school, excessive absenteeism, whether teachers have been assaulted.
• ^[4]
  Sex, educational level and ethnicity of the principal, principal's experience in that position and in the school, principals motivation of teachers, whether they encourage innovation, continuing education, and students' learning, information on whether the principal helps with the textbook in the school, foreign interference in the management and number of meetings of the teaching advisory and deliberative council (Conselho de Classe) and School Council.
Source: Elaborated by authors.

From Table 7 we observe that: a) in the state schools and 9^th grade in which the principal was selected by “public exam”, the average IDEB tends to be higher compared to the “political appointment” method (base method) in the POLS results. However, for the 5^th grade the result is the opposite, indicating that the “public exam” method is related to lower IDEB rates when compared to “political appointment” process, also using POLS estimation. Besides the mixed signs with POLS estimation, the fixed effect method did not present significant results, making it difficult to reach conclusions about the relation between the IDEB rate and the “public exam” selection method in comparison to “political appointment”; b) the same happens to the “public exam and election” method -it is positively correlated to IDEB rates with POLS estimation and negatively correlated to IDEB rates with FE estimation to state schools and 9^th grade, when compared to “political appointment”; c) the “election” method showed significant results for state schools, 9^th and 5^th grades for both POLS and FE estimations. In all the cases, the average impact on IDEB rate was worse in schools that selected their principals by “election” when compared to schools whose principal were politically appointed; d) the average impact of using the “appointment by technical staff” selection process significantly decreases the IDEB rate in state schools and 5^th grade (both under FE estimation) and increases it in the 9^th grade (POLS estimation), in comparison to schools with politically appointed principals; and e) the “appointment by others” method also showed significant (positive) results only with POLS estimation for state schools and 9^th grade.

Therefore, we observe that although the “election” process registered a negative average impact on IDEB rate in all specifications, no assertive conclusion can be drawn with regard to the direction of the impact of the various selection methods.

In relation to students’ performance in mathematics (Table 8), the results are mixed: depending on the estimation method, the signs of the impact alternate for all selection processes except “appointment by others”, which shows a positive average effect on mathematics scores (with POLS estimation) in state schools, 5^th and 9^th grades in comparison to “political appointment”. It is noteworthy that even the “election” method, which registered negative correlation with the IDEB rate, presents an unclear impact on mathematics score, being positively correlated to mathematics score with POLS estimation and negatively correlated with FE estimation, in all specifications (state schools, 5^th and 9^th grades).

Therefore, the evidence suggests that the “selection process effect” on students’ performance is not conclusive, since it is possible to find better and/or worse results (for IDEB and mathematics) by selection method compared to political appointment for all specifications examined. In a complementary analysis, we study the states where the legislation was amended and apply a differences-in-differences model using the propensity score.

Tables 7, 8 and 9 summarize the estimations of the effect of the change on the treated group. In relation to students’ average performance in Case 1 (Table 9): a) The results for the 5^th grade are quite similar to the estimates for the state schools in Goiás and Mato Grosso do Sul, i.e., they show statistically significant positive results of the selection process switching from “election” to “public exam and election”, as observed through Portuguese and mathematics scores. The results for IDEB and dropout rate also indicate the same in both analyses, although significance was swapped. b) The results of the “public exam” process change for the Portuguese and mathematics scores, which are now significant in the differences-in-differences (DID) analysis for the 9^th grade, but lower in magnitude when compared to the results for the 5^th grade. As for dropout rate, the results of the fixed effects model are statistically equal to zero, while for the DID effect the results for the change are negative (positive for dropout rate).

Table 8
Selection process effect - average normalized score in mathematics (2007 and 2011)

Variables	State Schools		State - 5th year		State - 9th year
	POLS	FE	POLS	FE	POLS	FE
	(1)	(2)	(3)	(4)	(5)	(6)
Public Exam	-0.0284**	-0.0457**	-0.0788***	-0.0420	0.0307*	-0.0245
	(0.0137)	(0.0226)	(0.0195)	(0.0346)	(0.0181)	(0.0269)
Election	0.157***	-0.0969***	0.0831***	-0.109***	0.194***	-0.0684**
	(0.0112)	(0.0228)	(0.0160)	(0.0369)	(0.0147)	(0.0269)
Public Tender and Election	0.288***	-0.0409*	0.243***	-0.00317	0.317***	-0.0495*
	(0.0120)	(0.0222)	(0.0170)	(0.0356)	(0.0156)	(0.0262)
Appoitment by Technical Staff	0.0549***	-0.0601**	0.0168	-0.0577	0.0989***	-0.0235
	(0.0156)	(0.0241)	(0.0217)	(0.0369)	(0.0207)	(0.0291)
Appoitment by others	0.0878***	-0.0337	0.0469**	-0.00332	0.129***	-0.00751
	(0.0143)	(0.0215)	(0.0196)	(0.0329)	(0.0189)	(0.0260)
Other selection methods	-0.0545***	-0.0337	-0.0967***	-0.0202	-0.00306	-0.0160
	(0.0141)	(0.0230)	(0.0205)	(0.0359)	(0.0184)	(0.0270)
Year Dummy	Yes	Yes	Yes	Yes	Yes	Yes
Student Controls[1]	Yes	Yes	Yes	Yes	Yes	Yes
School Controls[2]	Yes	Yes	Yes	Yes	Yes	Yes
Teacher Controls[3]	Yes	Yes	Yes	Yes	Yes	Yes
Principal Controls[4]	Yes	Yes	Yes	Yes	Yes	Yes
Observations	49,200	49,200	18,345	18,345	30,855	30,855

• Robust Standard Errors in parentheses
• ^***
  p<0.01,
• ^**
  p<0.05,
• ^*
  p<0.1.
• Same notes as Table 7.
Source: Elaborated by authors.

Table 9
Selection process effect - DID with propensity score weighting, Case 1

Outcome	Case 1: Election to Election + Selection[3]
	9th grade[1]	5th grade[2]
IDEB rate	-0.245***	0.295***
Portuguese score	0.322***	0.639***
Mathematics score	0.377***	0.721***
Dropout rate	0.392***	0.331**

• ^[1]
  341 schools in the treatment group and 80 in the control group.
• ^[2]
  126 schools in the treatment group and 71 in the control group.
• ^[3]
  Covariates: number of employees, teacher absence, library at school, number of school council meetings, year dummy, and school fixed effects.
Source: Elaborated by authors.

We find the following evidence in Case 2 (Table 10): a) The results observed for the 5^th grade are quite similar to the estimated results for the state schools in Tocantins. The selection process negatively affects the students’ average performance. b) The results of the selection process changing to “political appointment” are not significant for all specifications with the exception of the score in Portuguese for students in the 5^th grade. By estimating using DID, we identify that the change has a positive impact on the students’ performance on the Portuguese exam (different from the results observed by the fixed effects model, although the latter was statistically non-significant).

Table 10
Selection process effect - DID with propensity score weighting, Case 2

Outcome	Case 2: Selection to Political Appointment[3]
	9th grade[1]	5th grade[2]
IDEB rate	-0.461***	-0.585***
Portuguese score	0.232	0.422***
Mathematics score	0.301	-0.200
Dropout rate	0.468**	0.730***

• ^[1]
  28 schools in the treatment group and 36 in the control group.
• ^[2]
  30 schools in the treatment group and 38 in the control group.
• ^[3]
  Covariates: number of employees, teacher absence, library at school, number of school council meetings, year dummy, and school fixed effects.
Source: Elaborated by authors.

The results for the students’ average performance in Case 3 (Table 11) in comparison with the regressions by state show the following. a) The results of the selection process changing from “political appointment” to “public exam and election” are positive and significant for mathematics scores in 5^th grade in both specifications. b) With respect to the dropout rate, the results show a reduction when the political appointment is replaced by public exam and election.

Table 11
Selection process effect - DID with propensity score weighting, Case 3

Outcome	Case 3: Political Appointment to Selection + Election[3]
	9th grade[1]	5th grade[2]
IDEB rate	-0.190	0.395
Portuguese score	-0.238	-0.0262
Mathematics score	-0.115	0.782**
Dropout rate	-0.308	-2.403**

• ^[1]
  96 schools in the treatment group and 36 in the control group.
• ^[2]
  7 schools in the treatment group and 13 in the control group.
• ^[3]
  Covariates: number of employees, teacher absence, library at school, number of school council meetings, year dummy, and school fixed effects.
Source: Elaborated by authors.

Although some of the results are consistent with the estimated effects by the fixed effects model, there appears to be little consensus on the direction of the “selection process effect” in state schools (some evidence of the negative effect of political appointments and positive effects of election with public exam). In this sense, we run some tests to check if the effecttransmission mechanism is given indirectly, i.e., by selecting a principal with different characteristics.

6.2 Principal quality effect

We use fixed effects model to evaluate the “principal quality effect”, or to identify which characteristics of the principal (both personal and managerial) influence (positively or negatively) the students’ performance. Thus, we present in Table 12 the average effect of principal characteristics (that are statistically relevant to school performance²⁷) on IDEB rate (column 1), mathematics and Portuguese scores (columns 2 and 3 respectively) and dropout rate (column 4).

Table 12
Principal quality effect (2007 and 2011)

Principal Variables	IDEB	Math	Portuguese	Dropout
	FE	FE	FE	FE
	(1)	(2)	(3)	(4)
Female	-0.0106	0.00136	-0.00140	-0.0103
	(0.0103)	(0.0105)	(0.0101)	(0.0152)
Teacher Educational Program (High School) - Magistério	0.0652	0.143***	0.192***	-0.00305
	(0.0507)	(0.0483)	(0.0480)	(0.0799)
High School	-0.0468	0.0961	0.136**	-0.108
	(0.0567)	(0.0602)	(0.0587)	(0.0871)
Teacher Educational Program (College)	0.000598	0.00976	0.0626*	-0.00871
	(0.0349)	(0.0345)	(0.0336)	(0.0547)
Black	-0.0213	-0.0277	-0.0183	-0.0121
	(0.0172)	(0.0170)	(0.0170)	(0.0268)
Works as principal for > 15 years	0.0198	0.0389**	0.0431**	-0.0180
	(0.0183)	(0.0191)	(0.0184)	(0.0284)
Works in the same school between 2 and 4 years	-0.0138	-0.0170	-0.0138	-0.00622
	(0.0114)	(0.0118)	(0.0113)	(0.0174)
Works in the same school between 5 and 10 years	0.00432	-0.00964	0.00245	0.0219
	(0.0125)	(0.0131)	(0.0125)	(0.0188)
Works in the same school between 11 and 15 years	0.0712***	0.0600***	0.0571***	0.0547*
	(0.0186)	(0.0194)	(0.0187)	(0.0284)
Works in the same school > 15 years	0.0101	-0.0235	-0.0220	0.00990
	(0.0246)	(0.0261)	(0.0257)	(0.0394)
Promotes continuing teachers' training programs	0.0299***	0.0271***	0.0274***	0.0203*
	(0.00800)	(0.00830)	(0.00806)	(0.0120)
Year dummy	Yes	Yes	Yes	Yes
Number of Observations	49,134	49,200	49,200	49,123

• Robust Standard Errors in parentheses
• ^***
  p<0.01,
• ^**
  p<0.05,
• ^*
  p<0.1

The results in Table 12 show that the principals’ characteristics with the strongest impact on the students’ average performance are as follows. a) Principal who works in the same school between 11 to 15 years has a very significant positive effect in all the specifications analyzed²⁸ (IDEB, mathematics score, Portuguese score and dropout rate). b) Principal who promotes teachers’ continuing training programs also has a very significant positive impact on the average student performance in all the aspects mentioned (IDEB, mathematics score, Portuguese score and dropout rate). c) The characteristics that also have a positive impact but with lesser relevance include holding a diploma from a high school teacher training program (mathematics and Portuguese scores) and principal who works in the position (not in the same school) for more than 15 years (mathematics and Portuguese scores). d) The principal with high school diploma and teacher educational program (college degree) has a positive impact on Portuguese scores.

On the basis of the statistically significant characteristics of the principal profile listed above (works in the same school between 11 to 15 years, promotes teachers’ continuing training programs, high school teacher training program and is principal for more than 15 years (not in the same school), we estimate the relationship between the different principal selection methods and such characteristics. To do so, we use the logit fixed effects model (Table 13) broken down by state school (columns 1, 4, 7 and 10), 5^th grade (columns 2, 5, 8 and 11) and 9^th grade (columns 3, 6, 9 and 12).

Table 13
Principal quality effect - main characteristics of the principal (2007 and 2011)

Variables	Teacher education program (magistério)			Principal n the same school between 11 and 15 years			Princ. promotes continuing educ. programs to the teachers			Principal for more than 15 years
	state schools	5th year	9th year	state schools	5th year	9th year	state schools	5th year	9th year	state schools	5th year	9th year
	(1)	(2)	(3)	(7)	(8)	(9)	(10)	(11)	(12)	(13)	(14)	(15)
Public Exam	-0.648	-0.829	-0.472	0.497	0.475	0.509	0.545***	0.649***	0.491***	0.450*	0.343	0.494*
	(0.539)	(0.772)	(0.663)	(0.339)	(0.475)	(0.350)	(0.101)	(0.169)	(0.139)	(0.248)	(0.429)	(0.284)
Election	-0.529*	-0.627	-0.525	1.366***	1.451***	1.348***	0.637***	0.592***	0.650***	0.216	0.198	0.245
	(0.303)	(0.538)	(0.411)	(0.295)	(0.536)	(0.363)	(0.111)	(0.210)	(0.148)	(0.260)	(0.428)	(0.294)
Public Tender and Election	-0.573**	-0.948	-0.497	1.425***	1.322***	1.498***	0.846***	0.947***	0.796***	0.333	0.477	0.279
	(0.264)	(0.635)	(0.319)	(0.317)	(0.483)	(0.337)	(0.0935)	(0.188)	(0.151)	(0.218)	(0.397)	(0.270)
Appoitment by Technical Staff	0.0892	-0.0301	0.0707	0.406	0.530	0.317	0.435***	0.463**	0.417***	-0.0462	0.170	-0.137
	(0.270)	(0.494)	(0.401)	(0.303)	(0.515)	(0.342)	(0.0987)	(0.185)	(0.140)	(0.259)	(0.471)	(0.271)
Appoitment by others	-0.420	-0.596	-0.415	-0.0989	0.118	-0.235	0.124	0.223	0.0668	-0.496**	-0.336	-0.570**
	(0.288)	(0.543)	(0.327)	(0.264)	(0.490)	(0.340)	(0.0919)	(0.142)	(0.141)	(0.209)	(0.376)	(0.283)
Year dummy	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Observations	794	302	492	5,266	1,846	3,420	14,304	4,662	9,642	4,628	1,514	3,114

• Standard errors in parentheses. All regressions were estimated by Logit FE.
• ^***
  p<0.01,
• ^**
  p<0.05,
• ^*
  p<0.1.
Source: Elaborated by authors.

Table 13 shows. a) The methods of “election” and “public exam and election” are the ones with the highest probability of selecting stable principals (who stay in the position in the same school between 11 and 15 years). Principals who are appointed politically remain in their schools for less time (thus impairing the students’ performance) compared to the “election” and “public exam and election” methods. b) The methods of “election”, “public exam and election”, “public exam” and “appointment by technical staff” are the ones with the highest probability of selecting principals who promote continuing educational training programs for teachers. Politically appointed principals give less encouragement to continuing the training of their teachers compared to all the previous methods of principal selection. c) The method of “public exam and election” presents a higher probability to select principals that remain in the position for more than 15 years (not in the same school) when compared to the “political appointment” method. On the other hand, the probability of a politically appointed principal staying in the position for more than 15 years is higher compared to the “appointment by others” method. d) On average, the probability that a politically appointed principal has graduated from a high school teacher training program is higher when compared to the “election” and “public exam and election” methods.

These results show that different selection methods seem to choose principals with different average characteristics/qualities (such as experience - practice and time, incentive to continuing training). Moreover, we can conclude that the “principal quality effect’ is favored when the principal is selected through more democratic/technical methods than when using “political appointment”.

7 Final remarks

Brazil is still moving at a slow pace as far as quality of education is concerned. Thus, the major current challenge of Brazilian education, especially in public schools, is how to improve quality, particularly by improving basic education, due to its importance for the future performance of individuals. The discussion of the role played by principals on educational quality is still incipient in the country. There are few studies in the international literature that discuss the importance of principals’ personal traits on students’ achievement. Considering the potential of the principal and the potential of this low-cost public policy, continued research on the topic can provide evidence to educational leaders and policymakers to improve the principal selection methods and ultimately raise student achievement in public schools.

In this study, we investigated how the different methods of selecting principals impact the performance of students in the 5^th and 9^th grades enrolled in state public schools in Brazil. We tested two assumptions: the “selection process effect” and the “principal quality effect”. We also tested two models to identify the first effect and results were mixed (5^th and 9^th grades). There was weak evidence that politically appointed or elected principals negatively influence students’ performance. This evidence might be a result of the high turnover observed for these selection methods. Although specific results seem interesting, we cannot conclude that any of the different selection methods analyzed is the best, as we are analyzing more correlations than causal effects. Therefore, we conducted the second analysis to check how the principals’ characteristics influence students’ achievements, or the “principal quality effect”.

By evaluating the results of the impact of the principals’ characteristics on performance, we found that the one positive impact is when the principal works in the position in the same school between 11 and 15 years. In this case, we understand that it takes time for the principal become acquainted with the reality of the school, to gain experience in the management approach and to implement major measures to improve the students’ performance (goal-based actions, other forms of incentives). In this sense, the average time of service seems to be an extremely positive factor in the students’ school performance.

Another relevant characteristic is the promotion of teachers’ continuing training programs. Although the educational literature does not indicate any consensus on the necessary training and skills to be a good teacher or on how to develop such skills, the fact that the principal encourages teachers seems to reveal important management characteristics to improve students’ performance, either due to more active leadership or to a continuous concern with teacher training.

Finally, the information about principals’ academic background shows that the principals who have graduated from a high school teacher training program positively influence the performance of students in schools. This result, although surprising, is corroborated by several managers of public education, owing to the closeness between this sort of background and the hands-on and instructional issues related to learning. There is a debate in the educational literature about the excessive formalism in the educational theory and practice courses in the country, which may not be as effective as the high school teacher training programs.

When verifying whether the principal selection methods affect the selection of principals with the positive characteristics mentioned above, we found consistent results. More transparent/technical selection processes (linked to “public exam” and/or “election” and “appointment by technical staff”) relate positively with most positive characteristics: principals who remain much longer in the schools and have positive leadership characteristics (as identified by the encouragement of teachers’ continuing education). In contrast, the principals chosen by “political appointment” do not have these characteristics. Just as a side comment on principals’ academic background, the selection-by-appointment processes gains an advantage when generally choosing more principals graduated from high school teacher training programs, which positively impacts the students’ performance.

We conclude that while the “selection process effect” is not clear in terms of defining which selection process creates the best environment for principals to develop their activities, the “principal quality effect” indicates that the more democratic and technical selection methods tend to select most qualified principals in terms of leadership and managerial abilities. Moreover, our results corroborate the international literature on the importance of school management for students’ performance: the ideal principal is one who adopts management practices in order to encourage teachers and has experience being the leader of the school. Therefore, in spite of the few existing incentives in Brazil for the position of principal (low wages, overload of duties and so forth), it is of the utmost importance for the principal selection process to choose professionals with the skills mentioned.

• 1
  Todos pela Educação (2014).
• 2
  The quality of education in the country improved slower than nations such as South Korea and Singapore, whose educational level in the 1960s was ranked alongside Brazil and which today have high educational quality measures.
• 3
  The OECD countries spend an average of 5.6% of GDP on education (OECD, 2014).
• 4
  Public exams include also other types of exams, depending on the rules from each public administration unit.
• 5
  A process that resembles political appointment in Brazil.
• 6
  Basic Education Development Index calculated by the Ministry of Education.
• 7
  Brazil has (often overlapping) public school systems at the “fundamental” level (through ninth grade) and secondary level run by the state and municipal governments.
• 8
  More recently, the approval of the National Educational Plan in 2014 (Law 13,005, target 19) states that the selection of school principals should consider merit and performance criteria and also consultation with the local community (teachers, students and parents). It also states that a national test should be adopted to select the candidates, therefore combining different selection methods to create a more democratic environment.
• 9
  Standardized national test applied to the 5^th and 9^th grades of Brazilian schools, conducted by Ministry of Education.
• 10
  National System of Evaluation of Brazilian Basic Education, conducted by Ministry of Education.
• 11
  A charter school is a public school managed independently, ensuring operational flexibility. The admission is by lottery, so authors use this characteristic to form the control group (students not admitted).
• 12
  The five index practices were suggested by over 40 years of qualitative research.
• 13
  The 2009 data could not be used due to a high rate of missing values in the questions under “Questionnaire of the Principal” in most states.
• 14
  This evaluation was carried out in schools with at least 20 students enrolled in the grades evaluated.
• 15
  In the period under analysis, there was a process of municipalization of the fundamental education schools. In 2007, there were 12,114 state schools that provided at least 5^th grade schooling, and in 2011, this number dropped to 10,026. On the other hand, the number of municipal schools providing the 5^th grade went from 23,928 in 2007 to 30,355 in 2011. With regard to the 9^th grade, the number of state schools remained almost the same between 2007 and 2011, while municipal schools jumped from 9,372 in 2007 to 13,873 in 2011.
• 16
  Question from the Principals’ Questionnaire: “You took over as principal of this school by: (A) Selection, (B) Election only, (C) Selection and Election, (D) Appointment by Technical Staff, (D) Appointment by Politicians, (F) Other Referrals, (G) Otherwise.” It is worth pointing out that the questionnaire is self-populated by school principals.
• 17
  The “other methods” are not specified, therefore, we did not analyze it.
• 18
  Bahia also decreased the adoption of “political appointment” in favor of “public exam and election”.
• 19
  Using difference-in-differences and propensity score estimation (Tables 7 to 9).
• 20
  Ministry Directive (Portaria) 2,783 of 2011, which establishes, inter alia, the selection process of school officers in state schools in Goiás.
• 21
  Law 3,479 of 20 December 2007, which establishes the selection process of school officers in state schools in Mato Grosso do Sul.
• 22
  We did not find the legal grounds supporting the switching of the selection process in the state.
• 23
  Decree 11,218 of 18 September 2008, which sets forth the criteria and procedures of the internal selection process to be conducted by the school for filling the posts of principal and vice principal in state schools in Bahia.
• 24
  At this stage, the results were estimated in order to identify the characteristics of principals with the strongest impact on students’ learning. Based on these characteristics, it was possible to see how different principal selection methods impact the selection of better or worse principals.
• 25
  Portuguese score and dropout rate were also scrutinized, and their results were similar to the IDEB rates and mathematics scores. These results are available upon request.
• 26
  We present the POLS estimates because they demonstrate the need for control by other covariates and by specific effects of schools in order to identify the desired effect.
• 27
  In addition to the principals’ characteristics in Table 12 (and Table 13), tests were run for the following characteristics, which showed no (or low) statistical relevance: principal holding degree in pedagogy (educational theory and practice) or other teacher education programs, principals’ participation in continuing education, exchange of information with principals from other schools, textbooks selected by the principal, meetings of the teaching advisory and deliberative council (Conselho de Classe) or school council, promotion of remedial education, a principal that encourages the introduction of innovations and so forth.
• 28
  Including 5^th and 9^th grade specifications, not shown in Table 12. These results are available upon request.

References

• BARROS, R. P. and MENDONÇA, R., 1998. “O Impacto de Três Inovações Institucionais na Educação Brasileira”. Texto para discussão IPEA 0566.
• BÉTEILLE,T., Kalogrides, D. and Loeb, S. (2012), Stepping stones: principal career paths and school outcomes, Social Science Research, vol.41(4), pp. 904-19.
• BLOOM, N.; LEMOS, R.; SADUN, R.; VAN REENEN, J., 2015. “Does Management Matter in Schools”. The Economic Journal, vol 125, issue 584: 647-674.
• BRANCH, G. F., HANUSHEK, E. A. and RIVKIN, S. G. (2012), Estimating the effect of leaders on public sector productivity: the case of school principals’, Working Paper No.17803, National Bureau of Economic Research.
• CLARK,D., MARTORELL,P. and ROCKOFF, J. E. (2009), School Principals and School Performance, Working Paper No.38, Center for Analysis of Longitudinal Data in Education Research.
• COELLI, M.; GREEN, D. A., 2012. “Leadership effects: school principals and student outcomes”. Economics of Education Review, Vol 31.
• DOBBIE, W.; FRYER, R., 2013. “Getting Beneath the Veil of Effective Schools: Evidence from New York City”. American Economic Journal: Applied Economics 5(4): 28-60.
• EBERTS, R.; STONE J., 1988. “Student achievement in public schools: Do principals make a difference?” Economics of Education Review, vol 7(3).
• FERNANDES, R.; SOUZA, A. P. F.; BOTELHO, F.; SCORZAFE, L. G. (org.), 2014. Políticas Públicas Educacionais e Desempenho Escolar dos Alunos da Rede Pública de Ensino Ribeirão Preto, SP: Funpec-Editora.
• FRYER, R. G. (2014), Injecting charter school best practices into traditional public schools: evidence from field experiments. Quarterly Journal of Economics, vol. 129(3), pp. 1355-407.
• FUNDAÇÃO VITOR CIVITA, 2011. Mapeamento das práticas de seleção e capacitação de diretores escolares. Estudos e Pesquisas Educacionais Website: http://www.fvc.org.br/pdf/selecao-capacitacao-diretores-relatorio-final.pdf Accessed on 25 Sep 2014.
  » http://www.fvc.org.br/pdf/selecao-capacitacao-diretores-relatorio-final.pdf
• MARIONI, L.; FREGUGLIA, R. S.; COSTA, A. B. M., 2014. “Impacts of school management on educational development: a longitudinal analysis from the teacher’s perspective”. Natal, Associação Nacional dos Centros de Pós Graduação em Economia (ANPEC) Disponível em: <https://www.anpec.org.br/encontro/2014/submissao/files_I/i12-38f0bce5d7a7feb0e96e5455e4eaaab4.pdf>.
  » https://www.anpec.org.br/encontro/2014/submissao/files_I/i12-38f0bce5d7a7feb0e96e5455e4eaaab4.pdf
• MIRANDA, J. G..; PAZELLO, E. T., 2014. “Rotatividade de Diretores e Desempenho da Escola”. Natal, Sociedade Brasileira de Econometria (SBE) Disponível em: <https://editorialexpress.com/cgi-bin/conference/download.cgi?db_name=SBE36&paper_id=171>.
  » https://editorialexpress.com/cgi-bin/conference/download.cgi?db_name=SBE36&paper_id=171
• OECD, 2014. Education at a Glance 2014: OECD Indicators, OECD Publishing. Disponível em: <http://dx.doi.org/10.1787/eag-2014-en>.
  » http://dx.doi.org/10.1787/eag-2014-en
• TODOS PELA EDUCAÇÃO, 2014. Anuário Brasileiro da Educação Básica São Paulo: Editora Moderna. Disponível em: <http://www.moderna.com.br/lumis/portal/file/fileDownload.jsp?fileId=8A8A8A824614522A01462512381317E1>.
  » http://www.moderna.com.br/lumis/portal/file/fileDownload.jsp?fileId=8A8A8A824614522A01462512381317E1
• WORLD MANAGEMENT SURVEY RESEARCH. 2014. “Management in Brazilian Schools: Describing management practices in the public and private schools in Brazil”.
• XAVIER, A. C. R.; SOBRINHO, J. A; MARRA, F., 1994. “Gestão escolar: desafios e tendências”. Brasília: IPEA (Série IPEA, 145)

Publication Dates

• Publication in this collection
  10 Oct 2019
• Date of issue
  May-Aug 2019

History

• Received
  27 July 2017
• Accepted
  18 Dec 2018