Transparency, corruption, and mismanagement: an analysis of Brazilian municipalities

Abstract

Does institutional commitment to public transparency reduce corruption and public mismanagement? The regulation of freedom of information laws (FOI) reduces the cost of accessing public information and the control of government officials over it, allowing for monitoring of the government. Thus, greater transparency improves government performance. To test the hypothesis, we used linear regression models for 320 Brazilian municipalities. We employ three measures of municipality institutional commitment to public transparency: FOI regulation in the municipality; the degree to which the local level regulation was based on the national law’s provisions; and time the municipality took to adhere to the legislation, as a measure of institutionalization of public transparency. As a performance measure, we consider corruption and mismanagement irregularities identified in the audit reports of the Comptroller General of Brazil (2011-2015), with classification using unsupervised machine learning. The results are threefold: the municipalities’ adherence to FOI is still very low; in the municipalities that regulated the law locally, transparency is not seen to be associated with government performance; only economic development and quality of bureaucracy have a negative correlation with the number of irregularities. These results indicate that the effort towards more transparent management has not yet generated the expected results.

Keywords:
transparency; freedom of information law; corruption; mismanagement; local government

Artigo

Compromisso institucional com a transparência pública tem relação negativa com corrupção e má gestão governamental? A instituição de Lei de Acesso à Informação (LAI) reduz o custo de acesso à informação pública e o controle dos governantes sobre ela, possibilitando o monitoramento do governo. Assim, maior transparência melhoraria a performance governamental. Para testar a hipótese, usamos modelos de regressão linear para 320 municípios brasileiros. Empregamos 3 medidas do compromisso institucional dos municípios com a transparência pública: a) a regulamentação da LAI no município; b) o grau de regulamentação com base nos dispositivos da LAI em nível local; e c) o tempo de adesão como medida de institucionalização da transparência pública. Como medida de performance, consideramos irregularidades de corrupção e má gestão identificadas nos relatórios de auditorias da Controladoria-Geral da União via sorteios públicos (2011-2015), com classificação usando aprendizado de máquina não supervisionado. Os resultados da análise são 3: a) a adesão dos municípios à LAI ainda é muito reduzida; b) nos municípios que aderiram, transparência não apresenta associação com a performance governamental; e c) apenas desenvolvimento econômico e qualidade da burocracia apresentam correlação negativa com o número de irregularidades. Os resultados indicam que o esforço na direção de uma gestão mais transparente ainda não gerou os resultados esperados.

Palavras-chave:
transparência; lei de acesso à informação; corrupção; má gestão; governo local

Resumen

¿El compromiso institucional con la transparencia pública reduce la corrupción y la mala gestión del gobierno? La institución de la Ley de Acceso a la Información (LAI) reduce el costo de acceso a la información pública y el control de los funcionarios del gobierno sobre ella, lo que permite que el gobierno sea monitoreado. Por lo tanto, una mayor transparencia mejoraría el desempeño del gobierno. Para probar la hipótesis, usamos modelos de regresión lineal en 320 municipios brasileños. Empleamos tres medidas del compromiso institucional de los municipios con la transparencia pública: la regulación de la LAI en el municipio; el grado de regulación basado en las disposiciones de la LAI a nivel local; y el tiempo de adhesión como medida de institucionalización de la transparencia pública. Como medida de rendimiento, consideramos las irregularidades de corrupción y mala gestión identificadas en los informes de auditoría de la Contraloría General de la Federación a través de sorteos públicos (2011-2015), con clasificación mediante aprendizaje automático no supervisado. Los resultados del análisis son tres: la adhesión de los municipios a la LAI todavía es muy baja; en los municipios que se adhirieron la transparencia no está asociada al desempeño del gobierno; y solo el desarrollo económico y la calidad de la burocracia tienen correlación negativa con el número de irregularidades. Los resultados indican que el esfuerzo hacia una gestión más transparente aún no ha generado los resultados esperados.

Palabras clave:
transparencia; ley de acceso a la información; corrupción; mala gestión; gobierno local

1. INTRODUCTION

Does the institutional commitment to public transparency reduce corruption and government mismanagement? There is a theoretical and practical expectation that transparency would increase participation, reduce corruption, and improve government performance (Calland & Bentley, 2013; Cucciniello, Grimmelikhuijsen, & Porumbescu, 2016; Fox, 2007; Meijer, Curtin, & Hillebrandt, 2012; Prat, 2005). This issue has attracted the attention of scholars, guided the activities of freedom of information advocates, and has been used to justify international reforms and initiatives (Bauhr & Grimes, 2012; Calland & Bentley, 2013; Fox, 2007). This study addresses how Brazilian municipalities set up the rules to implement the federal freedom of information law (LAI) (Lei n. 12.527 [LAI], LAI (2011)). The research focuses on a specific type of transparency related to the freedom of information regime (Heald, 2006; Kosack & Fung, 2014). Although 119 countries have already adopted some type of freedom of information regime (Freedom of Information, 2019), the effects of this policy are mixed, and its impact still needs extensive assessment (Calland & Bentley, 2013; Cucciniello et al., 2016; Michener, 2019).

The Brazilian freedom of information law (LAI, 2011) is a federal law with effects valid for all public administration bodies at federal, state, or municipal level. However, states and municipalities have to autonomously set up the rules regarding the local implementation of the law, regulating the channels through which accessing information, the instruments of passive and active transparency, and secrecy procedures. Therefore, it is possible to say that state and municipality regulations represent the formal commitment of the federative entities to the public transparency that the national freedom of information law stimulates. More importantly, the analysis of how these entities - particularly municipalities - engage in regulating the LAI allows identifying its diffusion nationwide and the tangible results of the adoption of transparency practices in the local public management performance. Thus, this study considers the local regulation of the LAI (2011) as the formalization of the municipalities’ institutional commitment with the national legal provisions designed to guarantee the quality and access of information about the government’s actions.

Freedom of information is often associated with fighting corruption and improving public management. According to the guidelines of the LAI (2011) for states and municipalities “access to public information is an important requirement for the fight against corruption, the improvement of public management, social accountability, and popular participation” (Controladoria-Geral da União [CGU], 2013, p. 6, our translation).

The purpose of this article is to test the argument that greater institutional commitment to transparency is associated with the reduction of mismanagement and corruption in Brazilian municipalities. Although intuitively obvious, the relationship between transparency and public management performance is not automatic, and there are arguments that transparency alone is not enough to reduce corruption (Bahur & Grimes, 2012; Fox, 2007). Therefore, this study intends to use a direct measure of corruption and mismanagement based on irregularities detected by the General-Comptroller of the Union (CGU). The study used a database formed by irregularities the CGU considers as mismanagement and corruption, identified in its program of randomly auditing municipalities.

The study establishes that the difference between corruption and mismanagement lies in the manager’s direct gain. When irregularities are associated with fraud in public procurement processes, they are classified as corruption. When the irregularities are related to acts that do not involve the manager’s gain, such as waste of medication or lack of control of school attendance, it is classified as mismanagement. The research identified more than 16,000 irregularities in the audits carried out by CGU in 320 municipalities. The period of analysis was from the year the LAI was enacted, in 2011, until 2015, the last year in which the CGU used the random methodology to select the municipalities to be audited (the change of methodology after 2015 would affect data comparability). The study used an unsupervised machine learning algorithm, carrying out an automated classification of irregularities. Three measures of the municipalities’ institutional commitment to the freedom of information law (LAI) were used, based on data from the Brazilian Institute of Geography and Statistics (IBGE): a) the rules established to implement the LAI in the municipality, b) the degree of the LAI implementation at the local level; and c) how long the municipality has been committed to the LAI, as a measure of the institutionalization of public transparency.

It is expected that a greater institutional commitment of municipalities to freedom of information - assessed through these three measures - is associated with fewer cases of irregularities, in general, and corruption and mismanagement, specifically. This assumption was tested based on a quantitative research design using a linear regression technique. The study aims to contribute to the debate on freedom of information and government performance by bringing unprecedented empirical evidence about the effects of policies to combat illicit actions on public management.

This article is organized into five sections, including this introduction. The following section presents the theoretical debate on the relationship between public transparency and government performance. Next is the methodology, followed by the results. The final section presents the conclusion.

2. TRANSPARENCY AND GOVERNMENTAL PERFORMANCE

The quality of government has been considered a key element in society’s social and economic problems (Rothstein & Teorell, 2008). In general terms, quality of government refers to the performance of the agents that execute the will of the principals (Fukuyama, 2013).

From a normative perspective, the quality of government can be understood as the impartial exercise of public power, i.e., to implement, accordingly, the provisions of laws and policies (Rothstein & Teorell, 2008). Thus, there is an assumption that impartiality implies sufficient state capacity. However, a state can be both impartial and unable to provide public services effectively (Fukuyama, 2013). From an empirical perspective, it is essential to consider the interaction between capacity, measured by the state’s extractive capacity and the qualification of its bureaucracy, as well as by the autonomy of this bureaucracy to execute policies (Fukuyama, 2013; Marenco, 2017; Marenco, Strohschoen, & Joner, 2017).

Based on these arguments:

• Government performance is considered here as the ability of governments to implement, accordingly, the provisions of policies and laws.

The violation of this impartiality can occur as a consequence of the inability to execute the legislation provisions or due to the intention of obtaining advantages from the partial execution of the law. These cases indicate mismanagement and corruption. Mismanagement is the behavior or decisions that lead to poor performance, which differs from misfortune, that is, from adverse situations that are outside the control of the organization and affect its performance (Andrews, Boyne, & Enticott, 2006). Corruption, on the other hand, is the sale of government property for personal gain (Shleifer & Vishny, 1993). Such advantages can be electoral or material gains (Nyblade & Reed, 2008). Thus, corruption refers to the transaction - between a politician and a private actor - of a public good or property right to obtain material or electoral gains (Batista, 2013). This transaction occurs in a way that goes against the rules of the game (Jain, 2002).

Against this backdrop, the study observed the irregularities detected by the General-Comptroller of the Union (CGU) in Brazil in its program of randomly auditing municipalities, which verifies the correct use of federal resources transferred to states and local governments. Several studies have used this data, especially as a proxy for corruption. They allow to (a) identify determinants of corruption at the local level (Albuquerque & Ramos, 2006), (b) observe the relationship between the amends to the government’s budget proposed by members of the congress and municipal corruption (Sodré & Alves, 2010), (c) examine how re-election affects corruption (Ferraz & Finan, 2011), or (d) verify how electoral competition affects corruption (Batista, 2013).

One of the arguments underlying these studies is the problem of delegating decision-making and implementing policies from the principal-agent model. In this sense, voter control over the governor depends on two factors, (a) regular and free elections, i.e., the possibility of losing the mandate, and (b) the degree of voter information, which affects the opportunities of government officials to engage in dishonest behavior (Adserà, Boix, & Payne, 2003). While Barro’s model (1973) predicts complete information, Ferejohn (1986) includes the assumption of asymmetric information - voters cannot directly observe the government’s behavior, which in turn, benefits from this asymmetry. Thus, the greater its informational advantage, the greater its ability to profit from its mandate.

This debate has influenced analyses of how variables of the electoral dimension, such as competition and reelection, discipline government officials (Batista, 2013; Chang, 2005; Chang & Golden, 2007; Ferraz & Finan, 2011; Pereira, Melo, & Figueiredo, 2008; Persson, Tabellini, & Trebbi, 2003). It has also influenced studies that examine how mechanisms of transparency and participation affect government performance, especially after initiatives such as the Open Government Partnership (Fukuyama, 2016). On the other hand, the rapid availability of information over the internet has generated negative externalities, such as investment instability, which sheds light on the unintended effects of transparency (Fukuyama, 2016).

Studies offering evidence of the positive relationship between information and improved government performance (Adserà et al., 2003; Cucciniello et al., 2016; Fukuyama, 2016; Lindstedt & Naurin, 2010), contrast with research that shows null or negative evidence of this same relationship, suggesting that access to information is not enough to achieve the expected results regarding performance (Bataglia, 2019; Bauhr & Grimes, 2012; Casalecchi & Oliveira, 2010; Costa, 2012; Escaleras, Lin, & Register, 2010; Fukuyama, 2016). Education, free press, and free and fair elections are also important factors in the relationship between information availability and government performance (Brunetti & Weder, 2003; Lindstedt & Naurin, 2010).

Nevertheless, the association between transparency and improvement in the behavior of government officials is not new. In the eighteenth century, Jeremy Bentham (1995) developed a prison project called pan-optic. In this project, guards were positioned in the center of the building so that they could see all the prisoners (Bentham, 1995). According to the study, the awareness of being monitored at all times was sufficient to encourage good behavior (Batista, 2017; Schedler, 1999). In this sense, for instance, government officials could anticipate that practices of transparency would generate monitoring and then avoid corrupt behavior - even if the actual monitoring activities do not really exist (Meijer et al., 2012). However, such ideas carry the expectation that individuals process information in the same way, without considering the variety of existing cognitive biases (Rodrigues, 2019).

In order to understand this multiplicity of arguments and evidence, it is essential to look at the different types of transparency and the results they can lead to. Prat (2005), for example, argues that while transparency about the consequences of agents’ actions can be beneficial, the transparency about the actions per se can have negative effects on the agents’ discipline and selection, harming the principal. Michener and Bersch (2013) define transparency according to its constituent elements: visibility and inferability. The first is associated with the degree to which the information is complete and ‘findable.’ The second is linked to the quality of information and occurs when it is disaggregated or simplified or verified (Michener & Bersch, 2013).

In turn, Heald (2006) presents four directions of transparency that vary according to what can be observed (externally or internally) and in which direction (vertical or horizontal) (Rodrigues, 2019). The vertical and upward direction allows the superior in the hierarchy to observe the subordinates, based on the principal-agent economic model. On the downward, the governed can observe their governors, which refers to accountability and democracy. In the horizontal axis, transparency can be turned outward, so the agent can observe what happens outside the organization; or inward, so it is possible to observe what happens inside an organization. This last type of transparency direction is linked to the freedom of information (FOI) regime, in addition to social accountability mechanisms. Such direction concerns surveillance and peer monitoring (Heald, 2006). Freedom of information is used by journalists and citizens so that individuals can better govern themselves democratically (Kosack & Fung, 2014).

From these directions, Heald (2006) derives three dichotomies that represent varieties of transparency: transparency in retrospect versus transparency in real-time, event versus process transparency, and nominal versus effective transparency. The gap among the three dichotomies indicates an illusion of transparency, i.e., when the government does not appropriately provide information or when the recipient does not have the cognitive capacity to interpret and use that information.

Rodrigues (2019) argues that the move from nominal transparency to full transparency demonstrates the institutions’ efforts to overcome the failures that prevent full transparency. For the author, full transparency (where there is both visibility and inferability of information) is often unfeasible in public management due to the need for secrecy. Rodrigues (2019) suggests assessing the degree of possible visibility and inferability according to the different types and variety of transparency highlighted by Heald (2006). Therefore, in nominal transparency, inferability is limited, but visibility is present. That is, the information is accessible, but it is not possible to make inferences from it.

When approaching the Brazilian freedom of information law (LAI (2011), this research analyzed transparency as the formalization of a government’s institutional commitment with legal provisions that aim to guarantee the visibility and inferability of the information the government discloses about its actions. In this case, there is no guarantee that transparency will be effective, but rather an indication of an institutional effort to make this happen. Although freedom of information is related both to the vertical downward and horizontal inward directions (Heald, 2006), this study is not focused on initiatives to promote accountability - which have a vertically downward direction (Heald, 2006) - or on users who use specific services (Kosack & Fung, 2014). The focus here is on the horizontal aspect of transparency, i.e., on the instruments adopted by local governments, related to the freedom of information law (LAI (2011). The expectation is that the government is providing quality information, considering the aspects of visibility and inferability discussed by Michener and Bersch (2013).

Freedom of information laws provide a legal framework for access to internal government information. It represents an institutional measure of transparency (Hollyer, Rosendorff, & Vreeland, 2014) by defining rules and procedures for its application, and reducing the control of representatives about the information, allowing to use it to monitor the government (Berliner, 2014). Such institutionalization makes it costlier for governments to return to a culture of silence (Berliner, 2014). Also, even when government officials regulate freedom of information law only to obtain electoral benefits, the attention of public opinion may shift from adoption to implementation of the legislation, which increases the reputational costs of returning to a culture of secrecy (Schnell, 2017).

The objective in this study is to understand whether the variation in the legal and institutional commitment that governments have regarding access to information - observed through LAI (2011) in three different measures - is associated with a lower incidence of irregularities related to corruption and mismanagement at the local level. The assumption is that this commitment is associated with reducing mismanagement and corruption and improving government performance. The idea is that municipalities that set up the rules to implement the freedom of information law (LAI (2011)), especially considering the degree of LAI implementation and how long the local government has been observing this law, have a greater commitment to freedom of information, a less rooted culture of silence, and more institutionalized transparency. The following hypotheses will be tested:

1. Municipalities that implemented the LAI have fewer irregularities related to mismanagement and corruption;

2. The greater the degree of LAI implementation, the fewer irregularities related to mismanagement and corruption in Brazilian municipalities;

3. The greater the institutionalization of LAI, the fewer irregularities related to mismanagement and corruption in Brazilian municipalities.

In addition to freedom of information, the study considers alternative explanations for government performance, which were only briefly discussed in this section due to space limitations. Therefore, the analysis included the following controls: a) electoral incentives (reelection); b) quality of the local bureaucracy; c) means of communication; d) economic development; and e) region¹. The expectation is that mayors in the first term will be associated with fewer irregularities, as electoral incentives can restrict deviant behavior. As for the bureaucracy, which is the element responsible for the implementation of the analyzed programs, it is expected that the higher the bureaucracy’s quality, the lower the number of irregularities in local management. Also, the study tested whether the presence of means of communication improves performance since the media channels potentially increase the reach of public information. Finally, the economic development and the region of the country are expected to control structural differences between municipalities in Brazil.

3. DATA AND METHODS

The study sought to identify the relationship between transparency and government performance, analyzing Brazilian municipalities with data from 2011, the year the Brazilian freedom of information law (LAI) was enacted, to 2015, the year the methodology to gather the data about government performance used in this research changed. The decision to observe the LAI at the municipal level was made because it keeps institutions fixed at the national level, while municipalities vary regarding how each local government adopts the LAI (2011) and the political and socioeconomic context.

The data on the implementation of LAI (2011)in Brazil were collected from a survey of the Brazilian Institute of Geography and Statistics (IBGE) on the profile of Brazilian municipalities, conducted in 2014. In that particular year, IBGE collected information on transparency and freedom of information, obtaining data on the regulations the local governments set up to respond to the LAI (2011), the mechanisms adopted, and the year the municipalities started to follow the law’s provisions, which allowed to identify how long the municipality has committed to the LAI (IBGE, 2020).

This study adopted a binary variable to indicate whether the municipality established rules related to the LAI (2011). The variable assumes a value of ‘1’ when there is regulation and ‘0’ otherwise. As for the institutionalization of the LAI (2011), the measurement is the number of years since the municipality implemented these rules. This variable assumes a maximum value of ‘4,’ for the municipalities that set up regulations in the first year after the law passed, i.e., during 2012, and a minimum value of 0, for those that have not yet established rules to implement the LAI (2011).

The degree of the LAI implementation is an index based on the presence of the provisions of the law in the municipalities’ regulation. Eight variables related to the provisions of the law were used to establish the degree of implementation, and all of them were binary - considered ‘1’ when the municipality presents a rule regarding the provision of the LAI and ‘0’ when there is no local regulation related to the provision. The final index is the sum of the indicators and, therefore, has a minimum value of ‘0’ and a maximum of ‘8.’ The variables are a) the municipality establishes a deadline to provide information; b) establishes procedures to categorize the information; c) provides for monitoring authority; d) provides for a central monitoring body; e) provides for the disclosure of a monitoring report; f) provides for more than one appealing instance, internal to the government agency that was required to provide information; g) establishes an appealing body that is autonomous and independent of the authority that issued the contested decision; and finally, h) establishes a procedure in case of failure to respond.

Three measures were used to measure the performance of the local government, and they are based on data from the Brazilian General-Comptroller of the Union (CGU) audits. The data is objective information about the management of federal resources at the municipal level. With the municipalization of some of the main social policies, the local level has become central to the management of public resources and the delivery of essential benefits and services to the population. For this reason, CGU’s audit data are simultaneously relevant to the object they evaluate and appropriate for a test of the effect of transparency, as they refer to the management of resources and not to their impacts on society, something that is influenced by countless other factors.

As well as being an objective measure, the data were randomly selected using a system of the state-owned bank Caixa Econômica Federal, which operates the federal government’s lotteries. The research used the reports produced in the period from 2011 to 2015. The program underwent reformulations in 2015 and was no longer random. During the research period, 320 municipalities were audited. The design of CGU’s audits provides that only small municipalities (population from 10,000 to 250,000 inhabitants) are audited. For this reason, the sample, although random, is not representative of the population of municipalities in Brazil. However, the study offers subsidies to understand the implications of transparency on the performance of local management.

CGU’s audit reports are text files describing the irregularities found. These reports can be very detailed in archives of up to 300 pages. Only the basic description of the irregularity was required for this research. A synthesis of the irregularities presented in the reports² was manually extracted. This first procedure generated a file containing 16,064 irregularities.

The next step was to classify the type of irregularity. An unsupervised machine learning algorithm was used to avoid subjectivity in the classification, called the Latent Dirichlet Allocation (LDA). This is a probability procedure that can automatically retrieve the words that make up the topic and the topics that make up the document (Blei, 2012). The synthesis of the irregularity was used as a document. The use of the algorithm, it was possible to estimate the probability of a word composing the document (β) and the distribution of topics in the document (δ). The LDA is a topic classification algorithm that retrieves multiple topics. Here, only the topic with the highest proportion in the text was used.

Four topics were retrieved from the words observed in the irregularities. Topic labels are the responsibility of the authors. The topics were named as follows: 1) “licitação” (public procurement); 2) “gastos” (spending); 3) “execução” (execution); and 4) “utilização” (use). Figure 1 shows the words (in Portuguese) associated with each topic. A sample of 160 irregularities was manually classified to validate the classification, with a hit rate greater than 90%, indicating the validity of the procedure.

Figure 1
Topics and words collected from the synthesis of the irregularities

Once the topics related to the irregularities were identified, the irregularities were divided into corruption and mismanagement. Irregularities classified as “public procurement” are a proxy for corruption since they involve fraud and irregularities in bidding processes, the main form of exchanges between public managers and market players. The other irregularities related to “spending,” “execution,” and “use” of public programs are a proxy for mismanagement. Examples of the topic “public procurement” are the irregularity “setting up the bidding process,” “bias in the bidding process to build sports courts and to refurbish schools, favoring a company that is not located at the informed address,” and “total subcontracting of the school transportation contract 0122.01/2013.01, which generated, in fiscal years 2013 and 2014, an overprice in the amount of R$ 120,525.00 with funds from Fundeb (Brazilian Basic Education Fund).” These irregularities are closer to the definition of corruption since they indicate deviations of funds.

The other topics involve minor irregularities, such as “application of resources from the Índice de Gestão Descentralizada (IGD) (decentralized management index) in expenses ineligible for the Program Bolsa Família (Brazilian conditional cash transfer program),” classified as “spending.” Other examples are the irregularities “social accountability body of the Bolsa Família program has not operated efficiently,” classified as “execution,” and “use of inappropriate vehicles for the transportation of students,” classified as “use.”

Based on the aggregation of irregularities, there is information on the count of total irregularities, corruption, and mismanagement. However, despite being the common procedure in analyses that use data from the CGU, the simple count of irregularities can be a biased measure, since in different municipalities, the number of programs audited can be different, and this directly affects the absolute number of irregularities identified. In order to solve this problem and present more valid measures, the absolute number of total irregularities, corruption, and mismanagement was considered according to the number of programs audited in the municipality. This information is included in the documents of the CGU audit, made available by Gehrke (2019).

The final variables are, therefore, the number of irregularities weighted by the number of objects/programs audited as reported in the CGU audit’s documents. The first specification concerns the total number of irregularities in the municipality, and the second is the number of irregularities classified as mismanagement. The third specification is the number of irregularities classified as corruption. As the dependent variables of the study are all continuous, linear least squares regression models were used.

The control variables included electoral incentives (re-election), quality of the local bureaucracy, means of communication, economic development, and region. Re-election assumed a value of ‘0’ when the mayor is in the first term and a value of ‘1’ when in the second. Quality of the local bureaucracy is indicated by the percentage of employees with higher education. Means of communication refers to the number of media vehicles available in the municipality (the options were community radio, AM radio, FM radio, internet, television). The variable assumes a minimum value of ‘0’ for municipalities without means of communication and ‘5’ for municipalities with all the options. Finally, GDP per capita was adopted as a measure of economic development, and the region of the country where the municipality was located, included as a set of binary variables. All data were retrieved from the Brazilian Institute of Geography and Statistics (IBGE).

The growing concern about valid causal inferences made political scientists look for ways to identify the counterfactual, or what would happen if the variable of interest did not occur. The use of regression models for this task has a series of limitations. Even with the inclusion of control variables to identify the relationship between X and Y and keep these factors constant, the problem of the omitted variable is still present, which can cause endogeneity problems. The strategy adopted here was the use of controls that the literature points out as relevant, and proceeding with caution in the interpretation of the results since they were not produced from a controlled experiment³.

4. RESULTS

As previously mentioned, for the effective implementation of LAI (2011), municipalities need to establish regulations at the local level, establishing their own procedures for accessing public information. Figure 2 shows the number and pace of local regulations related to the federal freedom of information law (LAI (2011)). The panel on the left shows all the municipalities in Brazil and the panel on the right shows the municipalities included in the sample of locations audited by CGU.

Figure 2
Adoption of the freedom of information law in Brazilian municipalities

Figure 2 shows that the number and pace of LAI regulation (2011) can be considered low and slow, respectively. The number of municipalities adopting rules related to the LAI grew, reaching the maximum number in 2013. However, this growth did not remain linear, with a reduction in the number of municipalities in 2014. Considering the accumulated number, only 1 in 5 municipalities in Brazil establish local rules related to the LAI (2011). The sample of municipalities audited by CGU includes only smaller municipalities. The distribution is similar to that of the population, but the numbers are much smaller. While 20% of Brazilian municipalities established regulations related to the LAI (2011), in the sample, this percentage dropped to 10%. This is because the municipalities more likely to establish such regulations are the largest ones and the capitals (Batista, 2017). Bearing in mind that the lack of regulation considerably limits access to information because the procedures are not formalized, transparency at the local level is still very limited.

Also, Figure 2 shows that after four years, the freedom of information law was enacted in Brazil, most municipalities have not yet established local regulations to implement federal legislation. More importantly, the number of municipalities that regulated the LAI decreased in 2014, indicating that there is no linear growth trend. This result indicates that the LAI (2011) does not automatically translate into the institutional commitment to transparency at the local level. It also suggests that greater investments are needed for municipalities to adapt to the new reality of public transparency. In addition, of the municipalities that established rules related to the LAI, the majority did it in 2013, indicating that there is still little experience with the law and, therefore, less institutionalization of procedures. Table 1 shows the results regarding the degree of LAI implementation at the local level.

Table 1
Degree of the LAI implementation in Brazilian municipalities

	Minimum	Maximum	Average
Population (total)	0	8.00	0.818
Population (municipalities that regulated the LAI)	1.00	8.00	4.239
Sample (total)	1.00	8.00	0.684
Sample (municipalities that regulated the LAI)	1.00	8.00	4.576

• Source: Elaborated by the authors.

The inclusion of this variable intends to offer a more refined measure of the procedures related to public transparency in the municipalities. Table 1 indicates that the degree of LAI implementation is still low, especially when considering all municipalities. The average of procedures, when considering both the municipalities that have and have not established rules related to the LAI, is 0.818 for the population, and 0.684 for the sample. As the maximum value of this index is 8, the formalization of procedures is considered extremely low. When considering only the municipalities that established regulations related to the LAI, the average is 4,239 for the population and 4,576 for the sample. Thus, even when considering only the municipalities committed to the LAI (2011), the formalization of procedures is far from ideal.

These results showed that there is still much to be done at the local level regarding the local implementation of the LAI (2011). The freedom of information law (LAI (2011)), at the federal level, does not automatically imply the formulation of procedures that authorize and encourage access to public information. If the diffusion of transparency at the local level is limited, how is the performance of local governments distributed?

Figure 3
Distribution of irregularities in local administration (2011-2015)

The three panels in Figure 3 show the distribution of irregularities observed in the municipalities audited by CGU. The absolute values of the irregularities are weighted by the number of programs audited in the municipality. The average is 3.5 irregularities per audited project, showing a significant number of problems in the management of federal resources by local governments. For mismanagement, the average is 2.6 cases per project. The average number of corruption cases is lower, with 0.88 cases per audited project. Thus, among the irregularities observed, mismanagement is the most prevalent, and corruption is observed to a lesser extent. In all cases, however, the numbers are significant, indicating that intervention is needed to reduce these problems in the delivery of public benefits and services.

In this next part of the analysis, the objective was identifying whether the municipalities that established regulations related to the LAI (2011) show any improvement in their performance. In other words, does the municipality’s institutional commitment to transparency have a negative relationship with corruption and mismanagement? Table 2 presents the results of the regression models that identify the association between transparency and performance, controlled by the factors that can also affect government performance.

Table 2
Transparency and irregularities in municipal management

	Dependent variable
	Irregularities			Mismanagement			Corruption
	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)
Regulation	0.123			0.114			0.009
	(0.224)			(0.186)			(0.114)
Degree		0.057			0.052*			0.005
		(0.036)			(0.030)			(0.018)
Institutionalization			0.037			-0.004			0.041
			(0.080)			(0.067)			(0.041)
Means of communication	-0.036	-0.038	-0.034	-0.057	-0.059	-0.053	0.021	0.021	0.020
	(0.045)	(0.045)	(0.045)	(0.038)	(0.037)	(0.038)	(0.023)	(0.023)	(0.023)
Bureaucracy	-0.016**	-0.016**	-0.016**	-0.009*	-0.010*	-0.009	-0.007**	-0.007**	-0.007**
	(0.006)	(0.006)	(0.006)	(0.005)	(0.005)	(0.005)	(0.003)	(0.003)	(0.003)
Re-elected	0.089	0.086	0.092	-0.002	-0.004	0.003	0.091	0.090	0.089
	(0.137)	(0.136)	(0.138)	(0.114)	(0.113)	(0.115)	(0.070)	(0.070)	(0.070)
GDP per capita	0.003*	0.003**	0.003*	0.002*	0.003**	0.003**	0.0005	0.001	0.0003
	(0.001)	(0.001)	(0.002)	(0.001)	(0.001)	(0.001)	(0.001)	(0.001)	(0.001)
Year	0.277***	0.279***	0.277***	0.163***	0.166***	0.168***	0.114***	0.114***	0.109***
	(0.043)	(0.042)	(0.043)	(0.035)	(0.035)	(0.036)	(0.022)	(0.021)	(0.022)
Northeast	0.807***	0.832***	0.801***	0.713***	0.735***	0.706***	0.095	0.097	0.095
	(0.249)	(0.248)	(0.249)	(0.207)	(0.206)	(0.207)	(0.127)	(0.127)	(0.127)
North	0.822***	0.853***	0.819***	0.800***	0.828***	0.792***	0.022	0.025	0.027
	(0.300)	(0.300)	(0.301)	(0.250)	(0.249)	(0.250)	(0.153)	(0.153)	(0.153)
Southeast	-0.156	-0.147	-0.160	0.202	0.209	0.191	-0.357***	-0.356***	-0.352***
	(0.257)	(0.256)	(0.257)	(0.214)	(0.212)	(0.214)	(0.131)	(0.131)	(0.131)
South	-0.178	-0.189	-0.180	0.140	0.130	0.148	-0.318**	-0.319**	-0.328**
	(0.275)	(0.274)	(0.276)	(0.229)	(0.228)	(0.230)	(0.140)	(0.140)	(0.140)
Constant	-554.101***	-558.747***	-555.043***	-326.594***	-330.970***	-336.533***	-227.507***	-227.776***	-218.510***
	(85.635)	(83.679)	(85.994)	(71.221)	(69.539)	(71.557)	(43.625)	(42.791)	(43.712)
Observation	285	285	284	285	285	284	285	285	284
R2	0.278	0.284	0.276	0.183	0.191	0.181	0.203	0.203	0.203
Adjusted R2	0.252	0.257	0.249	0.153	0.161	0.151	0.173	0.174	0.174
Residual Std. Error	1.086 (df = 274)	1.082 (df = 274)	1.088 (df = 273)	0.903 (df = 274)	0.899 (df = 274)	0.905 (df = 273)	0.553 (df = 274)	0.553 (df = 274)	0.553 (df = 273)
F	10.545*** (df = 10; 274)	10.843*** (df = 10; 274)	10.384*** (df = 10; 273)	6.148*** (df = 10; 274)	6.466*** (df = 10; 274)	6.040*** (df = 10; 273)	6.958*** (df = 10; 274)	6.966*** (df = 10; 274)	6.973*** (df = 10; 273)

• Note: ^*p^**p^***p<0.01 Source: Elaborated by the authors.

Table 2 shows three different models for each specification of the dependent variable. Thus, the three transparency measures proposed in this study are presented. The first is the establishment of rules related to the LAI (2011), the second is the degree of LAI implementation, and the third is how long the municipality has been committed to the LAI, as a measure of the institutionalization of public transparency.

The results indicate that in all specifications of the dependent variable that seek to capture government performance, there is no association statistically significant regarding the transparency measures. This finding suggests that keeping the other factors constant, transparency does not reduce the occurrence of irregularities in local management, nor does it reduce mismanagement or corruption.

The literature seeking to empirically identify the effect of transparency on government performance demonstrates mixed results. Some studies show the positive effect of transparency to improve government performance, mainly by reducing corruption. Other studies, however, have not found a statistically significant effect. This study contributes to the literature showing that, in the case of Brazilian municipalities, the institutional commitment to transparency does not prove to be a strong enough factor to reduce irregularities in local management, improving government performance. Bearing in mind that the greatest justification for the diffusion of transparency initiatives is the quality of the government, the lack of statistical significance in the results contribute to the literature - notably divided considering the mixed empirical results achieved in several studies -, indicating that initiatives on transparency need to go beyond the institutional commitment⁴.

From the point of view of research design, it is important to mention the possibility of endogeneity in the relationship between transparency and government performance, because the establishment of rules related to the LAI (2011) by municipalities, is not a phenomenon randomly distributed. That is, the municipalities choose to locally regulate the LAI, the degree to which they will implement the provisions of the freedom of information law, and when. Thus, municipalities with better results could choose to adopt LAI (2011) because they have “nothing to hide,” and this would characterize an endogenous relationship. However, if there was endogeneity (which cannot be ruled out considering the research design adopted in this study), and such endogeneity affected the results, the estimates obtained in the model would be in the opposite direction. That is, it would be more likely to find the negative association between transparency and performance since the municipalities with better performance would have more incentives to join LAI (2011) so that the previous performance would cause both transparency (independent variable) and performance (dependent variable). Given that the results indicate that transparency is not related to government performance, there is evidence that endogeneity is not a determinant of the results presented here. In summary, in the current state of diffusion of LAI (2011) in the Brazilian municipalities, the local regulation of the LAI, the degree of implementation, or the how long a municipality has been committed to transparency are not related to government performance.

Regarding the variables that present a statistically significant result, the quality of the bureaucracy reduces the number of irregularities, mismanagement, and corruption. In other words, the higher the education level of employees in the local administration, the fewer irregularities identified. This result is consistent with previous research, which shows the effect of the quality of local bureaucracy on the performance of municipalities (Batista, 2015; Marenco, 2017; Marenco et al., 2017). Quality bureaucracy has a significant effect in reducing the irregularities committed by managers and increasing the delivery of benefits and services to the population.

Another variable that showed a statistically significant result was economic development. The findings showed that except for corruption, the higher the local GDP per capita, the more irregularities are observed for all specifications. This result indicates more irregularities and cases of mismanagement in municipalities that are larger, more complex, and, consequently, managing more resources. The set of variables that indicate the region of the municipality, with the Brazilian Central-West region as a reference, indicates that the North and Northeast regions generally exhibit a greater number of total irregularities and cases of mismanagement. In cases of corruption, the South and Southeast regions have fewer cases than the Central-West region. Finally, the year variable also presents statistically significant results, showing that the number of irregularities is growing over time and has not been decreasing, as might be expected as a consequence of the implementation of LAI (2011) at the national level.

5. CONCLUSION

This article sought to answer the question: Does the Brazilian municipalities’ institutional commitment to public transparency reduce corruption and mismanagement? Based on the general argument that government officials behave better when monitored, there is a theoretical and practical expectation that transparency will reduce corruption and improve government performance. However, the evidence on the effects of transparency policies around the world is heterogeneous, pointing to different results.

Thus, this study contributed to the debate in the literature, using an objective measure of corruption and mismanagement from an original database, which compiles about 16 thousand irregularities identified by the Brazilian General-Comptroller of the Union in the period from 2011 to 2015. The tested hypotheses explored if establishing local rules related to the LAI, the degree of local implementation of the freedom of information law (LAI (2011)), and how long a municipality has been committed to transparency policies (indicating how institutionalized these policies are), are associated with a lower level of irregularities, corruption, and mismanagement.

The results indicate that transparency, in the three specifications used, does not reduce the occurrence of irregularities in local management, nor does it reduce mismanagement or corruption. Quality of bureaucracy and economic development, in turn, showed statistically significant results and in the expected direction. Therefore, this study contributed to the debate by bringing evidence of the relationship between institutional commitment and transparency on direct and objective measures of corruption and mismanagement as proxies of government performance.

Given that this is a probability and quantitative research design, the findings do not necessarily mean that LAI (2011), or transparency in government, more broadly, is not important for fighting corruption and improving public management. These results suggest the need for future studies to advance the research agenda for causal mechanisms in the relationship between transparency and performance, either by developing effective transparency measures or by identifying where in the causal chain the institutional commitment to transparency loses its ability to affect government performance. Studies based on process-tracing or that clearly identify the counterfactual are needed⁵.

Furthermore, as Michener (2019) emphasizes, the effects of transparency policies tend to be gradual, indirect, and diffuse. In this sense, the results indicate that there is still a long way to go in terms of strengthening the LAI (2011) at the local level. It is necessary to ensure the effective implementation of the law and mechanisms for accessing information so that it is possible to visualize its effects on society. Thus, we suggest deepening the analysis of the degree of implementation of the Lai (2011) in Brazilian municipalities.

ACKNOWLEDGEMENTS

This study was developed within the scope of the project “More Resources or Better Management.” The project is funded by the Foundation for Science and Technology Development of the State of Pernambuco (FACEPE), Process APQ-0158-7.09/15. The author José Luiz Alves dos Santos is grateful to FACEPE for his scholarship to support scientific initiation, which allowed the development of this research. The author Virginia Rocha is grateful to the Coordination for the Improvement of Higher Education Personnel (CAPES) for her doctoral scholarship.

REFERÊNCIAS

• Adserà, A., Boix, C., & Payne, M. (2003). Are you being served? Political accountability and quality of government. Journal of Law, Economics, & Organization, 19(2), 445-490.
• Albuquerque, B. E., & Ramos, F. S. (2006). Análise teórica e empírica dos determinantes de corrupção na gestão pública municipal. In Anais do 34o Encontro Nacional de Economia Salvador, BA.
• Andrews, R., Boyne, G. A., & Enticott, G. (2006). Performance failure in the public sector. Public Management Review, 8(2), 273-296.
• Barro, R. J. (1973). The control of politicians: an economic model. Public Choice, 14, 19-42.
• Bataglia, M. B. (2019). Acesso à informação e corrupção: investigando o contexto institucional da CGU (Dissertação de Mestrado). Universidade de Brasília, Brasília, DF.
• Batista, M. (2013). Incentivos da dinâmica política sobre a corrupção: reeleição, competitividade e coalizões nos municípios brasileiros. Revista Brasileira de Ciências Sociais, 28(82), 87-106.
• Batista, M. (2015). Burocracia local e qualidade da implementação de políticas públicas descentralizadas: uma análise da gestão de recursos federais pelos municípios brasileiros. Revista do Serviço Público, 66(3), 345-370.
• Batista, M. (2017). A difusão da lei de acesso à informação nos municípios brasileiros: fatores internos e externos Brasília, DF: Escola Nacional de Administração Pública.
• Bauhr, M., & Grimes, M. (2012, December). What is government transparency? New measures and relevance for quality of government (Working Paper Series 2012). Gothenburg, Sweden: The Quality of Government Institute.
• Beach, D., & Pedersen, R. B. (2019). Process-tracing methods: foundations and guidelines Ann Arbor, MI: University of Michigan Press.
• Bentham, J. (1995). The panopticon writings London, UK: Verso.
• Berliner, D. (2014). The political origins of transparency. The Journal of Politics, 76(2), 479-491.
• Blei, D. (2012). Probabilistic topic models. Communications of the ACM, 5(4), 77-84.
• Brunetti, A., & Weder, B. (2003). A free press is bad news for corruption. Journal of Public Economy, 87(7-8), 1801-1824.
• Calland, R., & Bentley, K. (2013). The impact and effectiveness of transparency and accountability initiatives: freedom of information. Development Policy Review, 31, 69-87.
• Casalecchi, A. R. C., & Oliveira, E. M. (2010). As auditorias da CGU e a transparência licitatória dos municípios paulistas. Cadernos Gestão Pública e Cidadania, 15, 49-62.
• Chang, E. C. C. (2005). Electoral incentives for political corruption under open-list proportional representation. The Journal of Politics, 67(3), 716-730.
• Chang, E. C. C., & Golden, M. A. (2007). Electoral systems, district magnitude and corruption. British Journal of Political Science, 37(1), 115-137.
• Controladoria-Geral da União. (2013). Manual da Lei de Acesso à Informação para estados e municípios Brasília, Autor.
• Costa, S. (2012). Do freedom of information laws decrease corruption? The Journal of Law, Economics, and Organization, 29(6), 1317-1343.
• Cucciniello, M., Grimmelikhuijsen, S., & Porumbescu, G. A. (2016). 25 years of transparency research: evidence and future directions . Public Administration Review, 77(1), 32-44.
• Escaleras, M., Lin, S., & Register, C. (2010). Freedom of information acts and public sector corruption . Public Choice, 145(3), 435-460.
• Ferejohn, J. (1986). Incumbent performance and electoral control. Public Choice, 50(1-3), 5-25.
• Ferraz, C., & Finan, F. (2011). Electoral accountability and corruption: evidence from the audit reports of local governments. American Economic Review, 101(4), 1274-1311.
• Fox, J. (2007). The uncertain relationship between transparency and accountability. Development in Practice, 17(4-5), 663-671.
• Freedom of Information. (2019). Alphabetical and chronological lists of countries with FOI regimes Recuperado dehttp://www.freedominfo.org/?p=18223
  » http://www.freedominfo.org/?p=18223
• Fukuyama, F. (2013). What is governance? Governance: An International Journal of Policy, Administration, and Institutions, 26(3), 347-386.
• Fukuyama, F. (2016). Governance: what do we know, and how do we know it. Annual Review of Political Science, 19, 89-105.
• Gehrke, M. (2019). How politicians react to anti-corruption investigations and enforcement. Evidence from Brazilian municipalities (Ph.D. Dissertation). University of California in Los Angeles, Los Angeles, CA.
• Heald, D. (2006). Varieties of transparency. In C. Hood, & D. Heald (Org.), Transparency: the key to better governance?(pp. 25-43). Oxford, UK: Oxford University Press.
• Hollyer, J. R., Rosendorff, P. B. & Vreeland, J. R. (2014). Measuring transparency. Political Analysis, 22, 413-434.
• Instituto Brasileiro de Geografia e Estatistica. (2020). Pesquisa de Informações Básicas Municipais - MUNIC Recuperado dehttps://www.ibge.gov.br/estatisticas/sociais/educacao/10586-pesquisa-de-informacoes-basicas-municipais.html?=&t=o-que-e
  » https://www.ibge.gov.br/estatisticas/sociais/educacao/10586-pesquisa-de-informacoes-basicas-municipais.html?=&t=o-que-e
• Jain, A. K. (2002). Corruption: a review. Journal of Economic Surveys, 15(1), 71-121.
• Kosack, S., & Fung, A. (2014). Does transparency improve governance? Annual Review of Political Science, 17, 65-87.
• Lei n. 12.527, de 18 de novembro de 2011 (2011). Regula o acesso a informações previsto no inciso XXXIII do art. 5º , no inciso II do § 3º do art. 37 e no § 2º do art. 216 da Constituição Federal; altera a Lei n. 8.112, de 11 de dezembro de 1990; revoga a Lei n. 11.111, de 5 de maio de 2005, e dispositivos da Lei n. 8.159, de 8 de janeiro de 1991; e dá outras providências. Brasília, DF.
• Lindstedt, C., & Naurin, D. (2010). Transparency is not enough: making transparency effective in reducing corruption. International Political Science Review, 31(3), 301-322.
• Marenco, A. (2017). Burocracias profissionais ampliam capacidade estatal para implementar políticas? Governos, burocratas e legislação em municípios brasileiros. Dados, 60(4), 1025-1058.
• Marenco, A., Strohschoen, M. T. B., & Joner, W. (2017). Capacidade estatal, burocracia e tributação nos municípios brasileiros. Revista de Sociologia e Política, 25(64), 3-21.
• Meijer, A., Curtin, D., & Hillebrandt, M. (2012). Open government: connecting vision and voice. International Review of Administrative Sciences, 78(1), 10-29.
• Michener, G. (2019). Gauging the impact of transparency policies. Public Administration Review, 79(1), 136-139.
• Michener, G., & Bersch, K. (2013). Identifying transparency. Information Polity,18(3), 233-242.
• Mitchell, G. (2004). Case studies, counterfactuals, and causal explanations. University of Pennsylvania Law Review, 152(5), 1517-1608.
• Nyblade, B., & Reed, S. R. (2008). Who cheats? Who loots? Political competition and corruption in Japan, 1947-1993. American Journal of Political Science, 52(4), 926-941.
• Pereira, C, Melo, M. A., & Figueiredo, C. M. (2008). The corruption-enhancing role of re-elections incentives? Counterintuitive evidence from Brazil’s audit reports. Political Research Quaterly, 62(4), 731-744.
• Persson, T., Tabellini, G., & Trebbi, F. (2003). Electoral rules and corruption. Journal of the European Economic Association, 1, 958-989.
• Prat, A. (2005). The wrong kind of transparency. The American Economic Review, 95(3), 862-877.
• Rodrigues, K. F. (2019). Desvelando o conceito de transparência: seus limites, suas variedades e a criação de uma tipologia. Cadernos EBAPE.BR, 18(2), 237-253.
• Rothstein, B., & Teorell, J. (2008). What is quality of government? A theory of impartial government institutions. Governance: An International Journal of Policy, Administration, and Institutions, 21(2), 165-190.
• Schedler, A. (1999). Conceptualizing accountability. In A. Schedler, L. Dimond, & M. F. Plattner(Ed.), The self-restraining State London, UK: Lynne Rienner.
• Schnell, S. (2017). Cheap talk or incredible commitment? (Mis)calculating transparency and anti-corruptionM. F. Plattner. Governance, 31(3), 415-430.
• Shleifer, A., & Vishny, R. W. (1993). Corruption M. F. Plattner. The Quarterly Journal of Economics, 108(3), 599-617.
• Sodré, A. C. A., & Alves, M. F. C. (2010). Relação entre emendas parlamentares e corrupção municipal no Brasil: estudo dos relatórios do programa de fiscalização da Controladoria-Geral da União. Revista de Administração Contemporânea, 14(3), 414-433.

• 1
  On the influence of electoral incentives related to re-election, see Pereira, Melo, and Figueiredo (2008) and Ferraz and Finan (2011). On the effects of local bureaucratic capacity, see Batista (2015). Finally, on the influence of the means of communication, see Ferraz and Finan (2011).
• 2
  Automated procedures were tried but were not able to correctly obtain the number of irregularities pointed out in the reports.
• 3
  The use of so-called quasi-experimental techniques is a statistical solution for situations where a controlled experiment is not feasible. Some of these techniques could be considered in this study. However, CGU data that reveal irregularities in the municipalities only exist for 320 municipalities. Thus, techniques such as matching or discontinuous regression have little applicability since few cases would remain after a matching procedure or the use of discontinuous regression with close elections as a break variable. In addition, these techniques have their own limitations, which must be considered.
• 4
  We emphasize the importance of disclosing null results, especially in public policies and government intervention. This is because the omission of statistically non-significant results characterizes publication bias and also encourages the maintenance of initiatives that often do not generate the expected results.
• 5
  Counterfactual is the identification of what would happen if X, the variable of interest, did not occur. It is how the causal effect of X on Y is isolated. The search for counterfactual is widely discussed in quantitative and experimental research designs. However, it is also possible to look for counterfactual in qualitative designs, which are case-oriented instead of variable-oriented. One way of looking for the counterfactual is through “mental experiments” (Mitchell, 2004). Another way of analyzing causal relationships in qualitative and case-oriented research designs is the so-called process-tracing - which focuses on the analysis of causal mechanisms. The proposal is to open the “black box” of causality using in-depth case study methods (Beach & Pedersen, 2019).

• [Translated version] Note: All quotes in English translated by this article’s translator.

Publication Dates

• Publication in this collection
  02 Nov 2020
• Date of issue
  Sep-Oct 2020

History

• Received
  12 Aug 2019
• Accepted
  06 May 2020