What works and what does not work to reduce homicides in Brazil: a systematic review

Abstract

This article aims to identify the scientific evidence in Brazil on what works and what does not work to reduce the number of homicides. This is the first systematic review ever conducted in the country on the effectiveness of public security programs, using the same methodology as the Maryland Report, conducted in 1997, which reviewed the studies produced in English and is considered a milestone in the emergence of the so-called evidence-based crime prevention paradigm (EBCP). The review resulted in a total of 13,352 studies, of which 41 met the inclusion criteria, resulting in the identification of 8 types of programs that work, 7 promising programs, 7 programs with undefined results, and 2 programs with evidence that shows they do not work. The study showed that the country has innovative programs that effectively reduce homicides, while actions traditionally cited as solutions do not present scientific evidence that they work. Although lagging behind international development, the EBCP presents the potential to increase the effectiveness of public spending, help the country to reduce violence, and overcome decision-making processes based on beliefs, corporatism, prejudice, and political populism.

Keywords:
crime prevention evidence-based; evidence-based public policies; violence prevention; policing and democracy

Resumo

Este artigo tem por objetivo identificar quais são as evidências científicas existentes no Brasil, hoje, sobre o que funciona e o que não funciona para reduzir o número de homicídios, por meio da primeira revisão sistemática já realizada no país acerca da efetividade de programas de segurança pública, utilizando a mesma metodologia do Relatório Maryland, realizado em 1997, o qual revisou os estudos já produzidos em língua inglesa e que é considerado um marco no surgimento da chamada segurança pública baseada em evidências (SPBE). A busca resultou em um total de 13.352 estudos analisados, dos quais 41 preencheram os critérios de inclusão, resultando na identificação de 8 tipos de programas que funcionam, 7 programas promissores, 7 programas com resultados indefinidos e 2 programas com evidências que não funcionam. O estudo mostrou que o país dispõe de programas inovadores que são efetivos para reduzir homicídios, enquanto ações tradicionalmente citadas como soluções para a área não apresentam evidências científicas de que funcionem. Embora atrasado em relação ao desenvolvimento internacional, o paradigma da SPBE apresenta grande potencial para aumentar a efetividade do gasto público, auxiliar o país a reduzir a violência e superar processos de tomada de decisão baseados em crenças, corporativismos, preconceitos e populismos políticos.

Palavras-chave:
segurança pública baseada em evidências; políticas públicas baseadas em evidências; prevenção à violência; polícia e democracia

Resumen

Este artículo tiene como objetivo identificar qué evidencia científica existe hoy en Brasil sobre lo que funciona y lo que no funciona para reducir el número de homicidios, a través de la primera revisión sistemática que se ha realizado en el país sobre la eficacia de los programas de seguridad pública, utilizando la misma metodología del Informe Maryland, realizado en 1997, que revisó los estudios ya realizados en inglés y que se considera un hito en el surgimiento de la llamada seguridad pública basada en la evidencia (SPBE). La búsqueda resultó en un total de 13.352 estudios analizados, de los cuales 41 cumplieron los criterios de inclusión, lo que dio como resultado la identificación de 8 tipos de programas que funcionan, 7 programas prometedores, 7 programas con resultados no definidos y 2 programas con pruebas de que no funcionan. El estudio demostró que el país tiene programas innovadores que son eficaces para reducir los homicidios, mientras que las acciones tradicionalmente citadas como soluciones para el área no presentan evidencia científica de que funcionen. Aunque va a la zaga del desarrollo internacional, el paradigma de la SPBE presenta gran potencial para aumentar la eficacia del gasto público, ayudar al país a reducir la violencia y superar los procesos de toma de decisiones basados en creencias, corporativismo, prejuicios y populismo político.

Palabras clave:
seguridad pública basada en la evidencia; políticas públicas basadas en la evidencia; prevención de la violencia; policía y democracia

1. INTRODUCTION

For half a century, Brazil has suffered the consequences of an epidemic of violence and inefficient actions of governments to deal efficiently with this problem. Between 1980 and 2019, the country averaged 40,000 homicides per year, resulting in at least 1.5 million Brazilians murdered (Instituto de Pesquisa Econômica Aplicada & Fórum Brasileiro de Segurança Pública, 2019) and the prison population has increased by more than 700% (Santos & Rosa, 2017). Despite the country having only 2.8% of the world population, 15.3% of all murders in the world, in 2016, occurred in Brazilian territory (Global Burden of Disease Collaborative Network [GBD], 2017) and in 2018, the country had 17 cities among the 50 most violent in the world (Seguridad, Justicia y Paz, 2019).

In response to this situation, the different levels of government and powers have carried out various actions and spent a substantial amount of money on public security. Since re-democratization, the Federal Government has launched at least four National Security Plans (Kopittke, 2016), carried out 101 Law and Order Guarantee Operations by the Armed Forces (Ministry of Defense, 2018), hundreds of Federal Police operations, several special actions and a Federal Intervention in the state of Rio de Janeiro, in addition to an average of five new federal laws on public security and criminal policy approved each year by the National Congress (Campos, 2010). However, public spending in the area is inefficient and generally very ineffective in reducing the rates of violence.

One of the reasons pointed out for the failure of crime prevention policies in Brazil is the fact that the country is not able to advance in the production and use of scientific evidence to solve the problem of violence, as occurs internationally with the advance of Crime Prevention Evidence Based (Beato, 2002; Beato & Silveira, 2014; Cano, Rojido, Chaves, Bezerra & Sá, 2016; Rolim, 2006; Willadino, Sento-Sé, Dias & Gomes, 2011).

The countries with consolidated democracies also experienced, between the 1960s and 1980s, a profound crisis of effectiveness, efficiency and legitimacy of the so-called Traditional Model of Crime Prevention, which was based, until that point, exclusively, on reactive, without focus and fragmented actions by the police, criminal justice system and imprisonment as a way to contain the levels of violence (Lum & Koper, 2017).

This crisis has stimulated the emergence of innovative programs and impact evaluation research resulting in what is called today Crime Prevention Evidence Based (CPEB), a concept that advocates that violence prevention programs, must be based on the best available scientific evidence on what works and what does not work, reducing the influence of beliefs, interests and prejudices, causing a substantial improvement in the effectiveness, efficiency and legitimacy of public policies aimed at reducing violence (Sherman, 1998; Welsh & Farrington, 2007).

The results of the first impact assessment surveys demonstrated that traditional beliefs, such as increasing the number of patrolling vehicles (Kelling, Pate, Dieckman & Brown, 1974), a reduction in response times to emergency calls (Bieck & Kessler , 1977; Spelman & Brown, 1981), assigning tutors to discipline young offenders (Cabot, 1940) or classes given by police officers on the dangers of alcohol and drug use (Ennet et al., 1994), did not have a positive impact and, in some cases, they worsened the indicators they sought to reduce.

On the other hand, innovative violence prevention strategies emerged, using scientific knowledge about risk factors and criminal dynamic patterns, making violence prevention actions more proactive and focused. Those strategies resulted in a drop in violence rates and an increase in efficient public spending. They are much cheaper and more effective than traditional criminal justice methods and incarceration actions, such as, the development of socio-emotional skills in early childhood (Weikart, Deloria, Lawser & Wiegerink, 1970), policing in hot spots (Sherman & Weisburd, 1995), focused deterrence on violent leadership (Kennedy, 1997), changes in urbanization (Newman & Franck, 1982), in addition to certain cognitive behavioral methodologies with highly violent prisoners (Andrews et al., 1990).

As the number of evidence grew, it became necessary to systematize evidence in order to deliver more synthetic responses to decision makers about what effectively works. As such, the area of ​​violence prevention began to use Systematic Reviews, a scientific method aimed at bringing together in a structured, rigorous and reproducible way the largest possible number of existing studies on a certain type of problem and thus producing stronger evidence than isolated studies. This technique is currently being considered the best level of evidence for public policy decision making (Kugley et al., 2016; Lipsey, 2009; Weisburd, Farrington & Gill, 2017).

An important historical milestone in the solidification of CPEB was the completion of a major Systematic Review, at the request of the US Congress, by a group of renowned researchers, with the aim of systematizing the knowledge that had been produced up to that moment, in English, about what worked and what did not reduce violence, which became known as the Maryland Report (Sherman et al., 1997).

The researchers found more than 500 studies that measured the impact of different programs on violent behavior, violent crimes and criminal recidivism that met the technical criteria established in the research. When presenting its conclusions about what worked and what did not, the study had a great impact worldwide over the following years, causing changes in the way Crime Prevention programs are formulated and implemented, in several countries around the world.

In addition, the Report established the standards used currently around the world to assess the impact of programs and produce syntheses of evidence, using a scientific scale that became known as the Maryland Scale (Weisburd et al., 2017).¹

Since then, impact evaluations and systematic reviews on the effectiveness of programs in the area have increased worldwide, reaching an average of 50 impact evaluations per year, between 2007 and 2013 (Huges et al., 2014 ), and 12 Systematic Reviews per year, which already total 474 published studies of this type (Waar, 2019).

As several authors have pointed out, Brazil was oblivious to this technical advance in the area. According to Soares (2006), the absence of a scientifically oriented debate on solutions means that discussions on Crime Prevention remain guided only by ideological polarizations and no technical orientation, without the concern of rigorously evaluating what is done. Rolim (2006) points out that, while in England and in other developed countries the culture of speculative and experimental reasoning has led to a revolution in criminology in the last few decades, with the introduction of experimental research, Brazil remained stuck in a “pre-history of Crime Prevention”, due to the absence of the use of scientific evidence in the area, with decisions being based on beliefs, archaic institutional cultures, private interests and prejudices.

Cano (2012) highlights that knowledge regarding impact evaluation was not only put aside during the solidification of the social sciences in Brazil, but met with great resistance and even ideological stigmatization regarding the use of more complex quantitative methods. Causing a lack of production of this type of knowledge in the country, which ends up delaying the progress of social policies in general. Beato and Silveira (2012) highlight that the absence of a culture of evaluation on crime prevention programs makes it impossible to advance in the identification of causal relationships on what works to reduce the rates of violence, maintaining whether the idea that general social policies or very comprehensive and generic security programs will be able to reduce violence. The lack of evidence towards the effectiveness of certain actions in the country has also been highlighted by several non-systematic reviews carried out in the country (Beato & Silveira, 2014; Cano et al., 2016; Muggah & Aguirre, 2013; Willadino et al., 2011).

We did not find any Systematic Review on the evaluation of program effectiveness in Brazil, which proves the result already indicated by Abt & Winship (2016) in its worldwide review. Other reviews have already been carried out in the country on the topic of violence, addressing issues such as risk factors for violence (Murray, Castro Cerqueira & Kahn, 2013), characteristics of school violence (Nesselo et al., 2014), the relationship between violence in adolescence and cortisol (Lugarinho, Avanci & Pinto, 2017), different perspectives on the causes and consequences of violence against women (Marasca et al., 2013; Patias, Bossi & Dell’Aglio, 2014; Sá & Werlang, 2013; Schein & Martins, 2013; L. E. L. D. Silva & Oliveira, 2015; Souza, 2016), the prevalence and characteristics of violence against elderly people (Clementino, Marcolino & Oliveira, 2013; Espíndola & Blay, 2007; Minayo, Souza & Paula, 2010), among others.

Worldwide Systematic Reviews found a very small number of impact assessments already produced in the country. A global systematic review of assessments of the effectiveness of policing programs on developing countries found only five assessments made in these countries, of which only one was conducted in Brazil (Higginson, Mazerolle,Sydes, Davis & Mengersen, 2015).

2. METHODOLOGY

To carry out this systematic review, we use the requirements proposed in the Campbell Collaboration Manual (Kugley et al., 2016). The inclusion and systematization of evidence was carried out according to the criteria proposed by the Maryland Report (Sherman et al., 1997), with the aid of the Manual for Classification of Methods of the Evidence Center for Local Economic Development of the English Government (Madaleno & Waights, 2015), which helped in the analysis of the quality of the most current quasi-experimental methods, as detailed below.

The search and selection process was carried out by the research coordinator and two other research assistants, following the evidence to reduce search and typing errors (Edwards et al., 2002). At the end of each stage, systematization meetings were held and disagreements were resolved by the research coordinator. The research question (Higgins & Green, 2011) is: “what is the existing evidence in Brazil about what works to reduce homicides, robberies and rapes?”.

2.1 Inclusion criteria

To answer the research question “What is the existing evidence in Brazil about what works to reduce homicides, robberies and rapes?” and to define with transparency the inclusion criteria of the studies, with the least possible bias, we use the script of the acronym PICOS (Richardson et al., 1995), usually used in systematic reviews:

• Participants: The units of analysis (people, territories, schools, cities, states, etc.) that were affected by the intervention must be in Brazilian territory;

• Intervention: any type of intervention (social programs, prevention through personal development, policing, situational prevention, criminal justice, rehabilitation, or others), implemented by new laws, policies, programs, projects or plans, developed by public and / or private agencies in Brazil;

• Control: the studies must have used a control group, constructed from randomisation in the case of experiments, or paired, statistically controlled and comparable in its main variables before intervention in the case of quasi-experimental studies;

• Outcome: The study must have measured the outcome of the intervention on homicides, thefts and studies, with data from an official database, in any of its reported forms, such as, intentional homicide, feminicide, death caused by police;

• Methodological design: The study must have used the experimental or quasi-experimental method to compare the situation of a group that received the intervention from another group that did not receive the intervention, reaching at least level 3 of the Maryland Scale. To assist in assessing the quality of the methods used in quasi-experimental studies, we follow the criteria proposed by Madaleno and Waights (2015).

Published or unpublished studies were eligible, including studies reported in scientific articles, theses, dissertations and research reports, prepared in any language and without time restriction. In this article we will present only the results related to homicides due to the limitations regarding the length of the article.

Studies that analyzed the impact of the Crime Prevention budget, the increase in the number of police officers and the increase in incarceration were also included, as we consider that these issues are the result of political decisions and, therefore, we consider them interventions. We also included evaluations of programs that had no stated goal of reducing violent crime, but that had their impact assessed on these outcomes, following the same methodology employed by Cano et al. (2016).

After the identification that a study met the inclusion criteria, the different modules within each study were identified. The module of a study is defined based on the outcomes analyzed (homicides, robberies or rapes) in relation to each of the different interventions evaluated by the study. Thus, it is possible that a single study analyzed the effect of two different interventions on homicides and robberies, being possible to extract, in this example, four modules from only one study, which are extracted and analyzed separately, in the result spreadsheet.

2.2 Exclusion Criteria

The following studies were excluded: research on the prevalence of violence; determinants of violence, which measured the impact of economic, social and demographic factors on violence rates; cost-benefit studies; risk factors and causes of violence. Studies that assessed the impact of crime prevention programs, but that measured their results in outcomes other than homicides, robberies and rapes, were also excluded, as were studies that measured their results on violent behavior, school performance, school violence, alcohol consumption and other drugs.

2.3 Search Methodology

Several strategies were implemented in the search for studies that met the inclusion criteria. Advanced searches were carried out on nine electronic portals for scientific works, between the months of March and April 2018, in the following national portals: 1) Capes journals; 2) SciELO; 3) Virtual Library on Violence and Health (VHL / VS); 4) Capes Thesis and Dissertation Catalog. The following international portals were searched: 1) PUBMED; 2) Education Resources Information Center (ERIC); 3) PsycINFO; 4) International Bibliography of the Social Sciences (IBSS); 5) International Initiative for Impact Evaluation. As well as the search platforms: Google and Google Scholar.

The search was done using the following keywords: i) in Portuguese: policiamento, prevenção, tecnologia, segurança pública, guardas municipais, violência, crime, criminalidade, homicídio, morte violenta, crimes violentos letais intencionais, roubo, estupro, avaliação de impacto, avaliação de programa; ii) in English: Brazil, policing, crime prevention, violence, crime, homicides, robbery, thefts, robberies, burglary, recidivism, reincidence, program impact evaluation, experiments.

For each portal, a specific search strategy was developed, according to its internal rules for the use of Boolean connectors, hierarchization and organization of studies (Kugley et al., 2016). Regarding Google and Google Scholar, each of the researchers performed searches with different predefined phrases, covering the search question descriptors, within a limit of up to 400 results per search.

A great effort of manual research was also carried out, looking for gray literature, analyzing the following websites: 1) Fórum Brasileiro de Segurança Pública (Brazilian Public Security Forum) 2) Instituto Sou da Paz; 3) Igarapé Institute; 4) Centro de Estudos de Segurança e Cidadania - CESEc (Centre for Security and Citizenship Studies); 5) Centro de Estudos de Criminalidade e Segurança Pública (Crisp); 6) Instituto de Pesquisa Econômica Aplicada - IPEA (Institute of Applied Economic Research); 7) Banco Nacional do Desenvolvimento - BNDES (National Bank for Economic and Social Development), and 8) Reports of the Thinking about Security Program of the Ministry of Justice.

To complement the review and follow the recommendations of Campbell Collaboration (Kugley et al., 2016), we carried out a comprehensive active search, by sending emails to a variety of organizations including more than 400 specialists members of the Fórum Brasileiro de Segurança Pública (Brazilian Public Security Forum); 143 groups of research on security and violence of Brazilian universities registered on the Portal da Capes; 50 non-governmental organizations; various international bodies; Federal and State public security agencies; and to Courts of Accounts of the Union and the states. In addition, we made a search in the bibliographic references of all studies included in this research and an email was sent to the authors of the included articles, requesting the indication of other studies that could be analyzed.

2.4 Data Summary

The process of summarizing evidence followed the methodology proposed in the Maryland Report (Sherman et al., 1997), which is divided into two stages: i) individual analysis of the quality of the studies; ii) production of the evidence summary.

2.4.1 Individual analysis of the studies identified by the Systematic Review

Each of the studies identified in the review process and those pre-selected as impact assessments on homicide, theft and rape indicators were analyzed according to their ability to demonstrate the causal relationship and effect (internal validity), as shown in the Box below (Sherman et al., 1997).

Box 1
First Stage of the Maryland Scale - Evidence of cause and effect

Each of the study modules that met the inclusion criteria and were classified above level 3, were labeled as: i) positive, when the intervention produced a drop in violence indicators, with a significance of p <0.1, as commonly used in the literature in the area (Lipsey, 2005); ii) negative, when the intervention produced an increase in violence indicators (with p <0.1) and; iii) without impact, when the intervention did not result in a statistically significant impact.

2.4.2 Development of the Evidence Summary

After including the studies and analyzing each of the modules of said studies, the different modules were grouped into summaries of evidence whenever possible, if the main characteristics of the analyzed interventions were similar. Finally, the summary of evidence was classified according to the criteria established in the second part of the Maryland Scale, as shown in the Box below (Sherman et al., 1997).

Box 2
Second Stage of the Maryland Scale - Scale for the Definition of what works

3. RESULTS

3.1 Research results

From a total of 13,353 studies screened and analyzed, 41 studies were included in the final review, following the process shown in the flowchart below (Moher, Liberati, Tetzlaff & Altman, 2015), as shown in Figure 1.

Figure 1
Flowchart about the Systematic Review Information

The 41 studies included resulted in 86 modules, according to the characteristics highlighted on Table 1.

Table 1
Characteristics of the studies and modules included in the systematic review

	N	%		n	%
Type of publication (n = 41)			Design (n = 41)
Article	27	66	Experimental	0	0
Dissertation / Thesis	11	27	Quasi-experimental	41	100
Report	3	7	Level of the Maryland Scale (n = 41)
Year of Publication (n = 41)			Level 5	0	0
1995-2000	0	0	Level 4	5	12
2001-2005	5	12	Level 3	35	88
2006-2010	8	20
2011-2015	17	41	Type of Intervention (N = 85)
2016 - 2018	11	27	Management	5	6
			Policing	42	50
Language (n = 41)			Social Prevention	1	1
Portuguese	30	73	Situational Prevention	11	13
English	11	27	Criminal Justice	15	17
			Social Policy	6	7
Authors’ Area (n = 41)			Mixed	5	6
Economy	38	94
Social Sciences	1	2	Outcome (n = 85)
Political Science	1	2	Homicide	65	77
Business Administration	1	2	Theft	19	22
			Mixed	1	1
Place where the intervention was applied (N = 41)			Rape	0	0
Country	9	22
State	27	66	Result
Municipality	5	12	Works	52	60
			No Impact	29	35
			Worsens	4	5

• Note: In relation to the year 2018, only studies released until April, when the Review was carried out, were included. Source: Elaborated by the authors.

The summary of evidence brought forth the following results:

Box 3
What works and what does not work in reducing homicides in Brazil

	To reduce homicides
Works	Management by Results (Cabral, 2016; Cerqueira, Coelho & Silva, 2018; Soares & Viveiros, 2010)
	Fica Vivo (Staying Alive) Program (Castro, 2014; Peixoto, Andrade & Azevedo, 2008a, 2008b; Silva et al., 2016)
	Restricting hours of alcohol sale (Biderman, Mello & Schneider, 2009; Schneider, 2005; Scripilliti, 2006)
	Deploy Municipal Guards (Cabral, 2016; Ferreira, 2012; Pereira, 2016; Scripilliti, 2006)
	National Firearms Regulation Act - Disarmament Statute (Cerqueira, 2013; Cerqueira & Mello, 2013)
	National Law to prevent violence against women - Maria da Penha Law on femicides (Azuaga & Sampaio, 2017; Cerqueira et al., 2015; Schiavon, 2017)
	Pacifying Police Units - Unidade de Polícia Pacificadora - UPP on police killing (Butelli, 2015; Ferraz & Ottoni, 2013; Magaloni, Franco & Melo, 2015)
Promising	Create Municipal Security Department (Scripilliti, 2006)
	Crime Stoppers line (Scripilliti, 2006)
	Policing with a focus on weapons control (Oliveira & Rostirolla, 2017)
	Open School on weekend Program (Vasconcellos & Menezes, 2005)
	Elitrification/Public Lighting on Streets (Arvate, Falsete, Ribeiro & Souza, 2018)
	Domestic and Family Violence Court against Women on femicide (Schiavon, 2017)
	Increase the social assistance budget (Loureiro & Carvalho, 2006)
	Performance of the Judiciary (Abras, Araújo & Shikida, 2014)
	UPP on intentional homicide (Butelli, 2015; Ferraz & Ottoni, 2013)
Undefined	Police Size (Fajnzylber & Araújo, 2001; Pereira, 2016; Sachsida & Mendonça, 2013, 2014; Schuch, 2017)
	Increase on Budget of Crime Prevention (Kume, 2004; Loureiro & Carvalho, 2006; Pereira, 2016; Santos, 2009)
	Women's Police Station on feminicide (Perova & Reynolds, 2016; Schiavon, 2017)
	Gun Buyback Campaign (Scorzafave, Soares & Dorigan, 2015)
	2006 Drug Act (Sachsida & Mendonça, 2014)
	Reducing penal age (Costa et al., 2018)
Does not produce an impact	Cash Transfer Program (Bolsa Família Program) (Chioda, Mello & Soares, 2016; Loureiro, 2012)
	Increase Incarceration (Gaulez, Ferro & Moreira, 2018; Sachsida & Mendonça, 2013, 2014; Schuch, 2017)

• Source: Elaborated by the authors.

3.3 Description of the evidence that works

3.3. 1 Management by result

The intensive use of criminal analysis and the Compstat management model served as inspiration for several innovative experiences in the management of states and municipalities in the area of Security in Brazil. Although these programs were composed of several other initiatives, all of them were based on the intensive use of criminal analysis on integrated territories between the Military Police and the Civil Police, usually in management meetings, with strong political leadership involvement.

The Review found a total of 13 impact assessments of results-based management programs in relation to homicides, with 10 assessments not being included because they did not meet the inclusion criteria, the vast majority pointed out that the programs produced positive results: a) Police Project of Minas Gerais Results (Beato et al., 2001; Loschi et al., 2005); b) Sistema Integrado de Monitoramento - Rio de Janeiro’s Integrated Monitoring System (Cabral, 2016; Campagnac & Ferreira, 2013); c) Pacto Pela Vida - Pact for Life of Pernambuco (Oliveira, 2016; Penha et al., 2013; Ratton & Daudelin, 2018; B. F. A. Silva et al., 2016; Zaverucha & Nóbrega, 2015; Silveira et al., 2014).

The three studies included evaluated three different Programs, namely: Program for the Integration of Management in Public Security - IGESP, from Minas Gerais (Soares & Viveiros, 2010), the Infocrim de São Paulo (Cabral, 2016) and the Program Estado Presente (State Presence) of Espírito Santo (Cerqueira, Coelho, Silva Neto, 2018), it was concluded that this type of program produced a drop in homicides, between 9% to 17%, and in Soares and Viveiros (2010), the result did not present statistical significance, making the Management by Result Programs to be classified under “works” to reduce homicides.

3.3.2 Programa Fica Vivo (Program Stay Alive)

The Stay Alive Program was born in 2001 through an initiative by the Center for the Study of Crime and Public Security at the Federal University of Minas Gerais (CRISP / UFMG) (Beato et al., 2001), focusing on territories with high homicide rates, combining repression actions focused on violent individuals, proximity policing , social prevention programs for young people at risk and community mobilization, it currently operates, serving more than 12,500 young people at risk in 32 communities annually (Souza, 2016). The Review found nine impact assessments of the Stay Alive Program, five of which did not meet the inclusion criteria (Alves & Arias, 2012; Higginson et al., 2015; Matta & Andrade, 2005; Silveira, 2008; Silveira, Assunção, Silva & Beato, 2010) and four completed (Castro, 2014; Peixoto et al., 2008a, 2008b; B. F. A. Silva et al., 2016), which concluded that the program produces a substantial reduction in homicides in the areas that received the program, estimated at 43% reducion of the total homicides observed in the areas where the program operated between 2005 and 2015 (Silva, 2016) or from 20 to 23 homicides per 100 thousand inhabitants in the communities that received the program (Peixoto et al., 2008b), and that this result increases over time (Peixoto et al., 2008a; B. F. A. Silva et al., 2016).

3.3.3 Restriction of hours of alcoholic beverage sales

In the early 2000s, some municipalities in the metropolitan region of São Paulo established restrictions on the sale of alcoholic beverages at night. The Systematic Review found six impact assessment studies on this experience. Three of which did not meet the inclusion criteria (Duailibi et al., 2007; Kahn & Zanetic, 2005; Volpe, Ladeira & Fantoni, 2017) and three met the inclusion requirements (Biderman et al., 2009; Schneider, 2005; Scripilliti, 2006). Those included studies concluded that the restriction caused a reduction of between 10% and 20% in homicide rates, and the authors highlighted that, in addition to the approval of the law, the effectiveness of inspections was a fundamental factor in the result.

3.3.4 Disarmament Statute

In December 2003, Law No. 10,826, known as the Disarmament Statute, was passed, which severely restricted the firearms trade in the country and prohibited possession for the vast majority of the population. The Systematic Review found nine impact assessments of the Disarmament Statute, however, seven of them did not meet the inclusion criteria (Ary, Soares, Cerqueira, 2015; Hartung & Pessoa, 2009; Justus & Kassouf, 2013; Kahn & Zanetic, 2005; Souza et al., 2007; Organização das Nações Unidas para a Educação, a Ciência e a Cultura [UNESCO], 2005; Waiselfisz, 2015), and most of these studies indicated that the statute produced a significant drop in the number of homicides or hospitalizations in the country. Two studies were included (Cerqueira, 2014; Cerqueira & Mello, 2013) and concluded that the Statute caused a drop in homicide rates, estimated by Cerqueira and Mello (2013), at around 12.5%, without causing an increase in homicides caused by other means.

3.3.5 Proximity policing in communities of disorderly occupation, socially vulnerable and with strong presence of criminal groups: The Pacifying Police Units (UPPs)

The Pacifying Police Units (UPPs) were a Police Strategy program implemented by the State Government of Rio de Janeiro, in preparation for the 2014 World Cup and the 2016 Olympic Games. Between 2008 and 2014, 38 UPPs were installed, with almost 10 thousand police officers, covering a total of 700 thousand inhabitants. The Review found four UPP impact assessments, but one study did not meet the established inclusion requiriments (Cano, 2012), and three studies were included (Butelli, 2015; Magaloni et al., 2015; Ottoni & Ferraz, 2014), which identified that the UPPs caused a significant reduction in the number of deaths caused by the police, ranging from 60% to 78%, which fell from 1300 people killed by the police, in 2007, to 400, in 2013. Magaloni, Franco and Melo (2015) identified that the program was more successful in communities with a lower level of vulnerability, where levels of literacy and job opportunities were higher and in smaller rather than larger communities.

3.3.6 Maria da Penha Law

In 2006, the Maria da Penha Law (LMP) was passed, a legal framework aimed at organizing a system to prevent violence against women, including a service and protection network. The Review found the existence of three impact assessments on the Maria da Penha Law, and the three studies were included (Azuaga & Sampaio, 2017; Cerqueira, Matos, Martins & Pinto, 2015; Schiavon, 2017) and all of them concluded that the Maria da Penha Law and the implementation of its measures caused a reduction of around 10% to 19% on the homicides of women in Brazil.

3.3.7 Creation of Municipal Guards

The Review located five studies that assessed the impact of the creation of Municipal Guard on violent crimes. One did not meet the inclusion criteria (Kahn & Zanetic, 2005) and four studies were included, with two evaluating the impact only in São Paulo (Cabral, 2016; Scripilliti, 2006) and two others in relation to the entire country (Ferreira, 2012; Pereira, 2016). The four studies found that the creation of Municipal Guards led to a reduction in homicides, with Ferreira (2012) indicating that this impact was 15%. The study by Pereira (2016) pointed out that the size of (in number) of Guards is not related to their result. The author also found that the effect of the creation was greater in municipalities between 20 and 99 thousand inhabitants.

5. CONCLUSION

The results of the Review demonstrated that, although incipient, Brazil already has an expressive and relevant set of scientific evidence on what works and what does not work to reduce homicides, which is extremely relevant for the country to build efficient public policies to reduce violence, removing views based on beliefs, interests, prejudices or political populism.

The Review identified that, since the pioneering study (Fajnzylber & Araujo, 2001), the production of impact assessments has been growing since 2012, demonstrating that the Crime Prevention Evidence Based paradigm is advancing in Brazil.

While the number of localized impact assessments was relevant, the 41 studies included in this review represent only 20% of the total studies identified by the Maryland Report in 1997, which included a total of 204 impact assessments on violent crimes (Weisburd, Lum & Petrosino, 2001), demonstrating that the country, despite the severity regarding the issue of violence and the volume of resources invested in the area, still remains quite behind in the production of evidences about what works.

One of the most negative data found by the Review revolves around the fact that no Randomized Controlled Trial (RCT) on violent crimes was found, whereas the Maryland Report included 46 such studies (Weisburd et al., 2001) and another review done in 2009, found that at least 185 experiments have been carried out worldwide, only in the area of policing and Criminal Justice (Telep, 2009). Although they are more difficult to perform, as they require availability and technical knowledge on the part of decision makers and researchers, the RCTs represent a fundamental step towards evidence-based policy, due to the quality of the knowledge they produce and the impact on changing the direction of public policies (Weisburd et al., 2001).

Regarding the area of knowledge of the authors, it is important to note the great preponderance of authors in the area of economics, who were the first authors of 38 of the 41 studies (93%). This information proves the argument that has been highlighted by several authors in the country regarding the resistance of the use of quantitative methods in the social sciences in Brazil, which weakens the construction of a perspective of methodological pluralism fundamental to the improvement of public policies in the country, including in the area of violence prevention (Beato, 2002; Rolim, 2006; Willadino et al., 2011; Marino, 2012; Beato & Silveira, 2014; Cano et al., 2016; Ludmila & Niche, 2018).

Another result worth mentioning is the fact that three of the main beliefs usually spread in Brazil political debate as solutions to the problem of violence: the increase in the Public Security budget, the increase in the number of police officers and the increase in the number of prisoners, have not shown to be effective in reducing homicides. This data demonstrates that it is necessary to modify the way crime prevention is understood and made in the country, still linked to the traditional model of public security, which is summed up in reactive, no focus and fragmented actions aimed at incarceration (Carriaga & Worrall, 2015; Marvell & Moody, 1996; Lum & Koper, 2017).

It is important to highlight that these results are in line with the international evidence that has been produced since the beginning of CPEB, demonstrating that the mere increase in spending or staff does not reduce the rates of violence, if they are used in a reactive and unfocused manner (Carriaga & Worrall, 2015; Lum & Koper, 2017; Marvell & Moody, 1996).

The set of programs that work has some characteristics in common that can assist in the formulation of new effective programs in the country and that follow the general principles of effective programs at the international level (Abt & Winship, 2016; Lum & Koper, 2017; Kopittke, 2019). In general, they are:

1. proactive, as they increase the capacity for criminal analysis and allow preventive action, on criminal patterns, as in the case of the Management by Result programs;

2. highly focused, whether in certain specific territories with high levels of violence (as in the case of Stay Live and the UPPs), in certain circumstances that increases the risk of violence, like availability of weapons (as in the case of Desarmament Act), and alcohol (as show by the time of sale regulation experience), or in specific people who are at higher risk of victimization , as in the case of the Maria da Penha Law;

3. Developed in a multi-agency integrated manner, another characteristic that seems to increase the effectiveness of the programs, with a greater number of institutions involved and acting proactively and with the same focus, enhancing positive results.

It was possible to identify a high level of convergence between the results found by the Review and the results that have been indicated by the international literature, which indicates that Brazil should take advantage, in a more intensive way, of the strategies that already have scientific evidence that work at an international level, always trying to identify the local peculiarities that can hinder or enhance any given strategy.

Regarding violence against women, a total of five studies evaluated the impact of interventions on feminicides, equivalent to 12.5% ​​and identified that policies to protect women who are victims of violence have the potential to reduce the number of women murdered. However, this number is small compared to the 232 evaluations already produced in developed countries, many of them of an experimental type, especially when considering the fact that Brazil is one of the countries with the highest rates of violence against women. It is important to highlight that the use of more focused, integrated strategies can increase the effectiveness of actions (Schucan-Bird, Vigurs, Quy & Gough, 2016).

With these results, this review demonstrated that the country already has several types of programs with scientifically proven effectiveness and that the national evidence has great convergence with international evidence. For this reason, it is of great importance that this knowledge be used in decisions that are taken in the area, whether by the Executive, Judiciary or Legislative Branches, with the objective of improving the effectiveness, efficiency and legitimacy of the measures taken, reducing the rates of violence within the framework of the democratic rule of law.

6. RECOMMENDATIONS

6.1 Policy recommendations

Governments, international organizations and private funders can use systematic evidence-based knowledge to guide their decision making, in addition to supporting new assessments of policies that have promising evidence and not investing in actions in which the evidence indicates that the intervention does not work (Lum, Koper & Telep, 2011).

Another recommendation of great relevance, highlighted in the Maryland report (Sherman et al., 1997) and useful to Brazil, is that the international agencies and the federal government must do much more than transfer resources (funding), they must assume a leading role in the production and dissemination of evidence (guiding), helping states and municipalities promote change in the quality of policies developed in the area,

It is also of great importance that parliamentary houses - at federal, state or city level - use evidence when formulating new laws that may impact the levels of violence, and should build evidence production centers or partnerships with Research Centers that work with evidence.

Another relevant element for Brazil is the usage of the concept of CPEB as the center of the curricular process of training professionals in the police (Lum & Koper, 2017), which could be an initiative of the police academies and of Municipal Guards in the country.

Account Courts and the Public Prosecutor’s Office, can also promote the dissemination of the use of evidence, encouraging governments to justify the programs they develop on the basis of evidence. In this sense, it would be extremely relevant for the country to pass legislation making it mandatory to justify with cientific evidences the implementation of public policies.

Different spheres of government and institutions can also create Centers focused on the production and dissemination of evidence, along the lines of the What Works Center of the English Government, and online evidence platforms can also be created, capable of translating and disseminating the available evidence in a simple and practical manner.

6.2 Recommendations for Academic Research

The Review highlighted that today the effort in the production of evidence is primarily made by economists. It is of great importance that other areas, which have great scientific production related to issues of violence and criminology also approach the paradigm of CPEB and start producing impact assessments and systematic reviews on the interventions, with the objective of increasing their capacity to influence public policies. It is fundamental that researchers from all areas aim to develop scientific experiments to evaluate violence prevention programs and that funding agencies, public and private agencies finance this type of initiative.

ACKNOWLEDGEMENTS

Research developed under the Graduate Program in Public Policy of the Federal University of Rio Grande do Sul. The authors thank Fernando Gonçalvez, Juliana Escobar and Liciane Barbosa for their invaluable assistance in the Systematic Review process.

REFERÊNCIAS

• Abras, L. L. H; Araújo, A. F, Jr; Shikida, C. D; & Shikida, A. P. F. A. (2014). Mais armas, menos crimes? Uma análise econométrica para o estado de Minas Gerais. Rev. Ciênc. Empres, 15(1), 5-24.
• Abt, T; & Winship, C. (2016). What works in reducing community violence: a meta-review and field-study for the northern triangle (USAID Report). Washington, DC: United States Agency for International Development.
• Alves, M. C; & Arias, E. D. (2012). Understanding the Fica Vivo programme: two-tiered community policing in Belo Horizonte, Brazil. Policing and Society, 22(1), 101-113.
• Andrews, D. A; Zinger, I; Hoge, R. D; Bonta, J; Gendreau, P; & Cullen, F. T. (1990). Does correctional treatment work? A psychologically informed meta-analysis. Criminology, 28, 369-404.
• Arvate, P; Falsete, F. O; Ribeiro, F. G; & Souza, A. P. (2018). Lighting and homicides: Evaluating the effect of an electrification policy in rural Brazil on violent crime reduction. J Quant Criminol, 34, 1047-1078.
• Ary, G; Soares, D; & Cerqueira, D. R. C. (2015). Estatuto do desarmamento: Um tiro que não saiu pela culatra. Insight - Inteligência, 68(17), 78-86.
• Azuaga, F. L; & Sampaio, B. (2017). Violência contra mulher: o impacto da Lei Maria da Penha sobre o feminicídio no Brasil. In Anais do 45o Encontro ANPEC, Natal, RN.
• Beato, C. (2002). Crime and social policies in Latin America: problems and solutions Washington, DC: Woodrow Wilson Center Update on the Americas.
• Beato, C; Assunção, R. M; Silva, B. F. A. D; Marinho, F. C; Reis, I. A; & Almeida, M. C. D. M. (2001). Conglomerados de homicídios e o tráfico de drogas em Belo Horizonte, Minas Gerais, Brasil, de 1995 a 1999. Cadernos de Saúde Pública, 17, 1163-1171.
• Beato, C; & Silveira, A. M. (2014). Efetividade e avaliação em programas de prevenção ao crime em Minas Gerais (Artigos Estratégicos). Rio de Janeiro, RJ: Instituto Igarapé.
• Biderman, C; Mello, J. M; & Schneider, A. (2009). Dry laws and homicides: evidence from the São Paulo metropolitan area. The Economic Journal, 120(543), 157-182.
• Bieck, W; & Kessler, D. A. (1977). Response team analysis Kansas City, MO: Board of Police Commissioners.
• Butelli, P. H. (2015). Avaliação de impacto de políticas de segurança: o caso das Unidades de Polícia Pacificadora (UPPs) no Rio de Janeiro (Tese de Doutorado). Fundação Getulio Vargas, São Paulo, SP.
• Cabot, R. D. Q. (1940). A long-term study of children: the Cambridge-Somerville Youth Study. Child Development, 11, 143-151.
• Cabral, M. V. F. (2016). Avaliação do impacto do Infocrim sobre as taxas de homicídios dos municípios paulistas: uma aplicação do método de diferenças em diferenças espacial (Tese de Doutorado). Universidade Federal de Juiz de Fora, Juiz de Fora, MG.
• Campagnac, V; Ferreira, M. (2013). Avaliação do Impacto do Uso da Informação na Prevenção de roubo a transeunte no Estado do Rio de Janeiro. In Anais do 6o Congresso Consad, Rio de Janeiro, RJ.
• Campos, M. S. (2010). Crime e Congresso Nacional no Brasil pós-1988: uma análise da política criminal aprovada de 1989 a 2006 (Dissertação de Mestrado). Universidade Estadual de Campinas, Campinas, SP.
• Cano, I. (2012). Nas trincheiras do método: o ensino da metodologia das ciências sociais no Brasil. Sociologias, 14(31), 94-119.
• Cano, I; Rojido, E; Chaves, R; Bezerra, J; & Sá, B. (2016). Mapeamento de programas de prevenção de homicídios na América Latina e Caribe Rio de Janeiro, RJ: Universidade do Estado do Rio de Janeiro.
• Carriaga, M. L; & Worrall, J. L. (2015). Police levels and crime. The Police Journal: Theory, Practice and Principles, 88(4), 315-333.
• Castro, T. E. (2014). Avaliação de impacto do Programa Fica Vivo! sobre a taxa de homicídios em Minas Gerais (Dissertação de Mestrado). Universidade Federal de Viçosa, Viçosa, MG.
• Cerqueira, D. (2014). Causas e consequências do crime no Brasil Rio de Janeiro, RJ: Banco Nacional de Desenvolvimento Econômico e Social.
• Cerqueira, D; Matos, M; Martins, A. P. A; & Pinto, J; Jr. (2015). Avaliando a efetividade da Lei Maria da Penha (Texto para Discussão IPEA n. 2048). Brasília, DF: Instituto de Pesquisa Econômica Aplicada.
• Cerqueira, D; & Mello, J. M. P. (2013). Evaluating a national anti-firearm law and estimating the causal effect of guns on crime (Texto para Discussão n. 607). Rio de Janeiro, RJ: Universidade Católica do Rio de Janeiro.
• Chioda, L; Mello, J. M. P; & Soares, R. R. (2012, fevereiro) Spillovers from Conditional Cash Transfer Programs: Bolsa Família and Crime in Urban Brazil (Working Paper, 024) . Rio de Janeiro, RJ: Rede de Economia Aplicada.
• Clementino, F. S; Marcolino, E. C; & Oliveira, K. K. D. (2013). Uma revisão sistemática acerca da violência doméstica contra idosos: conhecendo a vítima. In Anais do 3o Congresso Internacional do Envelhecimento Humano, Campina Grande, PB.
• Costa, F. J. M; Faria, J. S; Iachan, F. S; & Caballero, B. (2018, Fall). Homicides and the Age of Criminal Responsibility in Brazil: A Density Discontinuity Approach. Economìa, 19(1), 59-62.
• Duailibi, S; Ponicki, W; Grube, J; Pinsky, I; Laranjeira, R; & Raw, M. (2007). The effect of restricting opening hours on alcohol-related violence. American Journal of Public Health, 97(12), 2276-2280.
• Edwards, P; Clarke, M; DiGuiseppi, C; Pratap, S; Roberts, I; & Wentz, R. (2002). Identification of randomized controlled trials in systematic reviews: accuracy and reliability of screening records. Statistics in Medicine, 21(11), 1635-1640.
• Ennet, S. T; Rosenbaum, D. P; Flewelling R. L; Bieler, G. S; Ringwalt, C. L; & Bailey, S. L. (1994). Long-term evaluation of drug abuse resistance education. Addictive Behaviors, 19(2), 113-125.
• Espíndola, C. R; & Blay, S. L. (2007). Prevalência de maus-tratos na terceira idade: revisão sistemática. Revista de Saúde Pública, 41, 301-306.
• Fajnzylber, P; & Araujo, A. F; Jr. (2001). O que causa a criminalidade violenta no Brasil? Uma análise a partir do modelo econômico do crime: 1981 a 1996 (Texto para Discussão Cedeplar n. 162). Belo Horizonte, MG: Universidade Federal de Minas Gerais.
• Ferreira, L. R. C. (2012). O papel das guardas municipais na redução de homicídios: evidências empíricas para o Brasil (Dissertação de Mestrado). Fundação Getulio Vargas, São Paulo, SP.
• Gaulez, M. P; Ferro, A. R; & Moreira, G. C. (2018). O efeito do encarceramento de homicidas sobre a taxa de homicídios no Brasil. EALR, 9(2), 288-307.
• Global Burden of Disease Collaborative Network. (2017). Global burden of disease study Seattle, WA: Autor.
• Hartung, G; & Pessoa, S. (2009). O papel das armas de fogo na queda dos homicídios em São Paulo. In Anais do 37o Encontro ANPEC, Foz do Iguaçu, PR.
• Higgins, J. P. T; & Green, S. (Eds.). (2011). Cochrane handbook for systematic reviews of interventions Hoboken, NJ: John Wiley & Sons.
• Higginson, A; Mazerolle, L; Sydes, M; Davis, J; & Mengersen, K. (2015). Policing interventions for targeting interpersonal violence in developing countries: a systematic review (3ie Systematic Review, No. 20). London, England: International Initiative for Impact Evaluation.
• Huges, K; Bellis, M; Hardcastle, K; Butchart, A; Dahlberg, L; Mercy, J; & Mikton, C. (2014). Global development and diffusion of outcome evaluation research for interpersonal and self-directed violence prevention from 2007 to 2013: A systematic review. Aggress Violent Behav, 19(6), 655-662.
• Instituto de Pesquisa Econômica Aplicada, & Fórum Brasileiro de Segurança Pública. (2019). Atlas da violência 2019 Brasília, DF: Autor.
• Justus, M; & Kassouf, A. L. (2013). Evidence of the effect of wealth observed by criminals on the risk of becoming a victim of property crimes. Economia, 14, 88-101.
• Kahn, T; & Zanetic, A. (2005, julho). O Papel dos Municípios na Segurança Pública (Estudos Criminológicos, 4). São Paulo, SP: Observatório de Segurança Pública. Recuperado de http://www.observatoriodeseguranca.org/files/Papel%20dos%20munic%C3%ADpios%20na%20Seguran%C3%A7a%20P.pdf
  » http://www.observatoriodeseguranca.org/files/Papel%20dos%20munic%C3%ADpios%20na%20Seguran%C3%A7a%20P.pdf
• Kelling, G. L; Pate, T; Dieckman, D; & Brown, C. (1974). The Kansas City Preventive Patrol Experiment: a technical report Washington, DC: Police Foundation.
• Kennedy, D. M. (1997). Pulling levers: chronic offenders, high-crime settings, and a theory of prevention. Valparaiso University Law Review, 31, 449-484.
• Kopittke, A. L. W. (2016). Segurança pública e democracia. Uma história de desencontros Dissertação de Mestrado. Pontifícia Universidade Católica do Rio Grande do Sul, Porto Alegre, RS.
• Kopittke, A. L. W. (2019). Cinco Princípios sobre o que funciona para reduzir a violência. In Proceedings of Sociology of Law 2019: o direito na sociedade tecnológica (pp. 31-650). Canoas, RS: Unilasalle.
• Kugley, S; Wade, A; Thomas, J; Mahood, Q; Jørgensen, A. M. K; Hammerstrøm, K; & Sathe, N. (2016). Searching for studies: a guide to information retrieval for Campbell Systematic Reviews. Campbell Systematic Reviews, 13(1), 1-73
• Kume, L. (2004). Uma estimativa dos determinantes da taxa de criminalidade brasileira: uma aplicação em painel dinâmico Rio de Janeiro, RJ: FGV EPGE.
• Lei Lei n. 10.826, de 22 de dezembro de 2003 (2003). Dispõe sobre registro, posse e comercialização de armas de fogo e munição, sobre o Sistema Nacional de Armas - Sinarm, define crimes e dá outras providências. Brasília, DF.
• Lei Lei n. 11.340, de 7 de agosto de 2006 (2006). Cria mecanismos para coibir a violência doméstica e familiar contra a mulher, nos termos do § 8º do art. 226 da Constituição Federal, da Convenção sobre a Eliminação de Todas as Formas de Discriminação contra as Mulheres e da Convenção Interamericana para Prevenir, Punir e Erradicar a Violência contra a Mulher; dispõe sobre a criação dos Juizados de Violência Doméstica e Familiar contra a Mulher; altera o Código de Processo Penal, o Código Penal e a Lei de Execução Penal; e dá outras providências. Brasília, DF.
• Lipsey, M. W. (2009). The primary factors that characterize effective interventions with juvenile offenders: a meta-analytic overview. Victims and Offenders, 4(2), 124-147.
• Lipsey, M. W; Landenberger, N. A; & Wilson, S. J. (2007). Effects of cognitive-behavioral programs for criminal offenders. Campbell Systematic Reviews, 6, 1-27.
• Loschi, R. H; Gonçalves, F. B; & Cruz, F. R. B. (2005). Avaliação de uma medida de evidência de um ponto de mudança e sua utilização na identificação de mudanças na taxa de criminalidade em Belo Horizonte. Pesqui. Oper; 25(3), 449-463.
• Loureiro, A. (2012). Can Conditional Cash Transfers Reduce Poverty and Crime? Evidence from Brazil Edinburgh, UK: University of Edinburgh.
• Loureiro, A. O. F; & Carvalho, J. R. (2006). O impacto dos gastos públicos sobre a criminalidade no Brasil (Ensaios sobre Pobreza do CAEN, 9). Fortaleza, CE: CAEN.
• Lugarinho, L. P; Avanci, J. Q; & Pinto, L. W. (2017). Perspectivas dos estudos sobre violência na adolescência e cortisol: revisão bibliográfica sistemática. Ciência & Saúde Coletiva, 22, 1321-1332.
• Lum, C; & Koper, C. S. (2017). Evidence‐based policing: translating research into practice. Oxford, England: Oxford University Press.
• Lum, C; Koper, C. S; & Telep, C. W. (2011). The evidence-based policing matrix. Journal of Experimental Criminology, 7(1), 3-26.
• Madaleno, M; & Waights, S. (2015). Guide to scoring methods using the Maryland scientific methods scale London, England: What Works Centre for Local Economic Growth.
• Magaloni, B; Franco, E; & Melo, V. (2015). Killing in the slums: an impact evaluation of police reform in Rio de Janeiro (Working Paper No. 556). Palo Alto, CA: Stanford.
• Marasca, A. R; Colossi, P. M; & Falcke, D. (2013). Violência conjugal e família de origem: uma revisão sistemática da literatura de 2006 a 2011. Temas em Psicologia, 21(1), 221-243.
• Marino, J. M. F. (2012). Fundamentos do ‘Paradigma Metodológico Causal’ nas Ciências Sociais, Impropriamente denominado ‘Métodos Quantitativos’. Sociologias, 14(31), 20-50.
• Marvell, T. B; & Moody, C. E. (1996). Specification problems, police levels, and crime rates. Criminology, 34(4), 609-646.
• Matta, R. A; Andrade, M. V. (2005). Economic evaluation of the impact of program control homicide Stay Alive. In Proceedings of the 33rd Brazilian Economics Meeting, Niterói, RJ.
• Minayo, M. C. D. S; Souza, E. R. D; & Paula, D. D. R. D. (2010). Revisão sistemática da produção acadêmica brasileira sobre causas externas e violências contra a pessoa idosa. Ciência & Saúde Coletiva, 15, 2719-2728.
• Ministério da Defesa. (2018). Pedido de Acesso à Informação 60502000911201845 Brasília, DF: Autor.
• Moher D; Liberati A; Tetzlaff J; Altman D. G. (2009) Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med, 6(7): e1000097.
• Muggah, R; & Aguirre, K. (2013). Mapping citizen security interventions in Latin America: reviewing the evidence (NOREF Report). Oslo, Norway: Norwegian Peacebuilding Resource Centre.
• Murray, J; Castro Cerqueira, D. R; & Kahn, T. (2013). Crime and violence in Brazil: systematic review of time trends, prevalence rates and risk factors. Aggression and Violent Behavior, 18(5), 471-483.
• Nessello, F; Sant’Anna, F. L; dos Santos, H. G; de Andrade, S. M; Mesas, A. E; & González, A. D. (2014). Características da violência escolar no Brasil: revisão sistemática de estudos quantitativos. Revista Brasileira de Saúde Materno Infantil, 14(2).
• Newman, O; & Franck, K. A. (1982). The effects of building size on personal crime and fear of crime. Population and Environment, 5(4), 203-220.
• Oliveira, C; & César Rostirolla, C. (2017). Mais armas de fogo, mais homicídios? Uma evidência empírica para a Região Metropolitana de Porto Alegre a partir de dados em painel. In Anais do 45o Encontro Nacional de Economia, Natal, RN.
• Oliveira, J. C. L. (2016). Avaliação dos resultados do Pacto pela Vida e a dinâmica dos homicídios nos municípios de Pernambuco (Dissertação de mestrado). Universidade Federal de Pernambuco, Recife, PE.
• Organização das Nações Unidas para a Educação, a Ciência e a Cultura. (2005). Vidas Poupadas: o impacto do desarmamento no Brasil Paris, France: Autor. Recuperado de http://unesdoc.unesco.org/images/0014/001408/140846por.pdf%0A
  » http://unesdoc.unesco.org/images/0014/001408/140846por.pdf%0A
• Ottoni, B; & Ferraz, C. (2014). Três ensaios em microeconometria sobre crime, política e migração (Tese de Doutorado). Pontifícia Universidade Católica do Rio de Janeiro. Rio de Janeiro, RJ.
• Patias, N. D; Bossi, T. J; & Dell’Aglio, D. D. (2014). Repercussões da exposição à violência conjugal nas características emocionais dos filhos: revisão sistemática da literatura. Temas em Psicologia, 22(4), 901-915.
• Peixoto, B. T; Andrade, M. V; & Azevedo, J. P. (2008a). Avaliação econômica do Programa Fica Vivo: o caso piloto (Texto para Discussão Cedeplar n. 336). Belo Horizonte, MG: Universidade Federal de Minas Gerais.
• Peixoto, B. T; Andrade, M. V; & Azevedo, J. P. (2008b). Prevenção e controle de homicídios: uma avaliação de impacto no Brasil Belo Horizonte, MG: Universidade Federal de Minas Gerais.
• Penha, E. F. F; Pontes, E; Jr ; & Vasconcelos, A. L. F. S. (2013). Políticas de Segurança Pública em Combate à Criminalidade no Estado de Pernambuco, Brasil Recuperado de http://docplayer.com.br/14598334-Politicas-deseguranca-publica-em-combate-a-criminalidade-no-estado-de-pernambuco-brasil.html
  » http://docplayer.com.br/14598334-Politicas-deseguranca-publica-em-combate-a-criminalidade-no-estado-de-pernambuco-brasil.html
• Pereira, O. A Filho . (2016). Três ensaios sobre mensuração de eficiência e avaliação de impacto em serviços de segurança pública no Brasil (Tese de Doutorado). Universidade de Brasília, Brasília, DF.
• Perova, E; & Reynolds, S. (2015). Women’s Police Stations and Domestic Violence. Policy Research (Working Paper 7497). Washington, DC: World Bank Group.
• Ramos, M. P. (2013). Métodos quantitativos e pesquisa em ciência social: lógica e utilidade do uso da quantificação nas explicações dos fenômenos sociais. Mediações - Revista de Ciências Sociais, 18(1), 55-65.
• Ramos, M. P. (2014). Pesquisa social: abordagem quantitativa com uso do SPSS Porto Alegre, RS: Escritos.
• Ramos, M. P; & Schabbach, L. M. (2012). O Estado da arte da avaliação de políticas públicas: conceituação e exemplos no Brasil. Revista de Administração Pública, 46(5), 1271-1294.
• Ratton, J. L; & Daudelin, J. (2018). Construction and Deconstruction of a Homicide Reduction Policy: The Case of Pact for Life in Pernambuco, Brazil. International Journal of Criminology and Sociology, 7, 173-183.
• Ribeiro, L; & Teixeira, A. N. (2017) O calcanhar de Aquiles dos estudos sobre crime, violência e dinâmica criminal. Revista Brasileira de Informação Bibliográfica em Ciências Sociais - BIB, 84(2), 13-80.
• Rolim, M. (2006). A síndrome da rainha vermelha Policiamento e segurança pública no século XXI Rio de Janeiro, RJ: Jorge Zahar.
• Sá, S. D; & Werlang, B. S. G. (2013). Personalidade de mulheres vítimas de violência doméstica: uma revisão sistemática da literatura. Contextos Clínicos, 6(2), 106-116.
• Sachsida, A; & Mendonça, M. (2013). Evolução e Determinantes da Taxa de Homicidios no Brasil (Texto para discussão 1808) . Rio de Janeiro, RJ: Instituto de Pesquisa Econômica Aplicada.
• Sachsida, A; & Mendonça, M. (2014). Combatendo Homicídios no Brasil: O que Funciona em São Paulo Funciona na Bahia? (Discussion Papers, 1979). Rio de Janeiro, RJ: Instituto de Pesquisa Econômica Aplicada .
• Santos, M. J. (2009). Dinâmica temporal da criminalidade: mais evidências sobre o efeito inércia nas taxas de crimes letais nos estados brasileiros. Revista Economia/ANPEC, 10(1), 170-194.
• Santos, T; & Rosa, M. I. (2017). Levantamento Nacional de Informações Penitenciárias (Infopen) . Brasília, DF: Departamento Penitenciário Nacional.
• Schein, S; & Martins, R. V. (2013). Revisão sistemática sobre a atenção à saúde da mulher vítima de violência (Monografia de Especialização Lato Sensu). Universidade Federal de Santa Maria, Santa Maria, RS.
• Schiavon, L. C. (2017). Essays on crime and justice (Tese de Doutorado). Pontifícia Universidade Católica do Rio de Janeiro, Rio de Janeiro, RJ.
• Schneider, A. A. (2005). Determinantes da criminalidade na Região Metropolitana de São Paulo Dissertação de Mestrado. Fundação Getulio Vargas, São Paulo, SP.
• Schucan-Bird, K; Vigurs, C; Quy, K; & Gough, D. (2016). Criminal justice interventions with perpetrators or victims of domestic violence: a systematic map of the empirical literature (Crime Reduction Systematic Review Series No 3) . London, England: What Works Centre for Local Economic Growth.
• Schuch, R. C. (2017). Determinantes da Criminalidade na Região Metropolitana de Porto Alegre-RS: Teorias e Evidências. Dissertação defendida ao Programa de Pós-Graduação em Economia da UFRGS. Porto Alegre, RS.
• Scorzafave, L. G; Soares, M. K; & Dorigan, T. A. (2015). Vale a pena pagar para desarmar? Uma avaliação do impacto da campanha de entrega voluntária de armas sobre as mortes com armas de fogo. Estud. Econ; 45(3), 475-497.
• Scripilliti, E. A. O. (2006). Crimes nos municípios paulistas: um estudo acerca dos condicionantes sócio-econômico e demográficos que contribuem para maior criminalidade e quais os efeitos das diferentes políticas (Dissertação de Mestrado). Universidade de São Paulo, São Paulo, SP.
• Seguridad, Justicia y Paz. (2019). Las 50 ciudades más violentas del mundo 2018 México, DF: Autor.
• Sherman, L. W. (1998). Evidence-based policing. Ideas in American policing Washington, DC: Police Foundation.
• Sherman, L. W; Gottfredson, D. C; MacKenzie, D. L; Eck, J; Reuter, P; & Bushway, S. (1997). Preventing crime: what works, what doesn’t, what’s promising: a report to the United States Congress Washington, DC: US Department of Justice.
• Sherman, L. W; & Weisburd, D. (1995). General deterrent effects of police patrol in crime “hot spots”: a randomized, controlled trial. Justice Quarterly, 1 ( 24), 625-648.
• Silva, B. F. A; Queiroz, B. L; Marinho, F. C; Caminhas, D. A; Pereira, F. N. A; Cisalpino, P. ... & Gomes, G. (2016). Políticas públicas de prevenção e redução de homicidios: a experiência do “Fica Vivo”, em Belo Horizonte e “Pacto pela Vida”, em Pernambuco (Relatório Final do Projeto Homicídios no Brasil) . Belo Horizonte, MG: Ministério da Justiça; Secretaria Nacional de Segurança Pública; Programa das Nações Unidas para o Desenvolvimento.
• Silva, D. R; Cerqueira Neto, D. R. C; & Coelho, D. S. C. (2018). Avaliação de política pública para redução de homicídio: um estudo de caso do Programa Estado Presente do Espírito Santo. In Anais do 46o Encontro Nacional de Economia, Rio de Janeiro, RJ.
• Silva, L. E. L. D; & Oliveira, M. L. C. D. (2015). Violência contra a mulher: revisão sistemática da produção científica nacional no período de 2009 a 2013. Ciência & Saúde Coletiva, 20, 3523-3532.
• Silveira, A M. (2008). A Prevenção de Homicídios: a experiência do Programa Fica Vivo no Morro das Pedras. Revista Educação & Realidade, 33(2), 163-176.
• Silveira, A. M; Assunção, R. M; Silva, B. A. F; & Beato, C. C Filho . (2010). Impacto do Programa Fica Vivo na redução dos homicídios em comunidade de Belo Horizonte. Rev. Saúde Pública, 44(3), 496-502.
• Silveira, R. M Filho , Ratton, J. L; Menezes, T. A; & Monteiro, C. (2014). Avaliação De Política Pública Para Redução Da Violência: O Caso Do Programa Pacto Pela Vida Do Estado De Pernambuco. In Anais do 41o Encontro Nacional de Economia, Brasília, DF.
• Soares, L. E. (2006). Legalidade libertária Rio de Janeiro, RJ: Lumen Juris.
• Soares, R; & Viveiros, I. (2010). Organization and information in the fight against crime: an evaluation of the integration of police forces in the State of Minas Gerais, Brazil (Discussion Paper Series, No. 5270). Bonn, Germany: Institute for the Study of Labor.
• Souza, M. F. M; Macinko, J; Alencar, A. P; Malta, D. C; & Morais, O. L. ; M Neto . (2007). Reductions In Firearm-Related Mortality And Hospitalizations In Brazil After Gun Control. Health Affairs, 26(2), 575-584.
• Souza, T. A. (2016). A implementação da política estadual de prevenção social à criminalidade de Minas Gerais: paradigmas, coalizões e incrementalismo (Dissertação de Mestrado). Fundação João Pinheiro, Belo Horizonte, MG.
• Spelman, W; & Brown, D. K. (1981). Calling the police: citizen reporting of serious crime Washington, DC: Police Executive Research Forum.
• Telep, C. W. (2009). Citation Analysis of Randomized Experiments in Criminology and Criminal Justice: A research note. Journal of Experimental Criminology, 5, 441-64.
• Vasconcellos, L; & Menezes, N Filho . (2005). Relatório de Avaliação Econômica - Programa Escola da Família da Secretaria de Educação do Estado de São Paulo São Paulo, SP: Fundação Itaú Social.
• Volpe, F. M; Ladeira, R. M; & Fantoni, R. (2017). Evaluating the Brazilian zero tolerance drinking and driving law: Time series analyses of traffic-related mortality in three major cities. Traffic Injury Prevention, 18(4), 337-343.
• Waar, J. (2019, julho). What Works? A systematic overview of recently published meta evaluations / synthesis studies within the knowledge domains of Situational Crime, Prevention, Policing and Criminal Justice Interventions, 1997-2019 (Working Paper).
• Waiselfisz, J. J. (2015). Mapa da violência 2015: adolescentes de 16 e 17 anos do Brasil: mortes matadas por armas de fogo. Brasília, DF: Centro Brasileiro de Estudos Latino-Americano. Recuperado de http://www.mapadaviolencia.org.br/pdf2015/mapaViolencia2015.pdf%0A
  » http://www.mapadaviolencia.org.br/pdf2015/mapaViolencia2015.pdf%0A
• Weikart, D. P; Deloria, D; Lawser, S; & Wiegerink, R. (1970). Longitudinal results of the Ypsilanti Perry Preschool Project (Monographs of the High/Scope Educational Research Foundation, No. 1). Ypsilanti, MI: High/Scope Press.
• Weisburd, D; Farrington, D. P; & Gill, C. (2017). What works in crime prevention and rehabilitation: an assessment of systematic review. Criminology & Public Policy, 16(2), 415-449.
• Weisburd, D; Lum, C. M; & Petrosino, A. (2001). Does research design affect study outcomes in criminal justice? The Annals of the American Academy of Political and Social Science, 578(1), 50-70.
• Welsh, Brandon C; & Farrington, David P. (2012). Crime Prevention and Public Policy. The Oxford Handbook of Crime Prevention Oxford, UK: Oxford Handbooks.
• Willadino, R., Sento-Sé, J. T., Dias, C. G., & Gomes, F. (2011). Prevenção à violência e redução de homicídios de adolescentes e jovens no Brasil Rio de Janeiro, RJ: Observatório de Favelas.
• Zaverucha, J., & Nóbrega, J. M. P. Jr . O Pacto pela Vida, os tomadores de decisão e a redução da violência homicida em Pernambuco. Dilemas: Revista de Estudos de Conflito e Controle Social, 8(2), 235-252. 2015

• 1
  This scale will be explained later in this article.

• [Translated version] Note: All quotes in English translated by this article’s translator.

Publication Dates

• Publication in this collection
  05 May 2021
• Date of issue
  Mar-Apr 2021

History

• Received
  15 May 2019
• Accepted
  28 Feb 2020