Acessibilidade / Reportar erro

Conceptual reflections on co-production of public policies and directions for future research

Abstract

Co-production has been pointed out as the main topic in public administration over the last decade, due to the fact that the delivery of many public services depends on the active participation of users. This approach however is still not used to its full potential because of the fragmentation and lack of systematization of the concepts presented in the literature on co-production. This study aims to contribute in this sense, identifying and discussing the elements that might form a more robust and integrated conceptual framework, supporting the research and practice on co-production in public policy. The literature review is based on the classic research produced on the subject, which has set the guidelines for the literature on co-production over the last 40 years. By systematizing and discussing the relevant concepts, this study intends to identify the relationships existing among them, as well as the potential for new interrelationships. Thus the main contribution of this article is to identify the concepts of co-production that are found in theoretical framework and point out the relationships among them. The discussion provided suggests a broad and rich agenda for future research on co-production.

Keywords:
Agenda; Co-production; Social participation; Public policy.

Resumo

A abordagem de coprodução tem sido apontada como a principal temática em administração pública na última década, pois muitos serviços públicos dependem da participação ativa dos usuários na entrega dos serviços. No entanto, a fragmentação e a falta de sistematização dos conceitos encontrados na literatura contribuem com a não materialização do potencial dessa abordagem. A fim de superar esse entrave, neste artigo são identificados e discutidos os conceitos da literatura de coprodução que servem para traçar um quadro teórico mais robusto e integrado para fundamentar o uso da abordagem em políticas públicas e, também, seu estudo. A revisão bibliográfica empreendida para atingir esse objetivo voltou-se, principalmente, aos trabalhos clássicos sobre o assunto, pois foram eles que “pavimentaram” a literatura ao longo dos últimos 40 anos. Por meio da sistematização e discussão dos conceitos relevantes, procura-se possibilitar a futura identificação de pontos de aderência que eles compartilham entre si, bem como o potencial que oferecem para estabelecimento de novas inter-relações. A identificação dos conceitos componentes de um marco teórico compreensivo de coprodução e a indicação de como identificar suas inter-relações são as contribuições principais deste artigo. A discussão empreendida abre uma ampla e rica agenda de pesquisas em coprodução.

Palavras-chave:
Agenda; Coprodução; Participação Social; Políticas Públicas

Resumen

El abordaje de la coproducción ha sido identificado como el tema principal en la administración pública en la última década, ya que muchos servicios públicos dependen de la participación activa de los usuarios en la prestación de servicios. Sin embargo, la fragmentación y la falta de sistematización de los conceptos encontrados en la literatura no contribuyen a la materialización del potencial de ese abordaje. Para superar ese obstáculo, este artículo tiene como objetivo identificar y discutir los conceptos de la literatura sobre coproducción como un intento de esbozar un marco teórico más sólido e integrado para fundamentar el uso del abordaje en la implementación de las políticas públicas y, también, su estudio. La revisión bibliográfica realizada se basa principalmente en los trabajos clásicos sobre el tema, pues estos cimentaron la literatura durante los últimos 40 años. A través de la sistematización y discusión de los conceptos relevantes, se busca posibilitar la futura identificación de puntos de adhesión que estos compartan entre sí, así como el potencial que ofrezcan para el establecimiento de nuevas interrelaciones. La comprensión de estos conceptos y cómo podrían interrelacionarse es una de las contribuciones de este artículo. Además, la discusión emprendida abre posibilidades para una amplia y abundante agenda de investigación sobre coproducción.

Palabras clave:
Agenda; Coproducción; Participación Pocial; Políticas Públicas

INTRODUCTION

Public policies are defined by Howlett and Ramesh (2003HOWLETT, M.; RAMESH, M. Studying public policy: policy cycles and policy subsystems. 2. ed. Oxford: Oxford University Press, 2003.) as the choices government makes in order to undertake some course of action. The authors present in their policy cycle the basic stages of formulation, implementation and evaluation (FREY, 2000FREY, K. Políticas públicas: um debate conceitual e questões referentes à prática da análise de políticas públicas no Brasil. Planejamento e Políticas Públicas, n. 21, p. 211-260, 2000.; DELEON and DELEON, 2002DELEON, P.; DELEON, L. What ever happened to policy implementation? An alternative approach. Journal of Public Administration Research and Theory, v. 12, n. 4, p. 467-492, 2002.). There is general understanding that this perfect consecutive ordering (linearity) of stages hardly occurs in practice, but this framework is a relevant reference for the analysis of public policies (HAM and HILL, 1993HAM, C.; HILL, M. The policy process in the modern capitalist State. 2. ed. London: Harvester Wheatsheaf, 1993.; FREY, 2000FREY, K. Políticas públicas: um debate conceitual e questões referentes à prática da análise de políticas públicas no Brasil. Planejamento e Políticas Públicas, n. 21, p. 211-260, 2000.). Although recognizing all the stages are essential, the reflections of this study are mainly directed to the implementation stage.

For O’Toole Junior (2000)O’TOOLE JUNIOR, L. J. Research on policy implementation: assessment and prospects. Journal of Public Administration Research and Theory, v. 10, n. 2, p. 263-288, 2000. implementation is the developments happening between the establishment of the government’s apparent intention of doing something and the impact of the action in reality. According to Palumbo and Calista (1990PALUMBO, D. J.; CALISTA, D. J. Implementation and the policy process: opening up the black box. New York: Greenwood, 1990.), the implementation stage is highlighted in the policy cycle when it is recognized as the “black box” 1 1 Other authors use the terms “lost link” or “puzzle” in order to deal with the problems in the implementation state - see Berman (1978) and O’Toole Junior (2000), respectively. for its effectiveness. Berman (1978BERMAN, P. The study of macro and micro implementation. Public Policy, v. 26, n. 2, p. 157-184, 1978.) and Winter (2010WINTER, S. C. Perspectivas de implementação: status e reconsideração. In: PETERS, G. B.; PIERRE, J. (Org.). Administração pública: coletânea. Brasília, DF: Ed. Enap, 2010. p. 209-228.) describe implementation as the stage where the differences between the inputs and outputs of the process are observed. The authors demonstrate that this relationship (input/output) is not well understood.

According to Barrett (2004BARRETT, S. M. Implementation studies: time for a revival? Public Administration, v. 82, n. 2, p. 249-262, 2004.), implementation studies aim to identify the causes of the problems observed, as well as to indicate solutions to increase the probability of success in the execution of the policy. Brinkerhoff (1996BRINKERHOFF, D. W. Process perspectives on policy change: highlighting implementation. World Development, v. 24, n. 9, p. 1395-1401, 1996.) argues that some alternative action was designed to address these problems during the implementation phase, in which several structures emerged and were combined to include: agency restructuring and decentralization, inter-organizational networks, delegation of functions to nonprofit organizations, public-private partnerships and privatizations. However, many of these alternative actions are still adopted in terms of management, neglecting citizens’ engagement and empowerment in public policy processes (DENHARDT, 2012DENHARDT, R. B. Teorias da administração pública. São Paulo: Cengage Learning, 2012.; PAULA, 2012PAULA, A. P. P. Por uma nova gestão pública: limites e potencialidades da experiência contemporânea. Rio de Janeiro: Ed. FGV, 2012.).

In this sense, some scholars of public policy recommend the adoption of more democratic ways of dealing with the delivery of public goods and services (BRINKERHOFF, 1996BRINKERHOFF, D. W. Process perspectives on policy change: highlighting implementation. World Development, v. 24, n. 9, p. 1395-1401, 1996.; O’TOOLE JUNIOR, 2000O’TOOLE JUNIOR, L. J. Research on policy implementation: assessment and prospects. Journal of Public Administration Research and Theory, v. 10, n. 2, p. 263-288, 2000.; DENHARDT, 2012DENHARDT, R. B. Teorias da administração pública. São Paulo: Cengage Learning, 2012.; PAULA, 2012PAULA, A. P. P. Por uma nova gestão pública: limites e potencialidades da experiência contemporânea. Rio de Janeiro: Ed. FGV, 2012.; and RANTAMAKI, 2017RANTAMAKI, N. J. Co-production in the context of Finnish social services and health care: a challenge and a possibility for a new king of democracy. Voluntas: International Journal of Voluntary and Nonprofit Organizations, v. 28, n. 1, p. 248-264, 2017.), suggesting they are a promising direction for the field of implementation (DELEON and DELEON, 2002DELEON, P.; DELEON, L. What ever happened to policy implementation? An alternative approach. Journal of Public Administration Research and Theory, v. 12, n. 4, p. 467-492, 2002.). These authors clarify that a democratically oriented implementation would be one in which there is direct or indirect participation of the population, and more direct forms of participation are preferred. This greater social participation in the public sphere is directly aligned with democratic values, highlighting the relevance of each citizen to the proper functioning of the State (DENHARDT, 2012DENHARDT, R. B. Teorias da administração pública. São Paulo: Cengage Learning, 2012.; VANLEENE, VERSCHUERE and VOETS, 2016).

DeLeon and DeLeon (2002)DELEON, P.; DELEON, L. What ever happened to policy implementation? An alternative approach. Journal of Public Administration Research and Theory, v. 12, n. 4, p. 467-492, 2002. consider the issue of social participation in implementation as a “bet”, remembering that the last decades showed that there is no “best” solution for all cases and situations. Likewise, Paula (2012PAULA, A. P. P. Por uma nova gestão pública: limites e potencialidades da experiência contemporânea. Rio de Janeiro: Ed. FGV, 2012., p.180) affirms that democratic participation breaks with consolidated administrative models, “because they are not capable of reflecting the complexity of political and social processes”. Jakobsen and Andersen (2013JAKOBSEN, M.; ANDERSEN, S. C. Coproduction and equity in public service delivery. Public Administration Review, v. 73, n. 5, p. 704-713, 2013.) point out that in today’s society, where problems have become increasingly complex and solutions less obvious, co-producing public services could be the way forward.

O’Toole Junior (2000)O’TOOLE JUNIOR, L. J. Research on policy implementation: assessment and prospects. Journal of Public Administration Research and Theory, v. 10, n. 2, p. 263-288, 2000., Brandsen and Honingh (2016BRANDSEN, T.; HONINGH, M. Distinguishing different types of coproduction: a conceptual analysis based on the classical definitions. Public Administration Review, v. 76, n. 3, p. 427-435, 2016.), Vanleene, Verschuere and Voets (2016)VANLEENE, D.; VERSCHUERE, B.; VOETS, J. The democratic quality of coproduction: a theoretical review and initial research design. 2016. Disponível em:<Disponível em:https://biblio.ugent.be/publication/6909645/file/6909646.pdf >. Acesso em: 16 mar. 2017.
https://biblio.ugent.be/publication/6909...
and Rantamaki (2017RANTAMAKI, N. J. Co-production in the context of Finnish social services and health care: a challenge and a possibility for a new king of democracy. Voluntas: International Journal of Voluntary and Nonprofit Organizations, v. 28, n. 1, p. 248-264, 2017.) argue that co-production is a promising approach to operationalize greater social participation in the implementation stage. Co-production is a cooperative relationship between professionals and users of public services in the production of public policy (PARKS, BAKER, KISER et al., 1981PARKS, R. B. et al. Consumers as coproducers of public services: some economic and institutional considerations. Policy Studies Journal, v. 9, n. 7, p. 1001-1011, 1981.; DELEON and DELEON, 2002DELEON, P.; DELEON, L. What ever happened to policy implementation? An alternative approach. Journal of Public Administration Research and Theory, v. 12, n. 4, p. 467-492, 2002.; OSTROM, 1996OSTROM, E. Crossing the great divide: coproduction, synergy, and development. World Development, v. 24, n. 6, p. 1073-1087, 1996.; BOVAIRD, VAN RYZIN, LOEFFLER et al., 2015BOVAIRD, T. et al. Activating citizens to participate in collective co-production of public services. Journal of Social Policy, v. 44, n. 1, p. 1-23, 2015.; RANTAMAKI, 2017RANTAMAKI, N. J. Co-production in the context of Finnish social services and health care: a challenge and a possibility for a new king of democracy. Voluntas: International Journal of Voluntary and Nonprofit Organizations, v. 28, n. 1, p. 248-264, 2017.), in which active participation of users is crucial for public policy objectives to be effectively achieved (PARKS, BAKER, KISER et al., 1981PARKS, R. B. et al. Consumers as coproducers of public services: some economic and institutional considerations. Policy Studies Journal, v. 9, n. 7, p. 1001-1011, 1981.; ALFORD, 1998ALFORD, J. A public management road less travelled: clients as co-producers of public services. Australian Journal of Public Administration, v. 57, n. 4, p. 128-137, 1998.; RYAN, 2012RYAN, B. Co-production: option or obligation? Australian Journal of Public Administration, v. 71, n. 3, p. 314-324, 2012.; BOVAIRD, VAN RYZIN, LOEFFLER et al., 2015BOVAIRD, T. et al. Activating citizens to participate in collective co-production of public services. Journal of Social Policy, v. 44, n. 1, p. 1-23, 2015.). According to Alford (1998ALFORD, J. A public management road less travelled: clients as co-producers of public services. Australian Journal of Public Administration, v. 57, n. 4, p. 128-137, 1998., p. 129), “co-production is not just a nice thing to have like volunteer assistance. Many public activities are actually impossible to do without it”.

Alford (1998ALFORD, J. A public management road less travelled: clients as co-producers of public services. Australian Journal of Public Administration, v. 57, n. 4, p. 128-137, 1998.) points out that social participation in co-production requires a more complex notion of citizenship, not based only on voting, exercising rights or taking part in public deliberations, but also on assisting in the production of services. In co-production, users play an active role in delivering services (WHITAKER, 1980WHITAKER, G. P. Coproduction: citizen participation in service delivery. Public Administration Review, v. 40, p. 240-246, 1980.; OSTO, 1996; O’TOOLE JUNIOR, 2000O’TOOLE JUNIOR, L. J. Research on policy implementation: assessment and prospects. Journal of Public Administration Research and Theory, v. 10, n. 2, p. 263-288, 2000.; BOVAIRD, 2007BOVAIRD, T. Beyond engagement and participation: user and community coproduction of public services. Public Administration Review, v. 67, n. 5, p. 846-860, 2007.; RYAN, 2012RYAN, B. Co-production: option or obligation? Australian Journal of Public Administration, v. 71, n. 3, p. 314-324, 2012.), they are not only ‘receiving’ public policies (ALFORD, 1998ALFORD, J. A public management road less travelled: clients as co-producers of public services. Australian Journal of Public Administration, v. 57, n. 4, p. 128-137, 1998.). Bovaird (2007BOVAIRD, T. Beyond engagement and participation: user and community coproduction of public services. Public Administration Review, v. 67, n. 5, p. 846-860, 2007.) and Rantamaki (2017RANTAMAKI, N. J. Co-production in the context of Finnish social services and health care: a challenge and a possibility for a new king of democracy. Voluntas: International Journal of Voluntary and Nonprofit Organizations, v. 28, n. 1, p. 248-264, 2017.) emphasize that a more active and participative role of players in co-production demands building a new understanding about the relationship between public agents and service users.

The literature presents some studies in Brazil worthwhile mentioning to illustrate the co-production modus operandi. One example is the study by Ostrom (1996OSTROM, E. Crossing the great divide: coproduction, synergy, and development. World Development, v. 24, n. 6, p. 1073-1087, 1996.), who observed the Brazilian water distribution system, in which the government provided water transmission mains to residential areas and users were responsible for connecting and maintaining them. Klein Júnior, Salm, Heidemann et al. (2012KLEIN JÚNIOR, V. H. et al. Participação e coprodução em política habitacional: estudo de um programa de construção de moradias em SC. Revista de Administração Pública, v. 46, n. 1, p. 25-47, 2012.) studied the policy on housing construction for low-income populations, in which the federal government subsidized part of the construction, the city council performed the technical supervision and the residents indicated their demands and controlled the execution of the works. Another example is the work by Chaebo and Medeiros (2016CHAEBO, G.; MEDEIROS, J. J. Conditions for policy implementation via co-production: the control of dengue fever in Brazil. Public Management Review, {s.n}, p. 1-18, 2016.), who studied the policy to combat dengue, in which health agents inspected homes and provided guidelines to residents who monitored and combated outbreaks.

The idea of ​​co-producing public services is not new (FOTAKI, 2015FOTAKI, M. Co-production under the financial crisis and austerity: a means of democratizing public services or a race to the bottom? Journal of Management Inquiry, v. 24, n. 4, p. 433-438, 2015.). Since the 1970s, academia has pointed to the active participation of citizens in the production of public services (CEPIKU and GIORDANO, 2014CEPIKU, D.; GIORDANO, F. Co-production in developing countries: insights from the community health workers experience. Public Management Review, v. 16, n. 3, p. 317-340, 2014.; BOVAIRD, VAN RYZIN, LOEFFLER et al., 2015BOVAIRD, T. et al. Activating citizens to participate in collective co-production of public services. Journal of Social Policy, v. 44, n. 1, p. 1-23, 2015.), and the interest in the subject has increased in the last decades (BRANDSEN and HONINGH, 2016BRANDSEN, T.; HONINGH, M. Distinguishing different types of coproduction: a conceptual analysis based on the classical definitions. Public Administration Review, v. 76, n. 3, p. 427-435, 2016.; VANLEENE, VERSCHUERE and VOETS, 2016VANLEENE, D.; VERSCHUERE, B.; VOETS, J. The democratic quality of coproduction: a theoretical review and initial research design. 2016. Disponível em:<Disponível em:https://biblio.ugent.be/publication/6909645/file/6909646.pdf >. Acesso em: 16 mar. 2017.
https://biblio.ugent.be/publication/6909...
). Co-production has been considered as one of the tendencies of the 21st century in public administration (RYAN, 2012RYAN, B. Co-production: option or obligation? Australian Journal of Public Administration, v. 71, n. 3, p. 314-324, 2012.), and Verschuere, Brandsen and Pestoff (2012VERSCHUERE, B.; BRANDSEN, T.; PESTOFF, V. Co-production: the state of art in research and the future agenda. International Journal of Voluntary and Nonprofit Organizations, v. 23, n. 4, p. 1083-1101, 2012.) and Jakobsen and Andersen (2013JAKOBSEN, M.; ANDERSEN, S. C. Coproduction and equity in public service delivery. Public Administration Review, v. 73, n. 5, p. 704-713, 2013.) argue that co-production has become the main subject between public managers and public policy researchers in the last decade because of the understanding that most public services rely on an active role of the individuals who consume them.

Although public managers and researchers point out the potential of co-production to become a widespread practice in public administration, the disconnection around the concept in literature hinders the materialization of this potential. It has not been part of the authors’ concerns to achieve a more integrated (or even cumulative) approach in co-production (VERSCHUERE, BRANDSEN and PESTOFF, 2012VERSCHUERE, B.; BRANDSEN, T.; PESTOFF, V. Co-production: the state of art in research and the future agenda. International Journal of Voluntary and Nonprofit Organizations, v. 23, n. 4, p. 1083-1101, 2012.). In fact, the term “co-production” itself remains a disputed territory within the literature (VERSCHUERE, BRANDSEN and PESTOFF, 2012VANLEENE, D.; VERSCHUERE, B.; VOETS, J. The democratic quality of coproduction: a theoretical review and initial research design. 2016. Disponível em:<Disponível em:https://biblio.ugent.be/publication/6909645/file/6909646.pdf >. Acesso em: 16 mar. 2017.
https://biblio.ugent.be/publication/6909...
; VAN EIJK and STEEN, 2014VAN EIJK, C. J. A.; STEEN, T. P. S. Why people co-produce: analyzing citizens’ perceptions on co-planning engagement in health care services. Public Management Review, v. 16, n. 3, p. 358-382, 2014.; BRANDSEN and HONINGH, 2016BRANDSEN, T.; HONINGH, M. Distinguishing different types of coproduction: a conceptual analysis based on the classical definitions. Public Administration Review, v. 76, n. 3, p. 427-435, 2016.). However, the discussion on this issue does not contribute to the purposes of this study, thus it will not be presented here.

Verschuere, Brandsen and Pestoff (2012VERSCHUERE, B.; BRANDSEN, T.; PESTOFF, V. Co-production: the state of art in research and the future agenda. International Journal of Voluntary and Nonprofit Organizations, v. 23, n. 4, p. 1083-1101, 2012.) emphasize the importance of theoretically understanding co-production in its various aspects. In the same direction, Brandsen and Honingh (2016) argue that the cumulative effect of past research on co-production was unsuccessful because it was not possible to relate the findings systematically. For Cepiku and Giordano (2014CEPIKU, D.; GIORDANO, F. Co-production in developing countries: insights from the community health workers experience. Public Management Review, v. 16, n. 3, p. 317-340, 2014.), the current state of development of co-production theory should already be able to analyze beyond the isolated effects of its variables. These notes reinforce the words of Alford (2014ALFORD, J. The multiple facets of co-production: building on the work of Elinor Ostrom. Public Administration Review, v. 16, n. 3, p. 299-316, 2014.) and Brandsen and Honingh (2016)BRANDSEN, T.; HONINGH, M. Distinguishing different types of coproduction: a conceptual analysis based on the classical definitions. Public Administration Review, v. 76, n. 3, p. 427-435, 2016. that there are still many theoretical and empirical development opportunities in co-production.

Thus, this article aims to systematize and establish connections among the concepts that make up the theoretical framework for the co-production approach presented in the literature, as a contribution to the construction of a more robust and integrated theory. To achieve this goal, a literature review on co-production was undertaken, supported mainly by scientific articles published in core journals (such as Public Management Review and Public Administration Review). The articles were identified by their title, verifying if they had the keyword “co-production”. As suggested by Saetren (2014SAETREN, H. Implementing the third generation research paradigm in policy implementation research: an empirical assessment. Public Policy and Administration, v. 29, n. 2, p. 84-105, 2014.), the analysis included the “false negative” articles that, although not specifically dealing with co-production, helped to discuss the concepts. The “false positive” articles (which although were about co-production, did not contribute to discuss the concept) were excluded.

The study identified 53 scientific articles on co-production published in core journals, and only 24 were included in the analysis undertaken in this study. It was decided not to exclude the other 29 identified articles, but not to include them, because some of the articles had little explicit contributions, helping little in the discussion. Others had similar contributions to other works, so there was no reasonable justification to add them in the analysis. This decision was made in the case of articles by the same author, in which contributions became more incremental from the first published article. Because co-production is an expanding subject (and consequently there are many works), some degree of parsimony was necessary in the effort to include the articles in the analysis.

In addition, it was observed that the concepts were discussed, complemented and/or even challenged by several authors, which could cause some kind of conceptual confusion if they were not well developed. Thus, there was a concern to prioritize the inclusion of articles aligned with more central discussions on co-production, also as a strategy to facilitate the connection among concepts. This concern led to a methodological decision of reviewing the classical studies on co-production, since these were the works that have provided the foundations to the co-production theory over the last 40 years. This choice is supported by Cepiku and Giordano (2014CEPIKU, D.; GIORDANO, F. Co-production in developing countries: insights from the community health workers experience. Public Management Review, v. 16, n. 3, p. 317-340, 2014., p. 319), who also found that in the co-production literature “some authors are impossible to avoid”. For the reasons mentioned, the national literature on co-production, which begin only in the last decade, was not prioritized in this discussion.

The next section will discuss the concepts of co-production of public services.

CO-PRODUCTION OF PUBLIC SERVICES

In recent years there has been a radical reinterpretation of how public policies are formed and delivered to users. The elaboration is no longer a purely top-down process, but a product of negotiation among participants in political systems. Similarly, public services are no longer delivered solely by professionals and staff of government agencies, but co-produced by users and communities (BOVAIRD, 2007BOVAIRD, T. Beyond engagement and participation: user and community coproduction of public services. Public Administration Review, v. 67, n. 5, p. 846-860, 2007.; VERSCHUERE, BRANDSEN and PESTOFF, 2012VERSCHUERE, B.; BRANDSEN, T.; PESTOFF, V. Co-production: the state of art in research and the future agenda. International Journal of Voluntary and Nonprofit Organizations, v. 23, n. 4, p. 1083-1101, 2012.; BRANDSEN and HONINGH, 2016BRANDSEN, T.; HONINGH, M. Distinguishing different types of coproduction: a conceptual analysis based on the classical definitions. Public Administration Review, v. 76, n. 3, p. 427-435, 2016.). This section presents an initial effort to identify and discuss the concepts, preparing for the following integration among them in order to outline a more comprehensive theoretical framework for the co-production approach.

Participants of coproduction

It is interesting, before discussing co-production itself, to expose a difficulty found in elaborating this study, which is how to name the ‘co-producers’, the parts involved in the implementation process. Lester and Goggin (1998LESTER, J. P.; GOGGIN, M. L. Back to the future: the rediscovery of implementation studies. Policy Currents, v. 8, n. 3, p. 1-11, 1998.) argue that it is important to identify who the implementers are, because this identification refers to the competencies expected for the roles to be fulfilled (who should do what?). Parks, Baker, Kiser et al. (1981PARKS, R. B. et al. Consumers as coproducers of public services: some economic and institutional considerations. Policy Studies Journal, v. 9, n. 7, p. 1001-1011, 1981.) define the individuals or groups of society, those who produce for the purpose of exchange, as “regular producers” (i.e. professionals) of the services they provide. At the same time, those individuals play the role of “regular consumers” (users) of public goods and services they consume.

According to Ostrom (1996OSTROM, E. Crossing the great divide: coproduction, synergy, and development. World Development, v. 24, n. 6, p. 1073-1087, 1996.), Verschuere, Brandsen and Pestoff (2012VERSCHUERE, B.; BRANDSEN, T.; PESTOFF, V. Co-production: the state of art in research and the future agenda. International Journal of Voluntary and Nonprofit Organizations, v. 23, n. 4, p. 1083-1101, 2012.) and Alford (2014ALFORD, J. The multiple facets of co-production: building on the work of Elinor Ostrom. Public Administration Review, v. 16, n. 3, p. 299-316, 2014.), public goods and services are potentially produced by professionals and those often treated as users. According to Ostrom (1996)OSTROM, E. Crossing the great divide: coproduction, synergy, and development. World Development, v. 24, n. 6, p. 1073-1087, 1996., the position of ‘professionals’ is usually occupied by government agencies. On the other hand, the literature review shows that there are several terms to designate who occupies the position of ‘users’. Depending on the author, “users” can be called “customers”, “regular consumers”, “co-participants” or “citizens”. To add to this confusion of terms, there are the ‘volunteers’ who act in the delivery of public services. Alford (2014)ALFORD, J. A public management road less travelled: clients as co-producers of public services. Australian Journal of Public Administration, v. 57, n. 4, p. 128-137, 1998. states that, as important as it is to conceptualize, it is equally important to understand how these participants fit into the delivery of services.

Alford (2002ALFORD, J. Why do public-sector clients coproduce? Toward a contingency theory. Administration & Society, v. 34, n. 1, p. 32-56, 2002.) makes some observations that help to better distinguish some of these concepts, especially among clients, citizens and volunteers. For the author, clients are those who receive private value for services provided by government agencies (it is important to note that, for most authors, this is also the definition for “users”). Volunteers do not receive any benefit from the services and they provide inputs voluntarily through organizations. Hence, it is possible to say that both volunteers and clients are co-participants (because they provide inputs), while the former acts disinterestedly, the latter seek their self-interest.

In the literature, we also observe the indistinct use of the terms “customers”, “consumers” and “users” to treat the self-interested individuals who receive services, without great attention to possible differences in their meanings. One example is the use of the terms “customers” and “consumers”, often used in the private sector referring to individuals who consume companies’ products and services, which may cause confusion or discomfort in the public sector. There are also problems with the term “user”, which in practice does not convey the idea of the action of co-participation. Actually, the term ‘users’ may give an idea of passive individuals.

Another category of individuals often confused with “users” is that of “citizens” who, unlike users, act in the name of the community and try to express their aspirations through voting or other mechanisms of political participation that make up the democratic process (ALFORD, 2002ALFORD, J. Why do public-sector clients coproduce? Toward a contingency theory. Administration & Society, v. 34, n. 1, p. 32-56, 2002.). According to Alford, ‘citizen’ seems to be more related to the influence of the individual in the process of policy making than in its actual implementation. Denhardt (2012DENHARDT, R. B. Teorias da administração pública. São Paulo: Cengage Learning, 2012.) reinforces the words of Alford (2002)ALFORD, J. A public management road less travelled: clients as co-producers of public services. Australian Journal of Public Administration, v. 57, n. 4, p. 128-137, 1998., pointing out that users focus primarily on their own wills while citizens focus on the common good and the long-term consequences for the community.

It is clear that, among the categories of participants exposed, that of users occupies a prominent place in co-production. It is also clear the difficulty of naming these individuals. Perhaps more precise terms for them would be “individual co-producers”, “social co-producers”, “non-regular co-producers”, or “user co-producers”. For now, it is not possible to reach any definitive conclusion on this matter, so this study will continue to use the term “users” to describe this category of individuals. However, the semantic difficulties with the current designations are evident and the need to define more adequate terms is pointed out here. Denominations are necessary to clarify who exactly is being analyzed, as well as to clarify what motivates each of these participants to action.

Coproduction motivation

According to Van Eijk and Steen (2014)VAN EIJK, C. J. A.; STEEN, T. P. S. Why people co-produce: analyzing citizens’ perceptions on co-planning engagement in health care services. Public Management Review, v. 16, n. 3, p. 358-382, 2014. and Fledderus, Brandsen and Honingh (2015FLEDDERUS, J.; BRANDSEN, T.; HONINGH, M. E. User co-production of public service delivery: an uncertainty approach. Public Policy and Administration, v. 30, n. 2, p. 145-164, 2015.), participants should not only be able to contribute with resources, but also need to be motivated to engage. Therefore, the issue of motivation plays a central role in the dynamics of co-production, because it allows understanding circumstances in which participants are willing to put effort (VERSCHUERE, BRANDSEN and HONINGH, 2012VERSCHUERE, B.; BRANDSEN, T.; PESTOFF, V. Co-production: the state of art in research and the future agenda. International Journal of Voluntary and Nonprofit Organizations, v. 23, n. 4, p. 1083-1101, 2012.; RANTAMAKI, 2017RANTAMAKI, N. J. Co-production in the context of Finnish social services and health care: a challenge and a possibility for a new king of democracy. Voluntas: International Journal of Voluntary and Nonprofit Organizations, v. 28, n. 1, p. 248-264, 2017.). Kiser and Percy (1980KISER, L. L.; PERCY, S. L. The concept of coproduction and its implications for public service delivery. In: WORKSHOP IN POLITICAL THEORY AND POLICY ANALYSIS; 1980, Bloomington. Proceedings... Bloomington, IN: {s.n}, 1980. p. 1-24,) and Alford (2014ALFORD, J. The multiple facets of co-production: building on the work of Elinor Ostrom. Public Administration Review, v. 16, n. 3, p. 299-316, 2014.) argue that professionals are usually motivated by cash payments, and that payment can also take the form of political support or other exchanges.

Alford (2002ALFORD, J. Why do public-sector clients coproduce? Toward a contingency theory. Administration & Society, v. 34, n. 1, p. 32-56, 2002.) notes that among the categories ‘users’, ‘citizens’ and ‘volunteers’, the one that generated least studies on motivation was “users”. Noting that little is known about what motivates these individuals, the author proposed 5 groups of motivating factors for users: (a) sanctions; (b) material rewards; (c) intrinsic rewards; (d) solidarity incentives; and (e) expressive values. However, Alford believes that material sanctions and rewards are ineffective in provoking the user’s contribution, and argues that users respond better to non-material incentives. Later, revisiting the subject, Alford (2014)ALFORD, J. A public management road less travelled: clients as co-producers of public services. Australian Journal of Public Administration, v. 57, n. 4, p. 128-137, 1998. suggests that, to a greater or lesser extent, users are influenced by all 5 groups of motivating factors. For Kiser and Percy (1980KISER, L. L.; PERCY, S. L. The concept of coproduction and its implications for public service delivery. In: WORKSHOP IN POLITICAL THEORY AND POLICY ANALYSIS; 1980, Bloomington. Proceedings... Bloomington, IN: {s.n}, 1980. p. 1-24,), Parks, Baker, Kiser et al. (1981PARKS, R. B. et al. Consumers as coproducers of public services: some economic and institutional considerations. Policy Studies Journal, v. 9, n. 7, p. 1001-1011, 1981.), Alford (2014)ALFORD, J. A public management road less travelled: clients as co-producers of public services. Australian Journal of Public Administration, v. 57, n. 4, p. 128-137, 1998., and Rantamaki (2017RANTAMAKI, N. J. Co-production in the context of Finnish social services and health care: a challenge and a possibility for a new king of democracy. Voluntas: International Journal of Voluntary and Nonprofit Organizations, v. 28, n. 1, p. 248-264, 2017.), users are primarily motivated by the increased quantity and quality of the services they consume.

Verschuere, Brandsen and Pestoff (2012VERSCHUERE, B.; BRANDSEN, T.; PESTOFF, V. Co-production: the state of art in research and the future agenda. International Journal of Voluntary and Nonprofit Organizations, v. 23, n. 4, p. 1083-1101, 2012.) and Jakobsen (2013JAKOBSEN, M.; ANDERSEN, S. C. Coproduction and equity in public service delivery. Public Administration Review, v. 73, n. 5, p. 704-713, 2013.) present an alternative view to explain what leads users to engage in co-production activities. For them, users tend to co-produce at least as long as they are highly dependent on the public service that co-production activity provides for them. If the person feels that the service provided is very important to themselves or their loved ones, they will be more motivated to co-produce (VERSCHUERE, BRANDSEN and PESTOFF, 2012VERSCHUERE, B.; BRANDSEN, T.; PESTOFF, V. Co-production: the state of art in research and the future agenda. International Journal of Voluntary and Nonprofit Organizations, v. 23, n. 4, p. 1083-1101, 2012.). Thus, the perception of the relevance of the good or public service to the well-being of the user and their family, would be fundamental to motivate their participation in co-production activities. According to Jakobsen (2013), future research should deepen the understanding of the psychological mechanisms that explain co-production by users.

Coproduction level

Although mapping the motivations of the users involved in co-production activities was not the goal of Brudney and England (1983BRUDNEY, J. L.; ENGLAND, R. E. Toward a definition of coproduction concept. Public Administration Review, v. 43, n. 1, p. 59-65, 1983.), the authors were able to identify 3 levels of co-production according to the scope of benefits: (a) individual co-production; (b) group co-production; and (c) collective co-production. In individual co-production, the user participates in the production of the good or service that they enjoy; In group co-production, the group of individuals improves the quality of the services provided to that group; And in collective co-production, the benefits are enjoyed by the entire community, from the continuous cooperation between professionals and users. For Alford (2014ALFORD, J. The multiple facets of co-production: building on the work of Elinor Ostrom. Public Administration Review, v. 16, n. 3, p. 299-316, 2014.) and Bovaird, Van Ryzin, Loeffler et al. (2015BOVAIRD, T. et al. Activating citizens to participate in collective co-production of public services. Journal of Social Policy, v. 44, n. 1, p. 1-23, 2015.) the level of co-production develops on a continuum. At one extreme would be the individual co-production, while at the other would be the collective co-production, with the existence of intermediate degrees that would bring value to groups. Bovaird, Van Ryzin, Loeffler et al. (2015)BOVAIRD, T. Beyond engagement and participation: user and community coproduction of public services. Public Administration Review, v. 67, n. 5, p. 846-860, 2007. call these intermediate degrees of co-production as “hybrids”. Pestoff (2014PESTOFF, V. Collective action and the sustainability of co-production. Public Management Review, v. 16, n. 3, p. 383-401, 2014.) suggests that future research should seek to understand how individual and collective co-production levels blend and relate.

Forms of co-production

For Whitaker (1980WHITAKER, G. P. Coproduction: citizen participation in service delivery. Public Administration Review, v. 40, p. 240-246, 1980.), three forms of co-production can be identified from relationships between professionals and users. In the first, the users ask professionals for help and the understanding of these demands leads to the reformulation of the administrative procedures used by the professionals (for example, individuals seeking medical attention at health care centers). In the second, users provide assistance to professionals - although professionals are not the direct beneficiaries - and both influence the implementation of policies (for example, parents assisting teachers in their own children’s education). In the third, users and professionals interact, adjusting to mutual expectations and establishing a common understanding of the problem and the attributions of each participant. Mutual adjustments are designed to modify the behavior of both professionals and users (for example, teachers and students entering into a common understanding of expectations about what they consider important, perhaps to improve the interest of the class).

Regarding the first form of co-production mentioned, it is important to observe that there is no activity developed by the public sector that would be consider co-production. In fact, what Whitaker (1980WHITAKER, G. P. Coproduction: citizen participation in service delivery. Public Administration Review, v. 40, p. 240-246, 1980.) argues is that if the activities required by users are not provided for in the formal bureaucratic procedures established by the administration, and if the appearance of such a claim results in a change in the established procedures, then it may be considered a form of co-production.

The forms of co-production described above are not necessarily limited to a single professional (in this case, a public agent) or one or more users. Normally, according to Bovaird (2007BOVAIRD, T. Beyond engagement and participation: user and community coproduction of public services. Public Administration Review, v. 67, n. 5, p. 846-860, 2007.) and Verschuere, Brandsen and Pestoff (2012VERSCHUERE, B.; BRANDSEN, T.; PESTOFF, V. Co-production: the state of art in research and the future agenda. International Journal of Voluntary and Nonprofit Organizations, v. 23, n. 4, p. 1083-1101, 2012.) and Bovaird, Van Ryzin, Loeffler et al. (2015)BOVAIRD, T. Beyond engagement and participation: user and community coproduction of public services. Public Administration Review, v. 67, n. 5, p. 846-860, 2007., they involve multiple relationships between users and professionals. For the authors, co-production activities may include a large number of users, or a user may be engaged in co-production activities with various professionals (for instance, a patient who has suffered a car accident may co-produce with physicians, physiotherapists, psychologists, etc.). Thus, Ostrom (1996OSTROM, E. Crossing the great divide: coproduction, synergy, and development. World Development, v. 24, n. 6, p. 1073-1087, 1996.) and Alford (2014ALFORD, J. The multiple facets of co-production: building on the work of Elinor Ostrom. Public Administration Review, v. 16, n. 3, p. 299-316, 2014.) define the spaces of professional/user interaction as “polycentric”, i.e. not centered on a single agent or individual.

The role of government in co-production

As pointed out at the beginning of this section, government is no longer seen as the only provider of public goods and services (BOVAIRD, 2007BOVAIRD, T. Beyond engagement and participation: user and community coproduction of public services. Public Administration Review, v. 67, n. 5, p. 846-860, 2007.; VERSCHUERE, BRANDSEN and PESTOFF, 2012VERSCHUERE, B.; BRANDSEN, T.; PESTOFF, V. Co-production: the state of art in research and the future agenda. International Journal of Voluntary and Nonprofit Organizations, v. 23, n. 4, p. 1083-1101, 2012.; BRANDSEN and HONINGH, 2016BOVAIRD, T. Beyond engagement and participation: user and community coproduction of public services. Public Administration Review, v. 67, n. 5, p. 846-860, 2007.). Co-production requires a broader understanding of the role of government: producer, regulator and/or provider of subsidies, as well as a catalyst for the efforts of individuals and groups (ALFORD, 1998ALFORD, J. A public management road less travelled: clients as co-producers of public services. Australian Journal of Public Administration, v. 57, n. 4, p. 128-137, 1998.; CEPIKU and GIORDANO, 2014CEPIKU, D.; GIORDANO, F. Co-production in developing countries: insights from the community health workers experience. Public Management Review, v. 16, n. 3, p. 317-340, 2014.). For Alford (1998ALFORD, J. A public management road less travelled: clients as co-producers of public services. Australian Journal of Public Administration, v. 57, n. 4, p. 128-137, 1998.) and Ryan (2012RYAN, B. Co-production: option or obligation? Australian Journal of Public Administration, v. 71, n. 3, p. 314-324, 2012.), once users recognize the need for co-production, the role of government agents necessarily turns more to the induction of users for co-production than to the implementation of the policy itself.

The government induces the participation of users in the following ways: (a) as a leader, presenting ideas and promoting activities; (b) as an arbiter, regulating the relationship of reciprocity between professionals and users; and (c) as a funder, providing financial resources for projects (LAM, 1996LAM, W. F. Institutional design of public agencies and coproduction: a study of irrigation associations in Taiwan. World Development, v. 24, n. 6, p. 1039-1054, 1996.). According to Alford (1998ALFORD, J. A public management road less travelled: clients as co-producers of public services. Australian Journal of Public Administration, v. 57, n. 4, p. 128-137, 1998.), to encourage users to contribute their time and effort, managers and their staff should not exercise direct authority, but rather indirect influence. Incidentally, this reveals a less coercive and more cooperative face on the part of the government towards users when it comes to co-production.

Institutional arrangements of co-production

Although government acts less coercively when in co-production, it is still necessary to establish systems of rules (or institutional arrangements) to define the incentives and the individual attributions of participants, as well as to regulate the abuse of authority in the relationships (LAM, 1996LAM, W. F. Institutional design of public agencies and coproduction: a study of irrigation associations in Taiwan. World Development, v. 24, n. 6, p. 1039-1054, 1996.; BIFULCO and LADD, 2006BIFULCO, R.; LADD, H. F. Institutional change and coproduction of public services: the effect of charter schools on parental involvement. Journal of Public Administration Research and Theory, v. 16, n. 4, p. 553-576, 2006.). According to Catalá (2015CATALÁ, J. P. Governabilidade democrática na América Latina. In: BRESSER PEREIRA, L. C.; SPINK, P. K. (Org.). Reforma do Estado e administração pública gerencial. Rio de Janeiro: Ed. FGV, 2015. p. 271-314.), it is necessary to ensure that the actors behave according to certain rules, in order to be able to respond to the challenges imposed by the social reality. The rules need to define the competencies of the professionals, the available resources and the possible forms of interaction with the users (VERSCHUERE, BRANDSEN and PESTOFF, 2012VERSCHUERE, B.; BRANDSEN, T.; PESTOFF, V. Co-production: the state of art in research and the future agenda. International Journal of Voluntary and Nonprofit Organizations, v. 23, n. 4, p. 1083-1101, 2012.). In addition, the rules simplify processes, establish routines and decrease uncertainty in relationships (FLEDDERUS, BRANDSEN and HONINGH, 2015FLEDDERUS, J.; BRANDSEN, T.; HONINGH, M. E. User co-production of public service delivery: an uncertainty approach. Public Policy and Administration, v. 30, n. 2, p. 145-164, 2015.).

However, Parks, Baker, Kiser et al. (1981PARKS, R. B. et al. Consumers as coproducers of public services: some economic and institutional considerations. Policy Studies Journal, v. 9, n. 7, p. 1001-1011, 1981.) argue that institutional arrangements can negatively or positively (item 4) affect the probability of co-production when: (1) they bar or limit the use of particular inputs; (2) fail to provide sufficient incentives for the employment of particular inputs; (3) mandate the employment of particular inputs; (4) call forth co-productive behavior where co-production is economically undesirable. According to Lam (1996LAM, W. F. Institutional design of public agencies and coproduction: a study of irrigation associations in Taiwan. World Development, v. 24, n. 6, p. 1039-1054, 1996.), some rules are formal, in the sense that they are created and executed by the governmental authority; other rules are considered informal, because they are maintained by individuals’ particular actions. Ostrom (1996OSTROM, E. Crossing the great divide: coproduction, synergy, and development. World Development, v. 24, n. 6, p. 1073-1087, 1996.) warns that designing institutional arrangements in ways that induce co-productive strategies is much more difficult than simply demonstrating their theoretical existence. When designing an institutional arrangement, one must consider the amount of time, money, technology, and effort that participants need to invest in the relationship (VERSCHUERE, BRAUNSEN and PESTOFF, 2012VERSCHUERE, B.; BRANDSEN, T.; PESTOFF, V. Co-production: the state of art in research and the future agenda. International Journal of Voluntary and Nonprofit Organizations, v. 23, n. 4, p. 1083-1101, 2012.). The quality of an arrangement will be demonstrated when repeated interactions occur. In other words, the quality consists in the fact that the arrangement results in users’ continuous participation (ALFORD, 2014ALFORD, J. The multiple facets of co-production: building on the work of Elinor Ostrom. Public Administration Review, v. 16, n. 3, p. 299-316, 2014.).

In addition to the institutional arrangements, there are some “institutional considerations” that influence the production efforts of professionals and users. While institutional arrangements refer to the systems of rules that establish opportunities and incentives for co-production, institutional considerations determine whether the mix of efforts is appropriate, i.e. whether “technical capacity” and “economic viability” are present in order to enable the relationship of co-production (PARKS, BAKER, KISER et al., 1981PARKS, R. B. et al. Consumers as coproducers of public services: some economic and institutional considerations. Policy Studies Journal, v. 9, n. 7, p. 1001-1011, 1981.). Although co-production literature separates these terms (institutional arrangements and considerations), it is clear that sometimes the rule systems can make a co-production relationship economically unfeasible, while at other times institutional arrangements must adapt to the technical resources available for the production of services (especially when it comes to the public sector). It is observed, however, that both institutional arrangements and institutional considerations are essential for the establishment of co-productive relationships. Future research may study institutional arrangements and considerations as technical, economic and legal factors that affect the dynamics of implementation.

Technical capacity and economic viability

Regarding institutional considerations, Parks, Baker, Kiser et al. (1981PARKS, R. B. et al. Consumers as coproducers of public services: some economic and institutional considerations. Policy Studies Journal, v. 9, n. 7, p. 1001-1011, 1981.) argue that technical capacity means that the quantity of outputs obtained would be a function of the inputs of “regular producers” (professionals) and “regular consumers” (users), while economic viability refers to the measure of efficiency in the relationship of regular producers and consumers. The case presented by Ostrom (1996OSTROM, E. Crossing the great divide: coproduction, synergy, and development. World Development, v. 24, n. 6, p. 1073-1087, 1996.) on the Brazilian water distribution system, mentioned in the introduction of this article, is an example of the presence of these two characteristics. The most feasible mix of efforts to provide the service was the one in which inputs from professionals and users (technical capacity) were present, which reduced production costs by 25% (economic viability).

When it comes to “technical capacity”, there are two types of possible relationships: (1) the inputs of professionals and users are substitutes for each other; or (2) the inputs are interdependent, in which without the presence of both (professionals and users), no output can be obtained (PARKS, BAKER, KISER et al., 1981PARKS, R. B. et al. Consumers as coproducers of public services: some economic and institutional considerations. Policy Studies Journal, v. 9, n. 7, p. 1001-1011, 1981.). Ryan (2012RYAN, B. Co-production: option or obligation? Australian Journal of Public Administration, v. 71, n. 3, p. 314-324, 2012.) defines inputs as the resources and capabilities that the parties bring to the co-production relationship. According to Ostrom (1996OSTROM, E. Crossing the great divide: coproduction, synergy, and development. World Development, v. 24, n. 6, p. 1073-1087, 1996.), when inputs are substitutable, no potential for synergy exists. However, when inputs are complementary, better outputs can be produced. According to Alford (1998ALFORD, J. A public management road less travelled: clients as co-producers of public services. Australian Journal of Public Administration, v. 57, n. 4, p. 128-137, 1998.; 2002ALFORD, J. Why do public-sector clients coproduce? Toward a contingency theory. Administration & Society, v. 34, n. 1, p. 32-56, 2002.), co-production inputs are complementary when there is a relationship of interdependence between professionals and users, where the purposes of the former cannot be achieved without users contributing their time and effort. O’Toole Junior (2010)O’TOOLE JUNIOR, L. J. Relações interorganizacionais no processo de implementação. In: PETERS, G. B.; PIERRE, J. (Org.). Administração pública: coletânea. Brasília, DF: Ed. Enap, 2010. p. 229-248. argues that if each party’s involvement is essential to successful implementation, then shared efforts will be made to produce results. As noted, the success of co-production depends on the users’ interest in cooperating (FLEDDERUS, BRANDSEN and HONINGH, 2015FLEDDERUS, J.; BRANDSEN, T.; HONINGH, M. E. User co-production of public service delivery: an uncertainty approach. Public Policy and Administration, v. 30, n. 2, p. 145-164, 2015.).

Notwithstanding, co-production cannot be guaranteed in any case, even if the outputs are optimally produced by the inputs of professionals and users. In this case, “economic viability” presents itself as a fundamental element to be considered. Some regular (professional) producers may feel that the opportunity costs are low and want to provide the services with their own resources. Other users may feel that opportunity costs exceed the salaries of regular producers and wish them to provide all services (PARKS, BAKER, KISER et al., 1981PARKS, R. B. et al. Consumers as coproducers of public services: some economic and institutional considerations. Policy Studies Journal, v. 9, n. 7, p. 1001-1011, 1981.; OSTROM, 1996OSTROM, E. Crossing the great divide: coproduction, synergy, and development. World Development, v. 24, n. 6, p. 1073-1087, 1996.). Verschuere, Brandsen and Pestoff (2012VERSCHUERE, B.; BRANDSEN, T.; PESTOFF, V. Co-production: the state of art in research and the future agenda. International Journal of Voluntary and Nonprofit Organizations, v. 23, n. 4, p. 1083-1101, 2012.) also point out the existence of transaction costs in the delivery of services. Thus, the participants assess the amount of time and effort required for implementation. If they think co-production is not the most viable alternative to implementation, both professionals and users will look for other alternatives to produce or access services.

There are critiques of this economic aspect of co-production, where the government will often see it co-production only as a possibility of reducing costs (CEPIKU and GIORDANO, 2014CEPIKU, D.; GIORDANO, F. Co-production in developing countries: insights from the community health workers experience. Public Management Review, v. 16, n. 3, p. 317-340, 2014.; PESTOFF, 2014PESTOFF, V. Collective action and the sustainability of co-production. Public Management Review, v. 16, n. 3, p. 383-401, 2014.; BOVAIRD, VAN RYZIN, LOEFFLER et al., 2015BOVAIRD, T. et al. Activating citizens to participate in collective co-production of public services. Journal of Social Policy, v. 44, n. 1, p. 1-23, 2015.). However, Lam (1996LAM, W. F. Institutional design of public agencies and coproduction: a study of irrigation associations in Taiwan. World Development, v. 24, n. 6, p. 1039-1054, 1996.) argues that the feasibility of providing services only by government agencies has been seriously challenged. Likewise, the isolated involvement of local organizations is also not the solution to all problems. Managing public affairs often involves complex tasks. Dealing with them may require capabilities and/or resources that citizens generally do not have. As discussed, the fundamental point of co-production is the participation of both professionals and users in the production of various public services, where the desired transformation is produced jointly (WHITAKER, 1980WHITAKER, G. P. Coproduction: citizen participation in service delivery. Public Administration Review, v. 40, p. 240-246, 1980.; BOVAIRD, 2007BOVAIRD, T. et al. Activating citizens to participate in collective co-production of public services. Journal of Social Policy, v. 44, n. 1, p. 1-23, 2015.). Evidence suggests that lack of technical capacity and/or economic viability may make co-production unfeasible. Thus, it is appropriate to take these issues into account in the choice of co-production for the implementation of a public policy.

Democratic aspects

This concept involves the ability of citizens to influence the course of public policy they are co-producing. In the implementation of public policy by co-production, government agents can encourage, demonstrate techniques and provide guidance, but fail to produce the desired result alone. Therefore, Whitaker (1980WHITAKER, G. P. Coproduction: citizen participation in service delivery. Public Administration Review, v. 40, p. 240-246, 1980.) indicates that instead of these agents presenting a “final product” (set of activities to be performed) to users, both parties should discuss what the desired transformation is. For King, Feltey and Susel (1998KING, C. S.; FELTEY, K. M.; SUSEL, B. O. The question of participation: toward authentic public administration in public administration. Public Administration Review, v. 58, n. 4, p. 317-326, 1998.), to arrive in the community with a preset agenda does not work well and users need to be included early in the process. These authors criticize the way in which the interaction between users and service providers usually occurs, since communication appears to be focused more on convincing the public than on establishing dialogue. For the authors, when including users in decision-making processes they are more likely to commit to the activities.

For DeLeon and DeLeon (2002)DELEON, P.; DELEON, L. What ever happened to policy implementation? An alternative approach. Journal of Public Administration Research and Theory, v. 12, n. 4, p. 467-492, 2002. and Fledderus, Brandsen and Honingh (2015FLEDDERUS, J.; BRANDSEN, T.; HONINGH, M. E. User co-production of public service delivery: an uncertainty approach. Public Policy and Administration, v. 30, n. 2, p. 145-164, 2015.), obtaining some degree of consent from the users brings legitimacy to the process. Bovaird (2007BOVAIRD, T. Beyond engagement and participation: user and community coproduction of public services. Public Administration Review, v. 67, n. 5, p. 846-860, 2007.) recommends that the relationship between professionals and users for co-production activities should imply that both parties “take risks”. In this context, taking risks means that users need to rely on suggestions and support from professionals, just as professionals need to be prepared to rely on users’ decisions and behaviors, rather than on one part to dictate decisions and behaviors to the other (BOVAIRD, 2007BOVAIRD, T. Beyond engagement and participation: user and community coproduction of public services. Public Administration Review, v. 67, n. 5, p. 846-860, 2007.).

Including users early in the process means not to consider them as mere “recipient” where policies are implemented, as discussed by Alford (1998ALFORD, J. A public management road less travelled: clients as co-producers of public services. Australian Journal of Public Administration, v. 57, n. 4, p. 128-137, 1998.). For Brandsen and Honingh (2016BRANDSEN, T.; HONINGH, M. Distinguishing different types of coproduction: a conceptual analysis based on the classical definitions. Public Administration Review, v. 76, n. 3, p. 427-435, 2016., p. 430), “it is not a question of one person having the supreme knowledge and applying it to the other”. Moreover, when government plays a major role in the activities, less co-productive efforts are created by users (FLEDDERUS, BRANDSEN and HONINGH, 2015FLEDDERUS, J.; BRANDSEN, T.; HONINGH, M. E. User co-production of public service delivery: an uncertainty approach. Public Policy and Administration, v. 30, n. 2, p. 145-164, 2015.). In order for there to be fairness between the parties, Jakobsen and Andersen (2013JAKOBSEN, M.; ANDERSEN, S. C. Coproduction and equity in public service delivery. Public Administration Review, v. 73, n. 5, p. 704-713, 2013.) propose that users must be well prepared, either in terms of knowledge or other necessary resources. These notes are necessary so that users are not seen as mere executors of policies, but also as partners capable of influencing the directions of public policy.

DeLeon and DeLeon (2002)DELEON, P.; DELEON, L. What ever happened to policy implementation? An alternative approach. Journal of Public Administration Research and Theory, v. 12, n. 4, p. 467-492, 2002. consider that a more democratic orientation in public policies would be aligned with this greater possibility of direct and indirect public involvement in the public sphere. Specifically regarding co-production, Vanleene, Verschuere and Voets (2016VANLEENE, D.; VERSCHUERE, B.; VOETS, J. The democratic quality of coproduction: a theoretical review and initial research design. 2016. Disponível em:<Disponível em:https://biblio.ugent.be/publication/6909645/file/6909646.pdf >. Acesso em: 16 mar. 2017.
https://biblio.ugent.be/publication/6909...
:5) establish that it leads to democratic quality when it “ensures an (1) inclusive relationship between the citizens and paid employees (of an organization), where the direct and active contribution of these citizens has an (2) actual influence on the outcome and they are (3) equally free of risks and have equal access to the benefits”. Fotaki (2015FOTAKI, M. Co-production under the financial crisis and austerity: a means of democratizing public services or a race to the bottom? Journal of Management Inquiry, v. 24, n. 4, p. 433-438, 2015.) and Rantamaki (2017RANTAMAKI, N. J. Co-production in the context of Finnish social services and health care: a challenge and a possibility for a new king of democracy. Voluntas: International Journal of Voluntary and Nonprofit Organizations, v. 28, n. 1, p. 248-264, 2017.) argue that the change in the relationship between professionals and users in co-production should be understood in a broader context, in terms of return of democracy to the people.

Although the term “users” is used in this subsection, Alford (2002ALFORD, J. Why do public-sector clients coproduce? Toward a contingency theory. Administration & Society, v. 34, n. 1, p. 32-56, 2002.) suggests the use of “citizens”, arguing that it is more appropriately focused on the political aspect of co-production. In fact, this discussion allows clarifying that individuals can be in more than one position within the co-production structure, acting as executors of the policy (user) and as influencers in the decision making process (citizens) (ALFORD, 2014ALFORD, J. The multiple facets of co-production: building on the work of Elinor Ostrom. Public Administration Review, v. 16, n. 3, p. 299-316, 2014.; VAN EIJK and STEEN, 2014VAN EIJK, C. J. A.; STEEN, T. P. S. Why people co-produce: analyzing citizens’ perceptions on co-planning engagement in health care services. Public Management Review, v. 16, n. 3, p. 358-382, 2014.; FLEDDERUS, BRANDSEN and HONINGH, 2015FLEDDERUS, J.; BRANDSEN, T.; HONINGH, M. E. User co-production of public service delivery: an uncertainty approach. Public Policy and Administration, v. 30, n. 2, p. 145-164, 2015.).

The Chart 1 below synthesizes the discussion of this section showing the concepts identified and the main authors.

Chart 1
Concepts of co-production in literature

It is expected that the systematization carried out in Chart 1 may be used in the future to facilitate a greater integration of the concepts that form the co-production approach, as well as allowing to analyze “one-by-one”, which seek to understand how a concept relates - or could relate - to another. It is important to emphasize that this analysis is a research agenda in co-production. Considering that the outline of a more integrated theoretical framework in co-production is a hard and time consuming task due to the number of concepts involved and the complexity of relationships, one-by-one analysis can work as the next step in this direction. As an example, by discussing only the last two concepts in Chart 1 (economic viability and democratic aspects), research could study whether the influence of citizens in the direction of a public policy increases or decreases the efficiency of the co-production relationship. Due to the space limitations, for future work the use of one-by-one analysis is suggested of the concepts.

This type of analysis has already been done by Klein Junior, Salm, Heidemann et al. (1990)KISER, L. L.; PERCY, S. L. The concept of coproduction and its implications for public service delivery. In: WORKSHOP IN POLITICAL THEORY AND POLICY ANALYSIS; 1980, Bloomington. Proceedings... Bloomington, IN: {s.n}, 1980. p. 1-24,, which in their study, related the concept of “co-production level”, by Brudney and England (1983BRUDNEY, J. L.; ENGLAND, R. E. Toward a definition of coproduction concept. Public Administration Review, v. 43, n. 1, p. 59-65, 1983.), with Whitaker’s concept of “co-production” (1980WHITAKER, G. P. Coproduction: citizen participation in service delivery. Public Administration Review, v. 40, p. 240-246, 1980.). The authors suggest that a co-production research agenda would be in line with the incorporation of more concepts in their analysis (KLEIN JUNIOR, SALM, HEIDEMANN et al., 2012KLEIN JÚNIOR, V. H. et al. Participação e coprodução em política habitacional: estudo de um programa de construção de moradias em SC. Revista de Administração Pública, v. 46, n. 1, p. 25-47, 2012.). This article contributes to understanding what these concepts would be and how they might relate. Another contribution is in terms of providing bases for consistent methodological paths for researchers studying co-production. The broad review of concepts carried out here can facilitate the elaboration of instruments for data collection and the definition of categories for data analysis in the study of public policies.

Considering the discussion in the previous sections, the conclusion below presents some reflections on the concepts of co-production.

CONCLUSION

This article aimed to systematize and discuss connections among the concepts forming the theoretical framework of the co-production approach, as a contribution to build a more robust and integrated theory. The motivation for this study was the perception that, although co-production is recognized as one of the topics with the greatest potential for public administration in the last decade (VERSCHUERE, BRANDSEN and PESTOFF, 2012VERSCHUERE, B.; BRANDSEN, T.; PESTOFF, V. Co-production: the state of art in research and the future agenda. International Journal of Voluntary and Nonprofit Organizations, v. 23, n. 4, p. 1083-1101, 2012.; JAKOBSEN and ANDERSEN, 2013JAKOBSEN, M.; ANDERSEN, S. C. Coproduction and equity in public service delivery. Public Administration Review, v. 73, n. 5, p. 704-713, 2013.; BRANDSEN and HONINGH, 2016BRANDSEN, T.; HONINGH, M. Distinguishing different types of coproduction: a conceptual analysis based on the classical definitions. Public Administration Review, v. 76, n. 3, p. 427-435, 2016.), this potential is not becoming reality because of, at least in part, the lack of connection among the concepts forming the approach.

The effort to integrate and systematize concepts in co-production was already indicated as a research agenda by Verschuere, Brandsen and Pestoff (2012VERSCHUERE, B.; BRANDSEN, T.; PESTOFF, V. Co-production: the state of art in research and the future agenda. International Journal of Voluntary and Nonprofit Organizations, v. 23, n. 4, p. 1083-1101, 2012.), Cepiku and Giordano (2014CEPIKU, D.; GIORDANO, F. Co-production in developing countries: insights from the community health workers experience. Public Management Review, v. 16, n. 3, p. 317-340, 2014.) and Brandsen and Honingh (2016)BRANDSEN, T.; HONINGH, M. Distinguishing different types of coproduction: a conceptual analysis based on the classical definitions. Public Administration Review, v. 76, n. 3, p. 427-435, 2016.. This article undertook a bibliographic review and presented a first attempt to integrate and systematize the concepts as shown in Chart 1. The following concepts were identified and discussed: co-production participants, co-production motivation, co-production level, co-production forms, role of government, institutional arrangements, technical capacity, economic viability, and democratic aspects. The study sought, by listing and describing the concepts and discussing them, to enable their joint analysis, as well as the identification of points of connection between them and opportunities of new interrelationships.

The study suggests, as the next step towards the integration of the concepts forming the theoretical framework of co-production, the use of one-by-one analysis on these concepts, in order to understand how a concept relates (or could relate) to another. In addition, it was pointed out that this analysis constitutes a research agenda in co-production. Although this suggestion may seem fragmented in relation to the scope of the general theoretical framework of co-production, it is able to guide a rich and wide agenda of co-production research, and it is characterized as necessary to understand the theoretical framework.

Some concrete research proposals have already been exposed throughout the text. One of them was to connect economic viability and democratic aspects, that is, whether the influence of citizens in the direction of public policy increases or decreases the efficiency of the co-production relationship. Another was to connect institutional arrangements, technical capacity and economic viability as legal, technical and economic factors that affect the dynamics of implementation. In any case, it is pointed out that any two (or more) concepts presented in Box 1, if connected, could foster relevant research in co-production.

In addition to indicating a possible direction towards greater theoretical integration of concepts, this article contributes to the establishment of a consistent methodological roadmap for researchers who wish to undertake co-production studies. The identification and definition of the concepts presented in the co-production dynamics, facilitate the elaboration of data collection instruments and the definition of categories of analysis for the public policies implemented through this approach. Part of this process is identifying the participants, how they participate, what motivates this participation, as well as understanding other fundamental aspects of the implementation through co-production. These aspects are: whether there is engagement and benefits expected in the co-production activities; the adjustment of users’ demands and professional procedures to create the policy, the roles of government, the institutional arrangements around the professional/user relationship, co-production activities and inputs, and the degree to which technical capacity, economic viability, and democratic quality are present in the implementation of the public policy via co-production.

All the notes made here, both theoretical and methodological, are expected to help in structuring a future conceptual model in co-production. This will require an effort on the part of researchers interested in the subject, not only to identify, refine and relate the concepts, but also to undertake empirical studies to elucidate whether the theoretical components reflect the reality of public policies.

Empirical studies from the points listed could demonstrate that public policies are being implemented through the co-production approach without being aware of it. Some policies that can be adapted to the analysis through the co-production approach are those that involve, for example: (i) waste recycling, since the separation of the recyclable waste by users would facilitate the recycling; (ii) emergency hotlines to report crimes, since the information provided by the population serves as input for the development of police actions and the fight against crime; (iii) child vaccination, because although the government provides materials and professionals, parents need to take the children to be vaccinated. In general, policies that can be analyzed as co-production are those that depend on a change in users’ behavior.

This article observed co-production in terms of public policy implementation. As mentioned before, the studies under implementation aim to increase the probability of success in the delivery of public policies (BARRETT, 2004BARRETT, S. M. Implementation studies: time for a revival? Public Administration, v. 82, n. 2, p. 249-262, 2004.). Implementing policies through co-production refers to a greater involvement of users in the delivery of services (O’TOOLE JUNIOR, 2000O’TOOLE JUNIOR, L. J. Research on policy implementation: assessment and prospects. Journal of Public Administration Research and Theory, v. 10, n. 2, p. 263-288, 2000.; BRANDSEN and HONINGH, 2016BRANDSEN, T.; HONINGH, M. Distinguishing different types of coproduction: a conceptual analysis based on the classical definitions. Public Administration Review, v. 76, n. 3, p. 427-435, 2016.) and, consequently, to the development of a new, more cooperative type of relationship between professionals and users (BOVAIRD, 2007BOVAIRD, T. Beyond engagement and participation: user and community coproduction of public services. Public Administration Review, v. 67, n. 5, p. 846-860, 2007.; RANTAMAKI, 2017RANTAMAKI, N. J. Co-production in the context of Finnish social services and health care: a challenge and a possibility for a new king of democracy. Voluntas: International Journal of Voluntary and Nonprofit Organizations, v. 28, n. 1, p. 248-264, 2017.). There are authors who argue that this greater direct and indirect involvement of users would be aligned with a more democratic orientation in public policies (see DENHARDT, 2012DENHARDT, R. B. Teorias da administração pública. São Paulo: Cengage Learning, 2012.; PAULA, 2012PAULA, A. P. P. Por uma nova gestão pública: limites e potencialidades da experiência contemporânea. Rio de Janeiro: Ed. FGV, 2012.), which can be considered a “bet” for the renewal of the field of studies in implementation (DELEON and DELEON, 2002DELEON, P.; DELEON, L. What ever happened to policy implementation? An alternative approach. Journal of Public Administration Research and Theory, v. 12, n. 4, p. 467-492, 2002.). Thus, this article contributes not only to the co-production approach, but also to the field of public policy implementation.

REFERÊNCIAS

  • ALFORD, J. A public management road less travelled: clients as co-producers of public services. Australian Journal of Public Administration, v. 57, n. 4, p. 128-137, 1998.
  • ALFORD, J. Why do public-sector clients coproduce? Toward a contingency theory. Administration & Society, v. 34, n. 1, p. 32-56, 2002.
  • ALFORD, J. The multiple facets of co-production: building on the work of Elinor Ostrom. Public Administration Review, v. 16, n. 3, p. 299-316, 2014.
  • BARRETT, S. M. Implementation studies: time for a revival? Public Administration, v. 82, n. 2, p. 249-262, 2004.
  • BERMAN, P. The study of macro and micro implementation. Public Policy, v. 26, n. 2, p. 157-184, 1978.
  • BIFULCO, R.; LADD, H. F. Institutional change and coproduction of public services: the effect of charter schools on parental involvement. Journal of Public Administration Research and Theory, v. 16, n. 4, p. 553-576, 2006.
  • BOVAIRD, T. Beyond engagement and participation: user and community coproduction of public services. Public Administration Review, v. 67, n. 5, p. 846-860, 2007.
  • BOVAIRD, T. et al. Activating citizens to participate in collective co-production of public services. Journal of Social Policy, v. 44, n. 1, p. 1-23, 2015.
  • BRANDSEN, T.; HONINGH, M. Distinguishing different types of coproduction: a conceptual analysis based on the classical definitions. Public Administration Review, v. 76, n. 3, p. 427-435, 2016.
  • BRINKERHOFF, D. W. Process perspectives on policy change: highlighting implementation. World Development, v. 24, n. 9, p. 1395-1401, 1996.
  • BRUDNEY, J. L.; ENGLAND, R. E. Toward a definition of coproduction concept. Public Administration Review, v. 43, n. 1, p. 59-65, 1983.
  • CATALÁ, J. P. Governabilidade democrática na América Latina. In: BRESSER PEREIRA, L. C.; SPINK, P. K. (Org.). Reforma do Estado e administração pública gerencial. Rio de Janeiro: Ed. FGV, 2015. p. 271-314.
  • CEPIKU, D.; GIORDANO, F. Co-production in developing countries: insights from the community health workers experience. Public Management Review, v. 16, n. 3, p. 317-340, 2014.
  • CHAEBO, G.; MEDEIROS, J. J. Conditions for policy implementation via co-production: the control of dengue fever in Brazil. Public Management Review, {s.n}, p. 1-18, 2016.
  • DELEON, P.; DELEON, L. What ever happened to policy implementation? An alternative approach. Journal of Public Administration Research and Theory, v. 12, n. 4, p. 467-492, 2002.
  • DENHARDT, R. B. Teorias da administração pública. São Paulo: Cengage Learning, 2012.
  • FLEDDERUS, J.; BRANDSEN, T.; HONINGH, M. E. User co-production of public service delivery: an uncertainty approach. Public Policy and Administration, v. 30, n. 2, p. 145-164, 2015.
  • FOTAKI, M. Co-production under the financial crisis and austerity: a means of democratizing public services or a race to the bottom? Journal of Management Inquiry, v. 24, n. 4, p. 433-438, 2015.
  • FREY, K. Políticas públicas: um debate conceitual e questões referentes à prática da análise de políticas públicas no Brasil. Planejamento e Políticas Públicas, n. 21, p. 211-260, 2000.
  • HAM, C.; HILL, M. The policy process in the modern capitalist State. 2. ed. London: Harvester Wheatsheaf, 1993.
  • HOWLETT, M.; RAMESH, M. Studying public policy: policy cycles and policy subsystems. 2. ed. Oxford: Oxford University Press, 2003.
  • JAKOBSEN, M. Can government initiatives increase citizen coproduction? Results of a randomized field experiment. Journal of Public Administration Research and Theory, v. 23, n. 1, p. 27-54, 2013.
  • JAKOBSEN, M.; ANDERSEN, S. C. Coproduction and equity in public service delivery. Public Administration Review, v. 73, n. 5, p. 704-713, 2013.
  • KING, C. S.; FELTEY, K. M.; SUSEL, B. O. The question of participation: toward authentic public administration in public administration. Public Administration Review, v. 58, n. 4, p. 317-326, 1998.
  • KISER, L. L.; PERCY, S. L. The concept of coproduction and its implications for public service delivery. In: WORKSHOP IN POLITICAL THEORY AND POLICY ANALYSIS; 1980, Bloomington. Proceedings... Bloomington, IN: {s.n}, 1980. p. 1-24,
  • KLEIN JÚNIOR, V. H. et al. Participação e coprodução em política habitacional: estudo de um programa de construção de moradias em SC. Revista de Administração Pública, v. 46, n. 1, p. 25-47, 2012.
  • LAM, W. F. Institutional design of public agencies and coproduction: a study of irrigation associations in Taiwan. World Development, v. 24, n. 6, p. 1039-1054, 1996.
  • LESTER, J. P.; GOGGIN, M. L. Back to the future: the rediscovery of implementation studies. Policy Currents, v. 8, n. 3, p. 1-11, 1998.
  • OSTROM, E. Crossing the great divide: coproduction, synergy, and development. World Development, v. 24, n. 6, p. 1073-1087, 1996.
  • O’TOOLE JUNIOR, L. J. Research on policy implementation: assessment and prospects. Journal of Public Administration Research and Theory, v. 10, n. 2, p. 263-288, 2000.
  • O’TOOLE JUNIOR, L. J. Relações interorganizacionais no processo de implementação. In: PETERS, G. B.; PIERRE, J. (Org.). Administração pública: coletânea. Brasília, DF: Ed. Enap, 2010. p. 229-248.
  • PALUMBO, D. J.; CALISTA, D. J. Implementation and the policy process: opening up the black box. New York: Greenwood, 1990.
  • PARKS, R. B. et al. Consumers as coproducers of public services: some economic and institutional considerations. Policy Studies Journal, v. 9, n. 7, p. 1001-1011, 1981.
  • PAULA, A. P. P. Por uma nova gestão pública: limites e potencialidades da experiência contemporânea. Rio de Janeiro: Ed. FGV, 2012.
  • PESTOFF, V. Collective action and the sustainability of co-production. Public Management Review, v. 16, n. 3, p. 383-401, 2014.
  • RANTAMAKI, N. J. Co-production in the context of Finnish social services and health care: a challenge and a possibility for a new king of democracy. Voluntas: International Journal of Voluntary and Nonprofit Organizations, v. 28, n. 1, p. 248-264, 2017.
  • RYAN, B. Co-production: option or obligation? Australian Journal of Public Administration, v. 71, n. 3, p. 314-324, 2012.
  • SAETREN, H. Implementing the third generation research paradigm in policy implementation research: an empirical assessment. Public Policy and Administration, v. 29, n. 2, p. 84-105, 2014.
  • VAN EIJK, C. J. A.; STEEN, T. P. S. Why people co-produce: analyzing citizens’ perceptions on co-planning engagement in health care services. Public Management Review, v. 16, n. 3, p. 358-382, 2014.
  • VANLEENE, D.; VERSCHUERE, B.; VOETS, J. The democratic quality of coproduction: a theoretical review and initial research design. 2016. Disponível em:<Disponível em:https://biblio.ugent.be/publication/6909645/file/6909646.pdf >. Acesso em: 16 mar. 2017.
    » https://biblio.ugent.be/publication/6909645/file/6909646.pdf
  • VERSCHUERE, B.; BRANDSEN, T.; PESTOFF, V. Co-production: the state of art in research and the future agenda. International Journal of Voluntary and Nonprofit Organizations, v. 23, n. 4, p. 1083-1101, 2012.
  • WHITAKER, G. P. Coproduction: citizen participation in service delivery. Public Administration Review, v. 40, p. 240-246, 1980.
  • WINTER, S. C. Perspectivas de implementação: status e reconsideração. In: PETERS, G. B.; PIERRE, J. (Org.). Administração pública: coletânea. Brasília, DF: Ed. Enap, 2010. p. 209-228.
  • 8
    Image Source: Pixabay. Available at: https://pixabay.com/pt/m%C3%A3o-m%C3%A3os-unidas-unidos-juntos-1917895/. Accessed on July 19, 2017.
  • 1
    Other authors use the terms “lost link” or “puzzle” in order to deal with the problems in the implementation state - see Berman (1978) and O’Toole Junior (2000), respectively.
  • 7
    {Translated version}

Publication Dates

  • Publication in this collection
    July 2017

History

  • Received
    01 July 2015
  • Accepted
    29 June 2017
Fundação Getulio Vargas, Escola Brasileira de Administração Pública e de Empresas Rua Jornalista Orlando Dantas, 30 - sala 107, 22231-010 Rio de Janeiro/RJ Brasil, Tel.: (21) 3083-2731 - Rio de Janeiro - RJ - Brazil
E-mail: cadernosebape@fgv.br