Acessibilidade / Reportar erro
This document is related to:

Structural model of deliberative citizenship: addition and validation of participation indicators

Abstract

This study aims to improve the structural model of deliberative citizenship (Costa & Pinto, 2021, 2023), which is based on studies related to the application of criteria (Tenório, 2012) to the theoretical field of social management. Data were analyzed using structural equation modeling (Hair, Hult, Ringle, & Sarstedt, 2017; Hair et al., 2021). A sample of 213 people from various regions of Brazil responded to an electronic questionnaire after being invited via social media, with emphasis on LinkedIn. In addition to confirming the statistical quality of the structural model of deliberative citizenship (Costa & Pinto, 2021, 2023), two indicators proposed for the “participation” construct (Costa & Pinto, 2021, 2023) were validated. The “deliberative process” construct is still in an exploratory phase, with a test proposition with an additional indicator related to the “cross-cutting spaces” criterion. The validation of additional indicators to the “participation” construct raised its quality above the exploratory zone, as its outer loadings were above 0.7. Therefore, the improvement carried out expands the theoretical and statistical understanding of the model, which can be applied to the various collegiate bodies in public management.

Keywords:
transparency; participation; social management; deliberative citizenship

Resumo

O objetivo deste estudo é aprimorar o modelo estrutural de cidadania deliberativa (Costa & Pinto, 2021, 2023), que tem por base os estudos relativos à aplicação de critérios (Tenório, 2012) ao campo teórico da gestão social. Os dados foram analisados mediante modelagem de equações estruturais (Hair, Hult, Ringle, & Sarstedt, 2017; Hair et al., 2021). Responderam ao questionário eletrônico 213 pessoas de variados territórios brasileiros, convidadas em redes sociais, com destaque para o LinkedIn. Além de ratificar a qualidade estatística do modelo estrutural de cidadania deliberativa (Costa & Pinto, 2021, 2023), validaram-se dois indicadores propostos ao construto “participação” (Costa & Pinto, 2021, 2023). O construto “processo deliberativo” segue em fase exploratória, tendo uma proposição de teste com um indicador adicional relativo ao critério “espaços de transversalidade”. A validação dos indicadores adicionais ao construto “participação” elevou sua qualidade acima da zona exploratória, pois suas cargas externas ficaram acima de 0,7. Dessa forma, o aprimoramento realizado amplia a compreensão teórica e estatística do modelo, apto a ser aplicado aos diversos órgãos colegiados na gestão pública.

Palavras-chave:
transparência; participação; gestão social; cidadania deliberativa

Resumen

El objetivo de este estudio es mejorar el modelo estructural de ciudadanía deliberativa (Costa & Pinto, 2021, 2023), que se basa en estudios relacionados con la aplicación de criterios (Tenório, 2012) al campo teórico de la gestión social. Los datos se analizaron mediante modelos de ecuaciones estructurales (Hair, Hult, Ringle, & Sarstedt, 2017; Hair et al., 20211). 213 personas de varios territorios brasileños respondieron el cuestionario electrónico, invitadas a través de las redes sociales, con énfasis en LinkedIn. Además de confirmar la calidad estadística del modelo estructural de ciudadanía deliberativa (Costa & Pinto, 2021, 2023), se validaron dos indicadores propuestos para el constructo participación (Costa & Pinto, 2021, 2023). El constructo proceso deliberativo aún se encuentra en fase exploratoria, con una propuesta de prueba con un indicador adicional relacionado con el criterio de espacios transversales. La validación de indicadores adicionales al constructo participación elevó su calidad por encima de la zona exploratoria, ya que sus cargas externas fueron superiores a 0,7. De esta forma, la mejora realizada amplía la comprensión teórica y estadística del modelo, pudiendo ser aplicado a los distintos órganos colegiados de la gestión pública.

Palabras clave:
transparencia; participación; gestión social; ciudadanía deliberativa

1. INTRODUCTION

In the republican view, the structure of society is formed by the “political will of private subjects” (Habermas, 1997Habermas, J. (1997). Direito e democracia: entre a facticidade e a validade. Rio de Janeiro, RJ: Tempo Brasileiro., p. 20, our translation). This relationship illustrates the concept of democracy, in which decentralized administration is valued through collective appropriation of state bureaucratic power. In this context, deliberative democracy refers to “a process of institutionalization of a set of practices and rules, both formal and informal” (Tenório, 2016Tenório, F. G., & Kronemberger, T. S. (2016). Gestão social e conselhos gestores. Rio de Janeiro, RJ: FGV ., p. 29, our translation), while the focus of deliberative citizenship is the legitimation of political decisions (Tenório, 2007Tenório, F. G. (2007). Cidadania e desenvolvimento local. Ijuí, RS: Editora Unijuí.).

Habermas (1997Habermas, J. (1997). Direito e democracia: entre a facticidade e a validade. Rio de Janeiro, RJ: Tempo Brasileiro.) values the collective construction of the common good and the necessary political-administrative apparatus. In general, the author discusses issues related to political deliberation, such as pluralism of ideas, communicative relationships, argumentation, and consensus building. According to the author, social problems experienced in the private sphere are taken to the public sphere through civil society movements and entities. Thus, both public sphere and civil society correspond to useful concepts for understanding deliberative citizenship (Tenório & Kronemberger, 2016Tenório, F. G., & Kronemberger, T. S. (2016). Gestão social e conselhos gestores. Rio de Janeiro, RJ: FGV .).

In accordance with the current Federal Constitution (Constituição da República Federativa do Brasil de 1988), the ultimate goal of deliberative citizenship is the common good. This aim has been studied in the theoretical field of social management, which is “based on understanding, argumentation, and not on negotiation [or convincing] in the utilitarian sense of the term” (Cançado, Tenório, & Pereira, 2011Cançado, A. C., Tenório, F. G., & Pereira, J. R. (2011). Gestão social: reflexões teóricas e conceituais. Cadernos EBAPE.BR, 9(3), 681-703. Recuperado de https://doi.org/ 10.1590/S1679-39512011000300002
https://doi.org/10.1590/S1679-3951201100...
, p. 696, our translation).

The use of the perspective of deliberative citizenship in the theoretical field of social management in Brazil began with Tenório (1998Tenório, F. G. (1998). Gestão social: uma perspectiva conceitual. Revista de Administração Pública, 32(5), 7-23. Recuperado de https://periodicos.fgv.br/rap/article/view/7754
https://periodicos.fgv.br/rap/article/vi...
), who influenced a tradition of qualitative studies applied to “collegiate management spaces, predominantly in territories and public policy councils” (Salgado, Santos, Resende, & Souza, 2019Salgado, R. J. S. F., Santos, L. F., Resende, T. C., & Souza, W. J. (2019). Cidadania deliberativa e gestão social: revisão sistemática de literatura no Brasil. Cadernos EBAPE.BR, 17(Especial), 817-831. Recuperado de https://doi.org/10.1590/1679-395176139
https://doi.org/10.1590/1679-395176139...
, p. 819, our translation). In this way, deliberative citizenship criteria were consolidated for the analysis of decision-making processes in different territories contributing to local development (Tenório, 2012Tenório, F. G. (2012). Cidadania e desenvolvimento local: critérios de análise. Rio de Janeiro, RJ: FGV.).

The quantification of these criteria began in a doctoral thesis at the Brazilian School of Public and Business Administration of the Getulio Vargas Foundation (FGV EBAPE) (Costa, 2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
). Through exploratory factor analysis, three factors were extracted from these criteria: transparency, participation, and deliberative process. Thus, the precursor version of the structural model of deliberative citizenship was formed - transparency and participation positively affect the deliberative process (Costa, 2018). Moreover, Costa and Pinto (2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
, 2023Costa, J. C., & Pinto, J. F. (2023). Modelo estrutural de cidadania deliberativa: metodologia quantitativa proposta à gestão social. Administração Pública e Gestão Social, 15(2), 1-20. Recuperado de https://doi.org/10.21118/apgs.v15i3.14137
https://doi.org/10.21118/apgs.v15i3.1413...
) added the common good construct and validated the hypothesis that transparency and participation, mediated by the deliberative process, positively affect the common good. These authors identified opportunities for improvement in two constructs and presented them as suggestions for future research. They proposed the addition of two indicators to the participation construct and text review in one indicator of the deliberative process construct.

The aim of this research is precisely to verify the relevance of these propositions. The proposed additional tests required a re-reading of the respective analysis criteria for decision-making processes from the perspective of deliberative citizenship (Tenório, 2012Tenório, F. G. (2012). Cidadania e desenvolvimento local: critérios de análise. Rio de Janeiro, RJ: FGV.) for the interpretation of their meanings in the constructs. The estimation was performed using structural equation modeling with the SmartPLS 4 software (Ringle, Wende, & Becker, 2022 Ringle, C. M ., Wende, S., & Becker, J. M. (2022). SmartPLS 4. Oststeinbek, Germany: SmartPLS GmbH. Recuperado de http://www.smartpls.com) with the same settings as performed previously.

2. DELIBERATIVE CITIZENSHIP CRITERIA IN THE CONTEXT OF SOCIAL MANAGEMENT

Among the ways of understanding the concept of social management, its goals, and its opposition to strategic management stand out, social management is based on the idea that participation in decision-making processes should foster dialogue for the common good (Tenório & Kronemberger, 2016Tenório, F. G., & Kronemberger, T. S. (2016). Gestão social e conselhos gestores. Rio de Janeiro, RJ: FGV .).

The theoretical field of social management is useful for understanding the concept of deliberative citizenship, from which it appears that “the legitimacy of political decisions must originate in discussion processes, guided by the principles of inclusion, pluralism, participatory equality, autonomy, and the common good” (Tenório, 2007Tenório, F. G. (2007). Cidadania e desenvolvimento local. Ijuí, RS: Editora Unijuí., p. 54, our translation). This definition summarizes the categories that group the criteria for analyzing decision-making processes (Box 1).

A preliminary version of the deliberative citizenship criteria was presented by Tenório et al. (2008Tenório, F. G., Villela, L. E., Dias, A. F., Gurjão, F. V., Porto, E. C., & Viana, B. (2008). Critérios para a avaliação de processos decisórios participativos deliberativos na implementação de políticas públicas. In: Anais do 3º Encontro de Administração Pública e Governança, Salvador, BA.). The consolidated version is available in the chapter “Methodological scope” (Villela, 2012Villela, L. A. (2012). Escopo metodológico. In F. G. Tenório (Org.), Cidadania e desenvolvimento local: critérios de análise. Rio de Janeiro, RJ: FGV ., our translation) of the book Citizenship and local development: analysis criteria, organized by Tenório (2012Tenório, F. G. (2012). Cidadania e desenvolvimento local: critérios de análise. Rio de Janeiro, RJ: FGV.). A relevant part of these criteria constitutes the indicators of the structural model of deliberative citizenship.

Box 1
Deliberative citizenship: analysis criteria

The Habermasian concept of deliberative citizenship was integrated into the theoretical field of social management by Tenório (1998Tenório, F. G. (1998). Gestão social: uma perspectiva conceitual. Revista de Administração Pública, 32(5), 7-23. Recuperado de https://periodicos.fgv.br/rap/article/view/7754
https://periodicos.fgv.br/rap/article/vi...
), with a focus on participation, which aims at “self-realization [...] under the logic [...] of social democracy through political and decision-making equality” (Salgado et al., 2019Salgado, R. J. S. F., Santos, L. F., Resende, T. C., & Souza, W. J. (2019). Cidadania deliberativa e gestão social: revisão sistemática de literatura no Brasil. Cadernos EBAPE.BR, 17(Especial), 817-831. Recuperado de https://doi.org/10.1590/1679-395176139
https://doi.org/10.1590/1679-395176139...
, p. 818, our translation). Thus, the term “social management” in this line of research is not limited to the actions of governments and public organs for the benefit of society. More than that, it involves social emancipation, through established consensus based on Habermasian communicative rationality (Salgado et al., 2019Habermas, J. (1997). Direito e democracia: entre a facticidade e a validade. Rio de Janeiro, RJ: Tempo Brasileiro.; Tenório, 2008Tenório, F. G. (2008). Um espectro ronda o terceiro setor: o espectro do mercado. Ijuí, RS: Editora Unijuí .).

Deliberative citizenship criteria represent the magnitude of participation in terms of relevance, by the person or public management, and amplitude (Costa, 2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
; Costa & Pinto, 2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
). In this construct, participation, the “relationship with other participatory processes”, “citizen appreciation”, and “participatory evaluation” were initially considered (Costa, 2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
). Later, Costa and Pinto (2021)Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
proposed the inclusion of the “participation of different actors” and the “profile of the actors”, which are criteria related to pluralism.

Transparency, in turn, is represented by the first two analysis criteria: dissemination channels and information quality. The literature on social management and deliberative citizenship values transparency as essential to communicative action, considering that knowledge must be shared in the discussion processes.

Thus, in a social relationship that intends to be participatory, knowledge must be convergent. The knowledge of those who studied should be used to support the discussions, but not as a first guide in the decision. In a collective relationship, power is diluted among the participants since knowledge and information are shared, with no ‘owners of truth’ (Tenório & Rozenberg, 1997Tenório, F. G., & Rozenberg, J. E. (1997). Gestão pública e cidadania: metodologias participativas em ação. Revista de Administração Pública, 31(4), 101-125. Recuperado de https://periodicos.fgv.br/rap/article/view/7882
https://periodicos.fgv.br/rap/article/vi...
, p. 163, our translation).

The dissemination channels and information quality can be interpreted in the conceptual terms of transparency developed by Michener and Bersch (2013Michener, G., & Bersch, K. (2013). Identifying transparency. Information Polity, 18(3), 233-242. Recuperado de https://doi.org/10.3233/IP-130299
https://doi.org/10.3233/IP-130299...
). In short, transparency is visibility (available access) and inference ability, which can be achieved by simplifying and disaggregating information, in addition to an independent audit that can attest to the veracity of information (Michener & Bersch, 2013Michener, G., & Bersch, K. (2013). Identifying transparency. Information Polity, 18(3), 233-242. Recuperado de https://doi.org/10.3233/IP-130299
https://doi.org/10.3233/IP-130299...
).

While the purpose of social management is the common good of society (Tenório, 2006Tenório, F. G. (2006). A trajetória do programa de estudos em gestão social (Pegs). Revista de Administração Pública, 40(6), 1145-1162. Recuperado de https://doi.org/10.1590/S0034-76122006000600011
https://doi.org/10.1590/S0034-7612200600...
) in clear distinction from strategic management (Cançado et al., 2011Cançado, A. C., Tenório, F. G., & Pereira, J. R. (2011). Gestão social: reflexões teóricas e conceituais. Cadernos EBAPE.BR, 9(3), 681-703. Recuperado de https://doi.org/ 10.1590/S1679-39512011000300002
https://doi.org/10.1590/S1679-3951201100...
; Costa, 2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
; Tenório & Kronemberger, 2016Tenório, F. G., & Kronemberger, T. S. (2016). Gestão social e conselhos gestores. Rio de Janeiro, RJ: FGV .), deliberative citizenship is related to criteria that legitimize collective efforts in this direction (Tenório, 2007Tenório, F. G. (2007). Cidadania e desenvolvimento local. Ijuí, RS: Editora Unijuí.). Thus, this research is in line with the idea of co-production of the public good (R. B. Denhardt & J. V. Denhardt, 2000Denhardt, R. B., & Denhardt, J. V. (2000). The new public service: serving rather than steering. Public Administration Review, 60(6), 549-559. Recuperado de http://www.jstor.org/stable/977437
http://www.jstor.org/stable/977437...
), which “starts from social participation, but necessarily goes through the deliberative process since the effect between participation and the common good is totally mediated” (Costa & Pinto, 2021, p. 13, our translation).

Habermas’ communicative rationality presupposes appreciating the world of life in political discussions. In this way, participation would be influenced by culture (more social aspect) and personality (more individual aspect). It is about the search for autonomy, but also for solidarity and the establishment of consensus (Salgado et al., 2019Salgado, R. J. S. F., Santos, L. F., Resende, T. C., & Souza, W. J. (2019). Cidadania deliberativa e gestão social: revisão sistemática de literatura no Brasil. Cadernos EBAPE.BR, 17(Especial), 817-831. Recuperado de https://doi.org/10.1590/1679-395176139
https://doi.org/10.1590/1679-395176139...
). In a literature review, Salgado et al. (2019)Tenório, F. G. (1998). Gestão social: uma perspectiva conceitual. Revista de Administração Pública, 32(5), 7-23. Recuperado de https://periodicos.fgv.br/rap/article/view/7754
https://periodicos.fgv.br/rap/article/vi...
identified that

[...] the original reading by Tenório (1998Tenório, F. G. (1998). Gestão social: uma perspectiva conceitual. Revista de Administração Pública, 32(5), 7-23. Recuperado de https://periodicos.fgv.br/rap/article/view/7754
https://periodicos.fgv.br/rap/article/vi...
) was maintained over 20 years and thus the concept of deliberative citizenship is aligned with that of social management and contributed to the theoretical debate and empirical studies in the field of civil society organizations, specifically in collegiate organs (Salgado et al., 2019Salgado, R. J. S. F., Santos, L. F., Resende, T. C., & Souza, W. J. (2019). Cidadania deliberativa e gestão social: revisão sistemática de literatura no Brasil. Cadernos EBAPE.BR, 17(Especial), 817-831. Recuperado de https://doi.org/10.1590/1679-395176139
https://doi.org/10.1590/1679-395176139...
, p. 829, our translation).

Although research involving social management and deliberative citizenship began in Brazil in 1998, Salgado et al. (2019Salgado, R. J. S. F., Santos, L. F., Resende, T. C., & Souza, W. J. (2019). Cidadania deliberativa e gestão social: revisão sistemática de literatura no Brasil. Cadernos EBAPE.BR, 17(Especial), 817-831. Recuperado de https://doi.org/10.1590/1679-395176139
https://doi.org/10.1590/1679-395176139...
) did not identify a quantitative methodology, which has been a newer construction (Costa, 2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
; Costa & Pinto, 2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
, 2023Costa, J. C., & Pinto, J. F. (2023). Modelo estrutural de cidadania deliberativa: metodologia quantitativa proposta à gestão social. Administração Pública e Gestão Social, 15(2), 1-20. Recuperado de https://doi.org/10.21118/apgs.v15i3.14137
https://doi.org/10.21118/apgs.v15i3.1413...
).

Costa and Pinto presented the Structural Model of Deliberative Citizenship at the EnANPAD 2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
(XLV Nacional Meeting of Postgraduate Studies and Research in Administration), based on the construction performed in Costa’s doctoral thesis (2018). Research by Costa and Pinto (2021)Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
identified opportunities for improving the referred structural model.

The developed method consists of four constructs: three identified in the thesis (Costa, 2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
) and one added later (Costa & Pinto, 2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
). It was demonstrated that transparency and participation, mediated by the deliberative process, positively affect the common good. As a suggestion for future research, Costa and Pinto (2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
) identified the need to add two indicators for the participation construct and the possibility of improving the text of an indicator of the deliberative process construct.

Box 2summarizes the criteria for deliberative citizenship in the structural models of previous research. It is observed that one criterion related to autonomy (“Possibility of exercising their willingness”) has not yet had an indicator formulated. This criterion was not the object of this research. If elaborated, the alignment of the respective indicator which the participation construct can be tested.

Box 2
Deliberative citizenship criteria (Tenório, 2012Tenório, F. G. (2012). Cidadania e desenvolvimento local: critérios de análise. Rio de Janeiro, RJ: FGV.) in previous research

3. STRUCTURAL MODEL OF DELIBERATIVE CITIZENSHIP

Costa (2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
) validated the proposition that transparency and participation positively affect the deliberative process, while Costa and Pinto (2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
) expanded the model, based on the recommendations for future research suggested in the thesis (Costa, 2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
). Thus, the current version was achieved (Figure 1), which includes the construct related to social welfare. In summary, transparency and participation, mediated by the deliberative process, positively affect the common good.

Figure 1
Structural Model of Deliberative Citizenship

The recommendation to expand the transparency construct (Costa, 2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
) was met by testing the TR3, TR4, and TR5 indicators (Costa & Pinto, 2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
), whose result is summarized in Box 3.

Box 3
Transparency indicator test results

Among the tested transparency indicators, one was not validated (TR5). This result may be due to the incipience of the theme “independent audit”, which would cause low consistency in the responses and consequently a low level of significance of the respective results. On the other hand, the indicators proposed for the common good (Costa & Pinto, 2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
), in response to Costa’s proposal (2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
), were fully validated, as shown in Box 4.

Box 4
Validated common good indicators

In addition to the analysis criteria, the indicators for the common good construct were based on the relationship between common good and social welfare (Costa & Pinto, 2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
, 2023Costa, J. C., & Pinto, J. F. (2023). Modelo estrutural de cidadania deliberativa: metodologia quantitativa proposta à gestão social. Administração Pública e Gestão Social, 15(2), 1-20. Recuperado de https://doi.org/10.21118/apgs.v15i3.14137
https://doi.org/10.21118/apgs.v15i3.1413...
). The following boxes present the indicators of the other constructs - participation and deliberative process -, as validated (Costa, 2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
) and confirmed (Costa & Pinto, 2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
, 2023Costa, J. C., & Pinto, J. F. (2023). Modelo estrutural de cidadania deliberativa: metodologia quantitativa proposta à gestão social. Administração Pública e Gestão Social, 15(2), 1-20. Recuperado de https://doi.org/10.21118/apgs.v15i3.14137
https://doi.org/10.21118/apgs.v15i3.1413...
) in previous research.

There is an understanding that the participation construct is related to the relevance and amplitude of the social actor’s involvement in the deliberative process (Costa, 2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
; Costa & Pinto, 2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
, 2023Costa, J. C., & Pinto, J. F. (2023). Modelo estrutural de cidadania deliberativa: metodologia quantitativa proposta à gestão social. Administração Pública e Gestão Social, 15(2), 1-20. Recuperado de https://doi.org/10.21118/apgs.v15i3.14137
https://doi.org/10.21118/apgs.v15i3.1413...
). This deduction is reinforced by the correlation between the analysis criteria of this construct (Costa, 2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
).

Box 5
Validated and confirmed participation indicators

The deliberative process construct should not be confused with participation. Instead of the social actor, the focus of the deliberative process is the public policy management council. This grouping of decision-making analysis criteria from the perspective of deliberative citizenship (Tenório, 2012Tenório, F. G. (2012). Cidadania e desenvolvimento local: critérios de análise. Rio de Janeiro, RJ: FGV.) was made possible by exploratory factor analysis (Costa, 2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
). In general, participation refers to the social actor, the deliberative process is due to the characteristics of the institutional environment and transparency is a result of the ability to infer relevant information (Tenório, 2007Tenório, F. G. (2007). Cidadania e desenvolvimento local. Ijuí, RS: Editora Unijuí., 2012Tenório, F. G. (2012). Cidadania e desenvolvimento local: critérios de análise. Rio de Janeiro, RJ: FGV.; Costa, 2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
; Costa & Pinto, 2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
, 2023Costa, J. C., & Pinto, J. F. (2023). Modelo estrutural de cidadania deliberativa: metodologia quantitativa proposta à gestão social. Administração Pública e Gestão Social, 15(2), 1-20. Recuperado de https://doi.org/10.21118/apgs.v15i3.14137
https://doi.org/10.21118/apgs.v15i3.1413...
).

Box 6
Validated and confirmed deliberative process indicators

Although Costa and Pinto (2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
, 2023Costa, J. C., & Pinto, J. F. (2023). Modelo estrutural de cidadania deliberativa: metodologia quantitativa proposta à gestão social. Administração Pública e Gestão Social, 15(2), 1-20. Recuperado de https://doi.org/10.21118/apgs.v15i3.14137
https://doi.org/10.21118/apgs.v15i3.1413...
) did not test new indicators for the participation and deliberative process constructs, they identified the need to increase their quality. To this end, they proposed testing two additional indicators for the participation construct (PA4 and PA5) and a change in the wording of the first indicator of the deliberative process construct (DP1). Box 7 shows the indicators created or redefined.

Box 7
Indicators created (participation) or redefined (deliberative process)

Indicator PA4 meets the criterion “participation of different actors” (Tenório, 2012Tenório, F. G. (2012). Cidadania e desenvolvimento local: critérios de análise. Rio de Janeiro, RJ: FGV.), previously tested with the following wording: “Movements, organizations and also unorganized people can interfere in the deliberative process of the departmental council” (Costa, 2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
, our translation). PA5 is also proposed, resulting from the redefinition of the respective indicator previously tested (Costa, 2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
), corresponding to the criterion “profile of the actors” (Tenório, 2012Tenório, F. G. (2012). Cidadania e desenvolvimento local: critérios de análise. Rio de Janeiro, RJ: FGV.). “Together, PA4 and PA5 refer to the multiplicity of actors involved in decision-making” (Costa & Pinto, 2021, p. 14, our translation).

As for the DP1 indicator, according to Costa and Pinto (2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
, p. 14, our translation), “the term ‘local’ was removed, the plural of the term ‘community’ was used and the scope of analysis was delimited by means of the expression ‘territory where I live’”. These alterations were expected to have provided a significant improvement in the measurement of constructs and the structural model. All proposed indicators (Costa & Pinto, 2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
, 2023Costa, J. C., & Pinto, J. F. (2023). Modelo estrutural de cidadania deliberativa: metodologia quantitativa proposta à gestão social. Administração Pública e Gestão Social, 15(2), 1-20. Recuperado de https://doi.org/10.21118/apgs.v15i3.14137
https://doi.org/10.21118/apgs.v15i3.1413...
) were validated, but the results related to the participation construct were more expressive.

4. METHODOLOGICAL PROCEDURES

The PA4 and PA5 indicators proposed for the structural model of deliberative citizenship (Costa & Pinto, 2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
), as well as the new wording for PD1, were tested with the Partial Least Squares Structural Equation Modeling (PLS-SEM), with the help of the SmartPLS 4 application (Ringle et al., 2022 Ringle, C. M ., Wende, S., & Becker, J. M. (2022). SmartPLS 4. Oststeinbek, Germany: SmartPLS GmbH. Recuperado de http://www.smartpls.com). For Hair et al. (2017Hair, J. F. Jr., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2017). A primer on partial least squares structural equation modeling (PLS-SEM). London, UK: Sage., 2021Hair, J. F. Jr, Hult, G. T. M., Ringle, C. M., Sarstedt, M., Danks, N. P., & Ray, S.. (2021). Partial least squares structural equation modeling (PLS-SEM) using R: a workbook. Recuperado de https://doi.org/10.1007/978-3-030-80519-7
https://doi.org/10.1007/978-3-030-80519-...
), this method privileges the explanatory capacity of the constructs that affect the common good. In measuring the constructs, criteria of convergent validity, internal consistency, and discriminant validity were adopted.

Table 1
Reference criteria for validation of constructs

The minimum value of 0.6 for the outer loadings of constructs is still acceptable in this phase of model enhancement. However, a minimum value of 0.7 was expected. “The expectation, following the recommendations of Hulland (1999Hulland, J. (1999). Use of partial least squares (PLS) in Strategic management research: a review of four recent studies. Strategic Management Journal, 20(2), 195-204. Recuperado de http://www.jstor.org/stable/3094025), is that the outer loadings below 0.7, which is the ideal minimum value, will increase according to the improvement of the scale” (Costa & Pinto, 2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
, p. 10, our translation). While this coefficient is useful for measuring the effect of the construct on each indicator belonging to it, the average variance extracted (AVE) represents the percentage of the variance of the set of indicators on each construct (Costa, 2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
; Hair et al., 2017Hair, J. F. Jr., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2017). A primer on partial least squares structural equation modeling (PLS-SEM). London, UK: Sage., 2021Hair, J. F. Jr, Hult, G. T. M., Ringle, C. M., Sarstedt, M., Danks, N. P., & Ray, S.. (2021). Partial least squares structural equation modeling (PLS-SEM) using R: a workbook. Recuperado de https://doi.org/10.1007/978-3-030-80519-7
https://doi.org/10.1007/978-3-030-80519-...
).

Internal consistency, in general terms, means how much the indicators converge to the construct. That is, it measures how the construct indicators, associated, are assertive in measuring it. While Cronbach’s alpha is a conservative measure, composite reliability (CR) is more liberal. Thus, the actual measure of reliability must be between both, expecting values between 0.7 and 0.95 (Costa, 2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
; Hair et al., 2017Hair, J. F. Jr., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2017). A primer on partial least squares structural equation modeling (PLS-SEM). London, UK: Sage., 2021Hair, J. F. Jr, Hult, G. T. M., Ringle, C. M., Sarstedt, M., Danks, N. P., & Ray, S.. (2021). Partial least squares structural equation modeling (PLS-SEM) using R: a workbook. Recuperado de https://doi.org/10.1007/978-3-030-80519-7
https://doi.org/10.1007/978-3-030-80519-...
).

Discriminant validity serves to distinguish each construct from the others. This test is important to avoid mistaken validation of direct and indirect effects. Cross loadings indicate whether the construct’s indicators have a stronger relationship with a construct other than the one to which it belongs. Although the cross loading test is common, HTMT is more reliable. In it, values are expected not to be greater than 0.85, in general cases, and 0.9, in the case of conceptually similar constructs (Costa, 2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
; Hair et al., 2017Hair, J. F. Jr., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2017). A primer on partial least squares structural equation modeling (PLS-SEM). London, UK: Sage., 2021Hair, J. F. Jr, Hult, G. T. M., Ringle, C. M., Sarstedt, M., Danks, N. P., & Ray, S.. (2021). Partial least squares structural equation modeling (PLS-SEM) using R: a workbook. Recuperado de https://doi.org/10.1007/978-3-030-80519-7
https://doi.org/10.1007/978-3-030-80519-...
). In addition, it is recommended to check whether, at a significance level of 5%, the HTMT values between the constructs are less than 1, given that higher values mean that there is an overlapping of constructs, which compromises the quality of the model to represent reality.

Data were collected via an online questionnaire (Google Forms), with the application of a seven-point scale (Likert, 1932Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 22(140), 1-55.) for each indicator, in addition to questions referring to the respondents’ profile. Invitations were sent to different profiles, considering academic backgrounds, to reduce possible selection bias. 213 people from different locations in Brazil participated in the survey, with an emphasis on the Southeast Region.

Graph 1
Respondents’ territorial origin

Although 124 respondents do not have experience working in deliberative councils of social interest, approximately 45.16% of them have knowledge in an area at the municipal, state, or national level. Among the 89 who not only know but also have experience acting as a representative, the main occurrences of segments were “public sector” (39) and “popular representative” (34). Representatives of a segment of the private sector totaled 16 people.

Compared to the previous research (Costa & Pinto, 2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
, 2023Costa, J. C., & Pinto, J. F. (2023). Modelo estrutural de cidadania deliberativa: metodologia quantitativa proposta à gestão social. Administração Pública e Gestão Social, 15(2), 1-20. Recuperado de https://doi.org/10.21118/apgs.v15i3.14137
https://doi.org/10.21118/apgs.v15i3.1413...
), the current one had lower participation of people with high school, undergraduate, or specialization academic training. On the other hand, it had greater participation of people with master’s, doctorate, or post-doctoral degrees. Although there was a decrease in some academic degrees, the data are well distributed.

Graph 2
Respondents’ academic degree

The survey is gender balanced (112 masculines, 101 feminines), and had people from different ethnic origins: white or Caucasian (108); black or brown (76); mixed race or mestizo (19); oriental, Asian or yellow (3); indigenous or red (2). Five people preferred not to declare their ethnic origin. The invitation to people with academic training in different areas of knowledge was also valued. Thus, data collection was carried out along the lines of the research in which the structural model of deliberative citizenship was proposed (Costa & Pinto, 2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
, 2023Costa, J. C., & Pinto, J. F. (2023). Modelo estrutural de cidadania deliberativa: metodologia quantitativa proposta à gestão social. Administração Pública e Gestão Social, 15(2), 1-20. Recuperado de https://doi.org/10.21118/apgs.v15i3.14137
https://doi.org/10.21118/apgs.v15i3.1413...
).

This diversity in the profile of the respondents provides quality in measuring the constructs and the cause-effect relationships (path coefficients) foreseen in the structural model. These path coefficients were tested by bootstrapping (with a significance level of 5%) [...]. This technique is used in estimates related to validation criteria for constructs, as well as for the structural model (Costa & Pinto, 2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
, p. 9, our translation).

In addition to testing the indicators, using the criteria in Table 1, improvements in path coefficients were analyzed based on the criteria shown in Box 8. For Q2, according to the construct cross-validated redundancy criterion, measured by blindfolding, it is expected that the value is not negative, indicating the predictive relevance of the structural model (Hair et al., 2017Hair, J. F. Jr., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2017). A primer on partial least squares structural equation modeling (PLS-SEM). London, UK: Sage.). The variance inflation factor (VIF) must be less than 5 so that there is no collinearity between predictor constructs greater than 0.20 (Hair et al., 2017Hair, J. F. Jr., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2017). A primer on partial least squares structural equation modeling (PLS-SEM). London, UK: Sage.). The f2 is used to measure the intensity of the effects relative to the path coefficients. For this purpose, Hair et al. (2017Hair, J. F. Jr., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2017). A primer on partial least squares structural equation modeling (PLS-SEM). London, UK: Sage.) resorted to Cohen (1988Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2a ed.). Hillsdale, MI: Lawrence Erlbaum Associates.), who defines that the effect is only considered when f2 is equal to or greater than 0.02 and that the values 0.15 and 0.35 mark medium and large effects, respectively (Costa, 2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
, p. 109).

Box 8
Criteria for the analysis of path coefficients

A broader assessment can be performed using the determination coefficients (R2) of the endogenous constructs: deliberative process and common good. R2 is expected between 0.50, moderate, and 0.75, substantial (Costa & Pinto, 2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
, 2023Costa, J. C., & Pinto, J. F. (2023). Modelo estrutural de cidadania deliberativa: metodologia quantitativa proposta à gestão social. Administração Pública e Gestão Social, 15(2), 1-20. Recuperado de https://doi.org/10.21118/apgs.v15i3.14137
https://doi.org/10.21118/apgs.v15i3.1413...
; Hair, Ringle, & Sarstedt, 2011Hair, J. F. Jr., Ringle, C. M., & Sarstedt, M. (2011). Pls-Sem: indeed a silver bullet. Journal of Marketing theory and Practice, 19(2), 139-152. Recuperado de https://doi.org/10.2753/MTP1069-6679190202
https://doi.org/10.2753/MTP1069-66791902...
; Henseler, Ringle, & Sinkvics, 2009Henseler, J., Ringle, C. M ., & Sinkovics, R. R. (2009). The use of partial least squares path modeling in international marketing. In J. Henseler, C. M. Ringle, & R. R. Sinkovics (Eds.), New challenges to international marketing. Bingley, UK: Emerald Group Publishing Limited.). An R2 below the moderate level may be due to a lack of knowledge of factors relevant to the variance of the explained construct.

5. ANALYSIS AND DISCUSSION OF THE RESULTS

Table 2 presents the results related to the measurement of the constructs. The transparency and common good constructs did not receive changes and remain with satisfactory convergent validity, internal consistency, and discriminant validity. The participation construct received two indicators (PA4 and PA5), which resulted in outer loadings higher than 0.7. Thus, it surpassed the quality of the construct in previous research (Costa & Pinto, 2023Costa, J. C., & Pinto, J. F. (2023). Modelo estrutural de cidadania deliberativa: metodologia quantitativa proposta à gestão social. Administração Pública e Gestão Social, 15(2), 1-20. Recuperado de https://doi.org/10.21118/apgs.v15i3.14137
https://doi.org/10.21118/apgs.v15i3.1413...
).

On the other hand, the construct deliberative process had a simple wording change in one of its indicators (DP1), which remained with outer loading between 0.6 and 0.7. Then an ideal quality was reached in the participation construct and there is still room for improvement in the deliberative process construct.

Table 2
Validation of constructs

In addition to outer loadings, convergent validity was measured by AVE. The lowest AVE was 0.611 (deliberative process), which satisfactorily meets the expected level (> 0.5). An improvement in this construct may provide an elevation of its AVE.

Two measures of internal consistency were used: Cronbach’s alpha and composite reliability. There are two types of composite reliability: a liberal one (rhoC), which presents higher values for internal consistency, and a more reasonable one, which is an intermediate value between Cronbach’s Alpha and the most liberal measure (Hair et al., 2021Hair, J. F. Jr, Hult, G. T. M., Ringle, C. M., Sarstedt, M., Danks, N. P., & Ray, S.. (2021). Partial least squares structural equation modeling (PLS-SEM) using R: a workbook. Recuperado de https://doi.org/10.1007/978-3-030-80519-7
https://doi.org/10.1007/978-3-030-80519-...
). Analyzing the decrease in the composite reliability index (rhoA) from 0.945 (Costa & Pinto, 2023Costa, J. C., & Pinto, J. F. (2023). Modelo estrutural de cidadania deliberativa: metodologia quantitativa proposta à gestão social. Administração Pública e Gestão Social, 15(2), 1-20. Recuperado de https://doi.org/10.21118/apgs.v15i3.14137
https://doi.org/10.21118/apgs.v15i3.1413...
) to 0.916, it can be seen that the common good construct reduced the redundancy of its indicators (Hair et al., 2021Hair, J. F. Jr., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2017). A primer on partial least squares structural equation modeling (PLS-SEM). London, UK: Sage.).

Discriminant validity was confirmed by analyzing the cross loadings and HTMT. While the first indicated that there is no indicator more correlated with another construct than the one to which it belongs, the second served to verify that there are no significantly similar constructs in this research. As seen in Table 3, the confidence interval does not include the number 1 because the highest upper limit is equal to 0.904. In other words, the transparency, participation, deliberative process, and common good constructs are in fact different from each other.

Table 3
HTMT

Thus, the inclusion of two indicators to the participation construct (PA4; PA5) was effective. In the deliberative process, in turn, the new wording for the DP1 indicator is an intermediate possibility, which contributes to the permanence of this construct in an exploratory phase.

This persistence of the DP1 indicator in the exploratory range directs to the review of one relevant analysis criterion: cross-cutting spaces, which are “spaces that cross sectors to integrate different points of view” (Tenório, 2012Tenório, F. G. (2012). Cidadania e desenvolvimento local: critérios de análise. Rio de Janeiro, RJ: FGV., p. 39, our translation). While DP1 contemplated the notion of spaces, it missed the perception of transversality. Thus, in line with the perspective absent in DP1, to complete the content proposed in the respective analysis criterion, it is proposed to test one new indicator: DP8 - in the territory where I live, there are spaces for the integration of different viewpoints, from varied sectors.

After measuring the constructs, the structural model was estimated based on direct and indirect effects between constructs. The analysis of path coefficients (Table 4) confirmed the lack of direct effect of participation on the common good. This inexpressiveness is attributed to the p-value (0.815) because, considering the significance level of 5%, a p-value of up to 0.05 is expected. The other direct effects exceed the level below which they are considered low or inexpressive (0.1).

Table 4
Path coefficients (direct effects)

It is worth emphasizing the increase in the effect of participation on the deliberative process in this research (0.264) compared to the previous one (0.163) (Costa & Pinto, 2023Costa, J. C., & Pinto, J. F. (2023). Modelo estrutural de cidadania deliberativa: metodologia quantitativa proposta à gestão social. Administração Pública e Gestão Social, 15(2), 1-20. Recuperado de https://doi.org/10.21118/apgs.v15i3.14137
https://doi.org/10.21118/apgs.v15i3.1413...
). This difference is a reflection of the improvement of the constructs involved. The effect of transparency on the common good (0.190) is the lowest among the valid direct ones. Beyond this direct effect, transparency, mediated by the deliberative process, positively affects the common good with a higher coefficient (0.282), considered average or typical. In Table 5, the indirect effects are presented.

Table 5
Indirect effects (mediation)

As there is a direct and indirect effect of transparency on the common good, it is concluded that the mediation between these constructs is partial. On the other hand, the effect of participation on the common good is fully mediated because there is only an indirect effect (0.127) since the direct effect was considered insignificant. It is also observed that this coefficient was low (0.063) in the previous research (Costa & Pinto, 2023Costa, J. C., & Pinto, J. F. (2023). Modelo estrutural de cidadania deliberativa: metodologia quantitativa proposta à gestão social. Administração Pública e Gestão Social, 15(2), 1-20. Recuperado de https://doi.org/10.21118/apgs.v15i3.14137
https://doi.org/10.21118/apgs.v15i3.1413...
). Thus, the quality of the proposed improvement to the structural model of deliberative citizenship is clear.

In practical terms, Costa (2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
) distinguished between the deliberative process and the main factors relevant to its quality: transparency and participation. Further on, Costa and Pinto (2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
, 2023Costa, J. C., & Pinto, J. F. (2023). Modelo estrutural de cidadania deliberativa: metodologia quantitativa proposta à gestão social. Administração Pública e Gestão Social, 15(2), 1-20. Recuperado de https://doi.org/10.21118/apgs.v15i3.14137
https://doi.org/10.21118/apgs.v15i3.1413...
) demonstrated, through indirect effects, how the deliberative process is a means to value the common good in society. The present research corroborates these results.

The deliberative process had the R2 increased from 0.547 (Costa & Pinto, 2023Costa, J. C., & Pinto, J. F. (2023). Modelo estrutural de cidadania deliberativa: metodologia quantitativa proposta à gestão social. Administração Pública e Gestão Social, 15(2), 1-20. Recuperado de https://doi.org/10.21118/apgs.v15i3.14137
https://doi.org/10.21118/apgs.v15i3.1413...
) to 0.572. Regarding the common good, its coefficient of determination (R2) was increased from 0.338 to 0.388. The change was small but consistent with the improvement made. These data indicate that 57.2% of the variation in the deliberative process and 38.8% of the common good are explained by the structural model being improved. Figure 2 summarizes the results.

Figure 2
Measurement of the structural model of deliberative citizenship

VIF analysis indicated no excessive collinearity between constructs and between indicators. Thus, it is reinforced that the components of the structural model are not redundant. The Q2 test had a positive result, which corroborates the predictive relevance of the structural model. The f2 analysis confirms the non-existence of the direct effect of participation on the common good (0.000), but casts doubt on the direct effect of transparency on the common good, given that the f2 relative to this effect is very low (0.027), p-value of 0.284.

Considering f2, it is stated that both participation and transparency, fully mediated by the deliberative process, positively affect the common good. Mediation is the main contribution of the structural model of deliberative citizenship, ratified in all scenarios presented in this research.

The structural model of deliberative citizenship has been improved. Despite not reaching a moderate R2 for the common good construct, it establishes consistent direct and indirect effects, in addition to validating two indicators of the “participation” construct, whose outer loadings had substantially improved values.

Therefore, the theoretical understanding of the criteria of deliberative citizenship and the factors related to the decision-making processes in the scope of social management was expanded. The two indicators proposed for the participation construct (Costa & Pinto, 2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
, 2023Costa, J. C., & Pinto, J. F. (2023). Modelo estrutural de cidadania deliberativa: metodologia quantitativa proposta à gestão social. Administração Pública e Gestão Social, 15(2), 1-20. Recuperado de https://doi.org/10.21118/apgs.v15i3.14137
https://doi.org/10.21118/apgs.v15i3.1413...
) were sufficient to reach the desired level of quality. On the other hand, the result of the new wording proposed for a deliberative process indicator (Costa & Pinto, 2021Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
http://anpad.com.br/uploads/articles/114...
, 2023Costa, J. C., & Pinto, J. F. (2023). Modelo estrutural de cidadania deliberativa: metodologia quantitativa proposta à gestão social. Administração Pública e Gestão Social, 15(2), 1-20. Recuperado de https://doi.org/10.21118/apgs.v15i3.14137
https://doi.org/10.21118/apgs.v15i3.1413...
), regardless of being satisfactory in exploratory research, provoked a closer look at the respective analysis criterion (cross-cutting spaces), leading to the proposition of one new indicator. Just as the outer loadings of the participation construct were satisfactorily adjusted, it is suggested that there is still room for adjustments in the deliberative process construct.

6. FINAL CONSIDERATIONS

The improvement of this instrument favors more precise analyses concerning several factors that affect the deliberative process. If the questionnaire is periodically applied, it can be inferred whether the quality of the deliberative process is increasing or decreasing. It is also possible to verify whether socioeconomic factors, such as the human development index (HDI), interfere with the perception of the common good.

This model, as a representation of reality, can be adapted according to what is most relevant to social actors interested in deliberative citizenship. Beyond the relationship between the constructs, the model allows testing of the interference of control variables (Costa, 2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
), such as the academic/professional profile or the type of representation in public policy management councils. This is a possibility for further analysis.

The notion of quality of management organs is implicit in the indicators. However, a specific construct was not created, even though there are, among the criteria of deliberative citizenship, two specific indicators for this purpose: existing agencies and monitoring agency, related to the discussion process. Another indicator related to the management is the “responsibility of the actors”, relative to the local administrations (Tenório, 2012Tenório, F. G. (2012). Cidadania e desenvolvimento local: critérios de análise. Rio de Janeiro, RJ: FGV., p. 39). The quality of management is relevant, but it can also be measured by more traditional indicators.

It is important to have public policy monitoring agencies, as well as the quality of the deliberative process and the effectiveness of social management in valuing the common good. Thereby, recognition by the actors of a participatory methodology is necessary in the social, political, and technical ambits, that is, a social, political, and technical acceptance, criterion related to the inclusion (Tenório, 2012Tenório, F. G. (2012). Cidadania e desenvolvimento local: critérios de análise. Rio de Janeiro, RJ: FGV., p. 39). Perhaps this criterion was not absorbed by the model due to its triple scope. Regardless of not being included in the model, it is a process that is very relevant to the co-production of the public good.

The application of this methodology is a form of social participation. Although elaborated by researchers, the assimilation and use of this knowledge by people who participate more actively in public policy management councils are encouraged. For this purpose, it is necessary to value the criteria of deliberative citizenship related to autonomy, with emphasis on the possibility of exercising their willingness. It is expected that the participants, upon learning about the factors that limit the quality of the deliberative process in their respective municipalities, will propose scientifically based solutions to the problems of social management. Then local and national initiatives are valued, to monitor social management, with regard to the criteria adopted in the structural model of deliberative citizenship and its improvements.

In this research, the municipal scope was chosen due to its proximity to a greater number of social actors. However, the text of the indicators can be adapted by changing the relevant parts. In fact, the transparency, participation, and deliberative process constructs were initially validated in the context of university management (Costa, 2018Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
http://hdl.handle.net/10438/25764...
). Considering that the context of citizen participation is relevant for the adaptation and application of the questionnaire, this model can be applied in conjunction with integrative methodologies, which search “to value sensibilities and intuitions already discarded by the classic view of science as tools of social management” (Giannella, 2014Giannella, V. (2014). Metodologias integrativas. In R. F. Boullosa (Org.), Dicionário para a formação em gestão social. Salvador, BA: CIAGS/UFBA., p. 113, our translation).

It is interesting to use the constructs of deliberative citizenship in bibliometric analyses. For example, it is to be expected an expressive amount of works that links transparency and participation, even in areas not specific to social management. However, it is notable that the discussion around the common good is less recurrent. These studies could initially be performed in journals in the field of public administration.

REFERÊNCIAS

  • Cançado, A. C., Tenório, F. G., & Pereira, J. R. (2011). Gestão social: reflexões teóricas e conceituais. Cadernos EBAPE.BR, 9(3), 681-703. Recuperado de https://doi.org/ 10.1590/S1679-39512011000300002
    » https://doi.org/10.1590/S1679-39512011000300002
  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2a ed.). Hillsdale, MI: Lawrence Erlbaum Associates.
  • Constituição da República Federativa do Brasil de 1988 (1988). Brasília, DF: Senado Federal. Recuperado de https://www.planalto.gov.br/ccivil_03/constituicao/constituicao.htm
    » https://www.planalto.gov.br/ccivil_03/constituicao/constituicao.htm
  • Costa, J. C. (2018). Transparência na gestão universitária sob a perspectiva da participação cidadã (Tese de Doutorado). Fundação Getulio Vargas, Rio de Janeiro, RJ. Recuperado de http://hdl.handle.net/10438/25764
    » http://hdl.handle.net/10438/25764
  • Costa, J. C., & Pinto, J. F. (2021). Modelo estrutural de cidadania deliberativa. In Anais do 45º Encontro da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ. Recuperado de http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
    » http://anpad.com.br/uploads/articles/114/approved/ca43108ded5aabc7793d3f9b928cdd54.pdf
  • Costa, J. C., & Pinto, J. F. (2023). Modelo estrutural de cidadania deliberativa: metodologia quantitativa proposta à gestão social. Administração Pública e Gestão Social, 15(2), 1-20. Recuperado de https://doi.org/10.21118/apgs.v15i3.14137
    » https://doi.org/10.21118/apgs.v15i3.14137
  • Denhardt, R. B., & Denhardt, J. V. (2000). The new public service: serving rather than steering. Public Administration Review, 60(6), 549-559. Recuperado de http://www.jstor.org/stable/977437
    » http://www.jstor.org/stable/977437
  • Giannella, V. (2014). Metodologias integrativas. In R. F. Boullosa (Org.), Dicionário para a formação em gestão social Salvador, BA: CIAGS/UFBA.
  • Habermas, J. (1997). Direito e democracia: entre a facticidade e a validade Rio de Janeiro, RJ: Tempo Brasileiro.
  • Hair, J. F. Jr., Ringle, C. M., & Sarstedt, M. (2011). Pls-Sem: indeed a silver bullet. Journal of Marketing theory and Practice, 19(2), 139-152. Recuperado de https://doi.org/10.2753/MTP1069-6679190202
    » https://doi.org/10.2753/MTP1069-6679190202
  • Hair, J. F. Jr., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2017). A primer on partial least squares structural equation modeling (PLS-SEM) London, UK: Sage.
  • Hair, J. F. Jr, Hult, G. T. M., Ringle, C. M., Sarstedt, M., Danks, N. P., & Ray, S.. (2021). Partial least squares structural equation modeling (PLS-SEM) using R: a workbook Recuperado de https://doi.org/10.1007/978-3-030-80519-7
    » https://doi.org/10.1007/978-3-030-80519-7
  • Henseler, J., Ringle, C. M ., & Sinkovics, R. R. (2009). The use of partial least squares path modeling in international marketing. In J. Henseler, C. M. Ringle, & R. R. Sinkovics (Eds.), New challenges to international marketing Bingley, UK: Emerald Group Publishing Limited.
  • Hulland, J. (1999). Use of partial least squares (PLS) in Strategic management research: a review of four recent studies. Strategic Management Journal, 20(2), 195-204. Recuperado de http://www.jstor.org/stable/3094025
  • Kline, R. B. (2016). Principles and practice of structural equation modeling New York, NY: Guilford Publications.
  • Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 22(140), 1-55.
  • Michener, G., & Bersch, K. (2013). Identifying transparency. Information Polity, 18(3), 233-242. Recuperado de https://doi.org/10.3233/IP-130299
    » https://doi.org/10.3233/IP-130299
  • Ringle, C. M ., Wende, S., & Becker, J. M. (2022). SmartPLS 4 Oststeinbek, Germany: SmartPLS GmbH. Recuperado de http://www.smartpls.com
  • Salgado, R. J. S. F., Santos, L. F., Resende, T. C., & Souza, W. J. (2019). Cidadania deliberativa e gestão social: revisão sistemática de literatura no Brasil. Cadernos EBAPE.BR, 17(Especial), 817-831. Recuperado de https://doi.org/10.1590/1679-395176139
    » https://doi.org/10.1590/1679-395176139
  • Tenório, F. G. (1998). Gestão social: uma perspectiva conceitual. Revista de Administração Pública, 32(5), 7-23. Recuperado de https://periodicos.fgv.br/rap/article/view/7754
    » https://periodicos.fgv.br/rap/article/view/7754
  • Tenório, F. G. (2006). A trajetória do programa de estudos em gestão social (Pegs). Revista de Administração Pública, 40(6), 1145-1162. Recuperado de https://doi.org/10.1590/S0034-76122006000600011
    » https://doi.org/10.1590/S0034-76122006000600011
  • Tenório, F. G. (2007). Cidadania e desenvolvimento local Ijuí, RS: Editora Unijuí.
  • Tenório, F. G. (2008). Um espectro ronda o terceiro setor: o espectro do mercado Ijuí, RS: Editora Unijuí .
  • Tenório, F. G. (2012). Cidadania e desenvolvimento local: critérios de análise Rio de Janeiro, RJ: FGV.
  • Tenório, F. G. (2016). Cidadania, território e atores sociais Rio de Janeiro, RJ: FGV .
  • Tenório, F. G., & Kronemberger, T. S. (2016). Gestão social e conselhos gestores Rio de Janeiro, RJ: FGV .
  • Tenório, F. G., & Rozenberg, J. E. (1997). Gestão pública e cidadania: metodologias participativas em ação. Revista de Administração Pública, 31(4), 101-125. Recuperado de https://periodicos.fgv.br/rap/article/view/7882
    » https://periodicos.fgv.br/rap/article/view/7882
  • Tenório, F. G., Villela, L. E., Dias, A. F., Gurjão, F. V., Porto, E. C., & Viana, B. (2008). Critérios para a avaliação de processos decisórios participativos deliberativos na implementação de políticas públicas. In: Anais do Encontro de Administração Pública e Governança, Salvador, BA.
  • Villela, L. A. (2012). Escopo metodológico. In F. G. Tenório (Org.), Cidadania e desenvolvimento local: critérios de análise Rio de Janeiro, RJ: FGV .
  • DATA AVAILABILITY

    The entire dataset supporting the results of this study is available upon request to the corresponding author. The link to the application form will be emailed to interested parties.

Reviewers:

  • 16
    Fernando Manuel Rocha da Cruz (Universidade Federal do Pará, Abaetetuba / PA - Brazil) https://orcid.org/0000-0002-1254-5601
  • 17
    José Roberto Pereira (Universidade Federal de Lavras, Lavras / MG - Brazil) https://orcid.org/0000-0003-1570-2016
  • 18
    Miguel Pacífico Filho (Universidade Federal do Tocantins, Araguaína / TO - Brazil) https://orcid.org/0000-0002-0316-2326
  • 19
    Rosinha Machado Carrion (Universidade Federal do Rio Grande do Sul, Porto Alegre / RS - Brazil)
  • 20
    Tamiris Cristhina Resende (Secretaria de Planejamento, Orçamento e Gestão-Prefeitura de Belo Horizonte, Belo Horizonte / MG - Brazil) https://orcid.org/0000-0002-7354-2658
  • 21
    One of the reviewers did not authorize the disclosure of their identity.
  • Reviewer

    Peer review report: the peer review report is available at this URL. https://periodicos.fgv.br/rap/article/view/89896/84345
  • [Translated version] Note: All quotes in English translated by this article’s translator.

Edited by

Editor-in-chief: Alketa Peci (Fundação Getulio Vargas, Rio de Janeiro / RJ - Brazil) https://orcid.org/0000-0002-0488-1744
Associate editor: Gabriela Spanghero Lotta (Fundação Getulio Vargas, São Paulo / SP - Brazil) https://orcid.org/0000-0003-2801-1628

Data availability

The entire dataset supporting the results of this study is available upon request to the corresponding author. The link to the application form will be emailed to interested parties.

Publication Dates

  • Publication in this collection
    04 Dec 2023
  • Date of issue
    2023

History

  • Received
    02 Jan 2023
  • Accepted
    13 July 2023
Fundação Getulio Vargas Fundaçãoo Getulio Vargas, Rua Jornalista Orlando Dantas, 30, CEP: 22231-010 / Rio de Janeiro-RJ Brasil, Tel.: +55 (21) 3083-2731 - Rio de Janeiro - RJ - Brazil
E-mail: rap@fgv.br