Monitoring and evaluating actions implemented to confront AIDS in Brazil: civil society's participation

Abstracts

A Declaração de Compromisso sobre o HIV/Aids das Nações Unidas recomenda que os governos realizem análises periódicas das suas ações frente à epidemia do HIV/Aids, com a participação da sociedade civil. Para isso, devem ser criados mecanismos e instrumentos específicos. O presente trabalho examina algumas das respostas do governo brasileiro a esta recomendação. Foi feita uma análise da proposta de seguimento contida na Declaração e sua adequação à realidade brasileira, em relação à participação da sociedade civil. Discutiram-se os limites e as potencialidades do MONITORAIDS, matriz de indicadores construída pelo Programa Nacional de DST/Aids para monitoramento da epidemia. Os resultados mostraram que a complexidade do MONITORAIDS dificulta sua utilização pelo conjunto de atores envolvidos na luta contra a Aids. Sugere-se que se estabeleçam mecanismos que facilitem a apropriação desse sistema por todos aqueles comprometidos com o enfrentamento da epidemia no País.

Síndrome de imunodeficiência adquirida; Avaliação de programas; Programas nacionais de saúde; Participação comunitária; Organização comunitária

ORIGINAL ARTICLES

Monitoring and evaluating actions implemented to confront AIDS in Brazil: civil society's participation

Wilza Villela^I; José Carlos Veloso^II

^IDepartamento de Medicina Preventiva. Escola Paulista de Medicina. Universidade Federal de São Paulo. São Paulo, SP, Brasil

^IIGrupo de Apoio a Prevenção da Aids de São Paulo. São Paulo, SP, Brasil

^{Correspondence}

ABSTRACT

The United Nations Declaration of Commitment on HIV/AIDS recommends that governments conduct periodic analysis of actions undertaken in confronting the HIV/AIDS epidemic that involve civil society's participation. Specific instruments and mechanisms should be created towards this end. This paper examines some of the responses of the Brazilian government to this recommendation. Analysis contemplates the Declaration's proposals as to civil society's participation in monitoring and evaluating such actions and their adequacy with respect to Brazilian reality. The limitations and potentials of MONITORAIDS, the matrix of indicators created by Brazil's Programa Nacional de DST/AIDS [National Program for STD/AIDS ] to monitor the epidemic are discussed. Results indicate that MONITORAIDS's complexity hampers its use by the conjunction of actors involved in the struggle against AIDS. The establishment of mechanisms that facilitate the appropriation of this system by all those committed to confronting the epidemic in Brazil is suggested.

Keywords: Acquired immunodeficiency syndrome, prevention & control. Program evaluation. National health programs, organization & administration. Consumer participation. Communitarian organization.

INTRODUCTION

The United Nations Declaration of Commitment on HIV/AIDS, which resulted from the United Nations General Assembly. Special Session on HIV/AIDS (UNGASS), identifies, in its last chapter, the need to follow up actions undertaken by governments in confronting the epidemic. "Periodic analysis of the progress achieved in the implementation of these commitments, conducted with the participation of civil society, in particular, with the participation of people living with HIV/AIDS, vulnerable groups and with those providing services", are suggested. The creation of "adequate mechanisms of observation and evaluation, which make it possible to measure and evaluate progress", are proposed as well as "appropriate instruments of observation and evaluation, with adequate epidemiological data" and "supervisory systems for the protection of human rights of people living with HIV".¹

The purpose of follow up actions is to "identify the problems and obstacles that emerged when undertaking commitments". The text also recommends "the widespread diffusion of results".¹

IN 2002, a Joint United Nations Programme on HIV/AIDS (UNAIDS) consolidated a list of indicators for governments to monitor actions.² Although the Declaration contains broad target, such as "accelerating the execution of national strategies for the eradication of poverty" or "developing political strategies and programs related to the factors that make people vulnerable", the indicators are specific for results with respect to the epidemic. These indicators are associated in four categories: national commitments and actions; knowledge and individual behavior; impact; and global commitments and actions.

During the same year (2002) the Programa Nacional de DST/AIDS (PN-DST/AIDS) [National Program for DST/AIDS] developed the Plano Nacional de Avaliação (PNA) [National Evaluation Plan] seeking to establish the foundations for a continual process of monitoring and evaluation of its actions. The matrix of the PNA is organized around the dimensions of monitoring and evaluation. Indicators of impact and results are defined for dimensions.

In 2005, the Plano Estratégico do Programa Nacional de DST/AIDS [Strategic Plan of the DST/AIDS National Program] assumed that one of its directives would be "the creation of an effective capacity for monitoring and evaluating", seeking to improve of the program's management. Consequentially, the Política Nacional de Monitoramento e Avaliação [Monitoring and Evaluation National Policy] was formulated, incorporating and updating the PNA. UNAIDS's system of indicators was also revised in 2005.³ According to this revision, it was recommended that data be disagregated according to sex and specific indicators be defined in function of the prevalence of the epidemic in different countries. Mechanisms of establishing civil society's participation were more detailed, including the production of qualitative and quantitative data and access to the national plans for collecting data.

PROPOSAL FOR FOLLOWING UP THE DECLARATION OF COMMITMENTS

The follow up activities set forth in the Declaration are periodic analyses, observation, evaluation and supervision. While recognizing the existing lack of consensus with respect to the scope of each of these activities, it may be acknowledged that, by and large, there is a certain continuum: periodic analyses would provide the necessary information to respond whether or not what was planned was in fact executed (monitoring); evaluation would provide responses as to whether what was planned and executed produced the desired results; and, supervision would attempt to resolve the discrepancies between what was planned and executed on the one hand, and, on the other, the results foreseen and those attained.

Considering these four phases, it is possible to perceive that following up the activities extrapolates the observation of the behavior of an indicator during a limited period of time. Observed variations should be interpreted vis-à-vis the actions proposed, those that were effectively carried out, the objectives set forth when the actions were proposed and the circumstances that interfered, whether positively or negatively, in attaining those results. This analytic and interpretive effort should be undertaken, for each of the 64 targets set forth in the Declaration, in partnership with civil society and people living with HIV/AIDS. The follow up proposal is, consequentially, a broad and complex target.

ADAPTING THE UNGASS PROPOSAL TO THE BRAZILIAN CONTEXT

The Declaration of Commitment was elaborated in midst of an intense and legitimate concern with the negative impacts that the epidemic has provoked in the African continent, and that it may provoke in Asia and Eastern Europe, where it is more recent, if viable, alternative means of treatment are not found. Thus, the focus of the Declaration is on countries or regions where there is a high prevalence of HIV or where the epidemic is in a rapid phase of expansion and that do not rely on a structured health system. In Brazil, the 0.6% prevalence is much lower than that of some African countries, such as Botswana, with a 40% prevalence.⁴ The epidemic has reached a phase of stabilization in Brazil, where there is access to anti-retroviral treatment, and a long history of partnership with civil society.

While acknowledging the pertinence of the Declaration for countries devasted by the epidemic, its directives should be adapted to Brazilian reality.

The pioneering character of Brazilian DST/AIDS policies is such that some of the principles that guide the Declaration of Commitments have already been partially or totally contemplated. This is the case, for example, in relation to the proposal of intersectoral collaboration that is already contemplated in the partnerships between the PN-DST/AIDS and other sectors of the Ministry of Health, between this program and other Ministries, such as the Ministry of Education, and other sectors, such as private enterprise, by means of the Business Council. The same is also true with respect to the integration between prevention and assistance, one of the characteristics of the PN-DST/AIDS since its creation, as is the participation of civil society in the definition and accompaniment of its actions.

Thus, the recommendation as to the "participation of civil society", present in the Declaration, seems to be based on realities distinct from the Brazilian reality, where it is still necessary to remember that the greater the number of people interested in this issue are directly involved, the easier it is to find a solution.

The engagement of Brazilian civil society in the struggle against AIDS does not mean that the dialogue between the State and the AIDS movement occurs free of tensions. On the contrary, this is a live, intense process, that is continually reconfigurating itself, with a constant flow of leaders from civil society to the PN-DST/AIDS and vice versa.

The characters denominated in the Declaration of Commitment as "civil society" – people living with HIV/AIDS, higher risk groups and health care providers – do not express the entire set of agents that build the social arch surrounding the struggle against AIDS in Brazil, which relies on several other actors, including those who produce knowledge on the subject. The latter should be specifically invited to participate in the monitoring actions, due to their potential contribution in the interpretation and analyses of data, redrafting of proposed actions and dissemination of the analysis and results.

At the same time, the utilization of fixed categories, such as "people living with HIV" and "groups living under greater risk", in order to define the subjects that should participate in follow up actions, requires caution. Doubtless, these subjects have a priviledged knowledge on habits, behaviors and language of their community. Their contribution may be fundamental in drawing up the strategies of prevention and care of their respective groups, as well as in the analysis of data concerning the accomplishments and shortcomings or obstacles arising from the implementation of strategies. However, the incorporation of these capacities in tasks that require the command of specific technical abilities, such as the follow up program proposed by UNGASS, is a challenge. Furthermore, the people that compose the groups defined in the Declaration as "civil society" also identify themselves in terms of social class, gender, ethnic and professional groups that also mark their experience of living with HIV/AIDS or belonging to some group under greater social exclusion. Thus, this experience may not always suffice to leave specific marks on policies, even if the subject was involved with their formulation and/or implementation.

The participation of people that are active in non-governmental organizations in different government instances or forums has been constantly problematised in the places where civil society organizes itself. Questions are raised as to which mechanisms may be utilized by people invited to compose a mixed environment (government/society) in order to assure that their opinions and perspectives in fact express the desire or reality of their reference group. Or still yet, up to what measure the opinions expressed in these forums are effectively brought to bare.³

PARTICIPATION OF CIVIL SOCIETY IN MONITORING ACTIONS

Monitoring their actions is among government's responsibilities, being relatively easy to define the actions that should be monitored and the method to be utilized towards this end: a priority among the actions to be undertaken is established, and in thesis, the same sequence is followed in the monitoring process. Definition of methods and the respective techniques utilized in collecting data is facilitated by governmental organ's access to existing data systems and due to the possibility, when these are not sufficient, of producing the necessary information. Although the culture of monitoring and evaluating health is relatively recent in Brazil, and the systematic production of information is not a practice that has been incorporated by health services, it is possible to establish modes and criteria for accompanying the HIV/AIDS epidemic. These modes and criteria may be the point of departure for evaluating the pertinence and efficacy of actions implemented. In fact, since the beginning of the epidemic, effort has been made, on the part of the PN-DST/AIDS, to systematize and disseminate, by means of its Boletim Epidemiológico [Epidemiological Bulletin], the information related to mortality due to this cause and the notification of new cases of AIDS.

Monitoring is a very complex procedure for civil society. Producing quantitative data of a populational character in order to accompany the behavior of the epidemic independently does not make sense nor is it possible. Thus, one should rely on data produced by the government to accompany actions that this same government defined in response to the problems that it identified. Many of these actions can only be accompanied by means of data provided by the government. Consequently, an effort must be made so that civil society does not become a mere spectator of the process. Apparently, the alternative proposed by UNGASS, is an effective partnership, in which government and society define in conjunction, similar levels of expertise, interest and commitment as well as the contents and strategies to be utilized to keep track of the epidemic. Due to differences in these actors' political perspectives, as well as differences as to their familiarity in handling the data and in their individual and institutional disposal for the task at hand, this alternative is not realistic. For this reason, in 2006, UNAIDS began to suggest that civil society develop its own processes of monitoring, including the production of qualitative data, that would make it possible to delve more deeply in the analysis of quantifiable data.⁵

The difficulty arising when civil society attempts to monitor public policies is not only operational or political, it is also conceptual. Often the idea of monitoring is confounded with that of social control. Although it is not possible to conduct any form of social control without some kind of monitoring – for it is this latter process that will supply the acts of social control with their content – these actions are distinct in their dynamics and in their purposes. One example of how civil society interlaces monitoring and social control is the way it responded to the shortage in medication supplies that occurred in 2004. Activists from different regions of the country developed and applied a questionnaire in order to diagnose the quality of assistance among HIV positive individuals, in an organized manner. It was completed in only four months. The results were disclosed to the public in mobilizations conducted simultaneously in all the municipalities involved in the initiative, unleashing actions of social control. Beyond the merits and political interests of this initiative, the fact that the diagnosis utilized was an instrument prepared by people living with HIV and who rely on the Sistema Único de Saúde (SUS) [Brazilian Unified Health System], permitted their experience to be incorporated in the definition of criteria related to the quality of assistance and satisfaction among individuals utilizing SUS. In this manner, knowledge was appropriated from people living with HIV for effective action in monitoring and social control.

SYSTEM FOR MONITORING INDICATORS

The PN-DST/AIDS's System for Monitoring Indicators (MONITORAIDS) was created with the "objective of providing partners and civil society as a whole, useful information that makes it possible to accompany the Brazilian response towards the control of AIDS and other STDs". It aggregates indicators considered relevant for monitoring the HIV/AIDS epidemic as well as other STDs; which are useful in evaluating the programmatic actions of the PN-DST/AIDS as well as being suggestive for future evaluations. The instrument is considered the "most relevant structural component of the Plano Nacional de Avaliação [National Evaluation Plan]".

MONITORAIDS is composed of indicators that attend the demands of UNGASS, the World Bank and the PN-DST/AIDS itself. It is comprised of a set of socioeconomic, health indicators related to the HIV/AIDS epidemic, originating from different databases and specific research. There is an index card for each indicator, in which its definition, the method utilized for calculating it and its limitations are explicited. It is foreseen that the system will be updated and re-edited each year. MONITORAIDS has been available on the internet since 2005.⁶

The result of exhaustive work, MONITORAIDS presents the same problems that any system of this stature tends to present, considering that Brazil is a large, populous country that has profound social inequalities.

The reports are presented in a national and regional sphere. Indeed, it is up to the federal manager to present nationally aggregated data, or regionally disaggregated data, since important regional differences in terms of access and service coverage, population morbimortality rates and living conditions are well-known.² Regional differences and inequalities also result in disparities in coverage and quality of information, and specific methodological artifacts are necessary to correct the data.¹ Aggregation of data, whether on a national, regional or state level, by homogenizing distinct realities, also produces a result that is quite distant from any reality. The utility of this information for purposes of local and immediate planning of health actions, including AIDS, is thus compromised. In this sense, expliciting the limitations of each MONITORAIDS indicator does not reduce the necessity of conducting more specific studies, when the intention is to plan, monitor and evaluate local actions.

Information issuing from SUS is not based on populational studies. Some of the data systems are structured and supplied information that is directed towards the payment of procedures, this is the case, for example, of the Sistema de Informações Ambulatórias (SIA) [Outpatient Information System], the Sistema de Informações Hospitalares (SIH) [Hospital Information System], the Sistema de Informações sobre o Pré-natal (SIS pré-Natal) [Pre-natal Care Information System], and others. Thus, there is a margin of error when this data is utilized for other purposes, since efforts may be made to adjust the procedures undertaken to those listed in the payment rosters and other distortions. Furthermore, the systems for payment of procedures do not indicate how many people where in fact beneficiaries. In this way, the limitations of these sources of information may be greater than indicated in MONITORAIDS, when it states that data originating from SUS does not cover the entire population and all health procedures. Finally, creating compatibility with respect to information that originates from distinct sources is not always possible, since the fields covered and the methodologies utilized in collecting data may be different.

The databases related to health assistance and administration have the above mentioned limitations; the systems related to the epidemiological profile, such as the Sistema de Informações em Mortalidade (SIM)[Mortality InformationSystem], the Sistema de Informações em Agravos Notificáveis (SINAN) [Notifiable Disorders Information System], and the Sistema de Informações sobre Nascidos Vivos (SINASC) [Live Births Information System], present problems with respect to coverage. For example, the national average for the underregistration of deaths is estimated at 20% and reaches a 40% average in Northern and Northeastern regions. Approximately 15% of the death declarations within the SIM database do not present the basic cause of death; as to SINASC, its estimated converage in the Northern and Northeastern regions is 75%; and the SINAN has not been implanted in all Brazilian municipalities.⁷

Some data derives from research whose replication or preriodicity is not guaranteed. This is the case, for example, of the Pesquisa Nacional por Amostragem de Domicílios (PNAD) [National Survey Based on a Residential Sample], that is irregular in terms of its periodicity and presents a specific content at each edition; and the Pesquisa Nacional de Demografia e Saúde [National Survey on Health and Demography]. Even the Demographic Census has occasionally presented problems of periodicity. However, despite some delays, these surveys do end up occurring. Guaranteeing periodicity is more troublesome with respect to the evaluation studies and surveys conducted by PN-DST/AIDS that depend on specific resources, political will power and available funds both within the Program and the Ministry of Health. Furthermore, it should be stressed that these studies are the only ones that provide information concerning the characteristics and behavior of certain populations, being essential, therefore, for planning actions.

CONCLUSIONS

Reflection concerning the monitoring of government actions in the struggle against AIDS presupposes that this is a political process, although it relies on technical instruments for this purpose.

However, it is possible to approach the process of monitoring from a technical perspective, attempting to reduce its political dimension. This seems to be the Brazilian government's option up to the present.

As proposed by UNGASS, instruments and mechanisms were created in order to conduct periodic analyses that make it possible to evaluate progress and identify obstacles. The effective incorporation of distinct social actors in this task, however, needs to be further developed. This includes building strategies for further appropriation and utilization of MONITORAIDS, as well as the establishment of mechanisms to facilitate the process of production and circulation of information by civil society. It is also necessary to stimulate the capacity for analysis and critique of information by the broad set of activists committed to the struggle against the epidemic. The more consolidated the partnership between civil society and government in accompanying the epidemic, easier it will become to conduct effective actions for its control.

REFERENCES

Received: 29/3/2006.

Presented during the "Monitoring and Assessment Seminar on Target and Commitment Fulfillment relating to the United Nations Declaration of Commitment on HIV and AIDS". São Paulo, November 21^st and 22^nd, 2005.

• 1. Duarte EC, Schneider MC, Paes-Sousa R, Ramalho WM, Vasconcelos Sardinha LM, Silva Júnior JB et al. Epidemiologia das desigualdades em saúde no Brasil: um estudo exploratório. Brasília (DF): Organização Pan-Americana da Saúde; 2002.
• 2. Nunes A, Santos JRS, Barata RB, Vianna SM. Medindo as desigualdades em saúde no Brasil: uma proposta de monitoramento. Brasília (DF): Organização Pan-Americana da Saúde, Instituto de Pesquisa Econômica Aplicada; 2001.
• 3. Villela W. Das interfaces entre os níveis governamentais e a sociedade civil. In: Parker R, Galvão J, Bessa M, organizadores. Saúde, desenvolvimento e política. São Paulo: ABIA Editora; 1999. p. 34.

1

Organização das Nações Unidas, Centro de Informação das Nações Unidas em Portugal. Declaração de compromisso sobre o VIH/SIDA. "Crise Mundial – Resposta Mundial". Disponível em

http://www.runic-europe.org/portuguese/ecosoc/AIDS/CompromissoVIHSIDA.pdf [acesso em 31 mar 2006]

2

UNAIDS. Guidelines on Construction of Core Indicators. Disponível em

http://www.who.int/hiv/strategic/me/en/isbn9291732389.pdf [acesso em 2006 Mar 31]

3

UNAIDS. Guidelines on Construction of Core Indicators. Disponível em

http://data.unaids.org/publications/irc-pub06/jc1126 -constrcoreindic-ungass_en.pdf [acesso em 7 abr 2006]

4

UNAIDS/WHO. AIDS epidemic update: December 2005. Disponível em

http://www.unaids.org/epi/2005/doc/report_pdf.asp [acesso em 31 mar 2006]

5

UNAIDS. Strengthening the response to HIV and AIDS at local, country and global level. The role and opportunities for civil society in tracking progress towards ungass targets. Report on a joint UNAIDS and civil society planning meeting on UNGASS 2006 preparation; 1 November 2005. Disponível em

http://data.unaids.org/Publications/IRC-pub06/Recife_Report.pdf. [acesso em 31 mar 2006]

6

Ministério da Saude. Programa Nacional de DST/Aids. Sistema de Monitoramento de Indicadores do Programa Nacional de DST/Aids - MONITORAIDS. Disponível em

http://www.aids.gov.br/monitoraids [acesso em 10 abr 2006]

7

Rede Interagencial de Informações em Saúde - RIPSA. Indicadores básicos de saúde no Brasil: conceitos e aplicações. Rede Interagencial de Informações para a Saúde - Ripsa. Brasília (DF): Organização Pan-Americana da Saúde; 2002. p. 267. Disponível em

http://www.opas.org.br/sistema/arquivos/matriz.pdf [acesso em 31 mar 2006]

Publication Dates

• Publication in this collection
  14 Sept 2006
• Date of issue
  Apr 2006

History

• Received
  29 Mar 2006