EVALUATION AND PROCESSES OF SUBJECTIVATION AT THE PRIMARY HEALTH CARE: EVALUATION AND SUBJECTIVATION

ABSTRACT

This study aimed to investigate the effects of evaluation practices in the processes of subjectivation experienced by professionals from Family Health Teams in Primary Health Care, in a health district in the municipality of Belo Horizonte, state of Minas Gerais. We also sought to understand the effects of evaluation on the subjectivity of the professionals involved. We conducted 16 semi-structured interviews with professionals and managers of two Basic Health Units. A thematic analysis of the interviews was carried out in three axes: the gains triggered by the evaluation practices, their problematic aspects and the subjectivation processes resulting from the assessments. The gains that the evaluations allowed were: detecting work failures and redirecting activities; improve service to the population and change behavior with coworkers and in relation to work. Criticism was directed towards the instruments’ indicators, the evaluation processes, the difficulty of putting into practice the results obtained and the non-compliance with the agreements by the municipal managers. The subjectivation processes promoted by the evaluations were, on the one hand, the self-reflection in relation to work and coworkers, producing changes in attitudes, and, on the other, the feelings of fear, guilt and frustration regarding evaluation process, its results and consequences.

Keywords:
Process evaluation (health care); subjectivity; primary health care

RESUMO

Desde a redemocratização no Brasil, o tema da subjetividade tem sido um componente importante na discussão e na formulação das políticas públicas no país. Existe um grande embate na literatura em torno dos méritos e dos prejuízos decorrentes dos processos de avaliação. O objetivo desta pesquisa é investigar os efeitos das práticas avaliativas nos processos de subjetivação vividos por profissionais das equipes de Saúde da Família na Atenção Básica, em um distrito sanitário do município de Belo Horizonte. Buscamos também compreender os efeitos da avaliação na subjetividade dos profissionais envolvidos. Realizamos 16 entrevistas semiestruturadas com os profissionais e gestores de duas Unidades Básicas de Saúde. Fizemos uma análise temática das entrevistas em três eixos: os ganhos desencadeados pelas práticas de avaliação, os seus aspectos problemáticos e os processos de subjetivação decorrentes das avaliações. Os ganhos que as avaliações permitiram foram: detectar as falhas no trabalho e redirecionar as atividades; melhorar o atendimento à população e modificar o comportamento com os colegas e em relação ao trabalho. As críticas se direcionaram aos indicadores dos instrumentos, aos processos de avaliação, à dificuldade de colocar em prática os resultados obtidos e ao não cumprimento dos acordos por parte dos gestores municipais. Os processos de subjetivação promovidos pelas avaliações foram por um lado a reflexão em relação ao trabalho e aos colegas, produzindo mudanças de atitudes, e por outro os sentimentos de medo, culpa e frustração em relação ao processo de avaliação, seus resultados e consequências.

Palavras-chave:
Avaliação de processos (cuidados de saúde); subjetividade; atenção primária à saúde

RESUMEN

Hay un embate en la literatura en torno a los méritos y los perjuicios derivados de los procesos de evaluación. Esta pesquisa investiga los efectos de las prácticas evaluativas en los procesos de subjetivación vividos por los profesionales de los Equipos de Salud de la Familia en la Atención Básica, en un distrito sanitario del municipio de Belo Horizonte. También buscamos comprender los efectos de la evaluación sobre la subjetividad de los profesionales involucrados. Realizamos 16 entrevistas semiestructuradas con los profesionales y gestores de dos Unidades Básicas de Salud. Hicimos un análisis temático de las entrevistas en tres ejes: las ganancias desencadenadas por las prácticas de evaluación, sus aspectos problemáticos y los procesos de subjetivación derivados de las evaluaciones. Las ganancias fueron: detectar los fallos en el trabajo y reorientar las actividades; mejorar la atención a la población y modificar el comportamiento con los colegas y con relación al trabajo. Las críticas se dirigieron a los indicadores de los instrumentos, a los procesos de evaluación, a la dificultad de poner en práctica los resultados obtenidos y al incumplimiento de los acuerdos por parte de los gestores municipales. Los procesos de subjetivación promovidos por las evaluaciones fueron, por un lado, la reflexión con relación al trabajo y a los colegas, lo que produjo cambios de actitudes, y, por otro lado, los sentimientos de miedo, culpa y frustración con relación al proceso de evaluación, sus resultados y consecuencias.

Palabras clave:
Evaluación de proceso (atención de salud); subjetividad; atención primária de salud

Introduction

The theme of evaluation in public health emerged, in an incipient way, in Brazil during the 1990s, but without its necessary institutionalization, which only began in the 2000s (Carvalho & Shimuzu, 2017). Comparing it with the education sector, in 1988 there was already the Basic Education Evaluation System and, in 1993, the Institutional Assessment Program for Brazilian Universities (Felix, Bertolin, & Polidori, 2017). However, in the context of the public machine, the health sector welcomed better the proposal, both managers and workers (Furtado & Vieira-da-Silva, 2014), than the education sector. Pacheco (2010) considers that the health sector in Brazil is better prepared to deal with evaluation than the resistant education sector, due to the logic of remuneration per procedure adopted by the SUS, which would have opened “[…] the way for the measurement of services provided” (Pacheco, 2010, p. 206). On the other hand, literature in the humanities and social sciences, in general, is much more skeptical and reactive to evaluation proposals than the Brazilian health sector (Ferreira Neto, 2017).

Debate in the literature around the merits and losses from evaluation processes, between authors in favor and against, there is a common point, considering the evaluation as one of the instruments of government or management (Menicken, & Miller, 2012; Pinto, 2014; Martuccelli, 2015; Triantafillou, 2017). In other words, it should not be understood as an isolated action that seeks exclusively to measure, but has a close link with the management of policies, programs, services and people. It is also a ‘government of men’ (Martucceli, 2015), of an action that aims to produce ‘calculable selves’ (Menicken & Miller, 2012), of changes in subjectivity.

A dissonant voice in the current critical literature is Deirdre Duffy (2017), who considers the current trend to reduce evaluation to mere problematic managerialism. According to her, this tendency fails to recognize that evaluation, even though it is not a perfect method, has the merit of emphasizing “[…] reflection on the present, to open up a discussion of the future” (Duffy, 2017, p. 2). In this research, we share her interest in the critical and creative possibilities of evaluation practices, even recognizing and highlighting their numerous problems.

The beginning of the debate on health evaluation in Brazil took place via induction by an international organization. In 1996, the SUS Reinforcement and Reorganization program (Reforsus) received transfers in the amount of 650 million dollars from the World Bank, which demanded the use of the evaluation “[…] both for the release of financing and for the evaluation of the impact of the projects” (Furtado & Vieira-da-Silva, 2014, p. 2647).

In 2003, the Ministry of Health (Ministério da Saúde - MS) developed actions to implement the Monitoring and Evaluation of Primary Health Care, “with the objective of institutionalizing the evaluation” in SUS (Felisberto, Freeze, Bezerra, Alves, & Samico, 2010, p. 1080), thus transforming it into a national policy. The institutionalization of evaluation is the main guideline of this effort, understood as the integration of evaluation into an organizational system, permanently associating it with management actions and work processes. The objective is to foster an evaluation culture that involves all subjects, from local workers to federal managers.

The two most consolidated evaluation instruments in the municipality where the research took place were: 1) National Program for Improving Access and Quality of Primary Health Care (Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica - PMAQ-AB), and 2) Server Performance Evaluation (Avaliação do Desempenho- AD).

The PMAQ aims to “[…] induce the expansion of access and the improvement of the quality of primary health care, guaranteeing a comparable quality standard nationally, regionally and locally, in order to allow greater transparency and effectiveness of government actions directed to Primary Health Care” (Saúde mais Perto de Você, 2012, p. 7).

It is applied in two-year cycles and program membership is voluntary. If contracted, the municipality receives 20% Primary Health Care Minimum Wage Variable Component per participating team. The rest is transferred depending on the team performance. The idea is to use the instrument associated with AB funding. The PMAQ has become an important instrument for financing decisions since its creation in 2011. Through it, the Ministry of Health has sought to implement a management model that values evaluation more and “[…] uses it for the planning, qualification and financing of its actions” (Pinto, 2014, p. 204). It consists of seven indicators: women health; child health; diabetes and hypertension control; oral health; general production; tuberculosis and leprosy; and mental health.

The Performance Evaluation (PE) is an instrument applied to all Belo Horizonte City Hall employees (Prefeitura de Belo Horizonte - PBH). It consists of 10 indicators, 8 from the common core (knowledge, organization, commitment, initiative, flexibility, interpersonal relationships, zeal for the patrimony and punctuality) and 2 from the health area (relationship with users and teamwork). Every year employees are evaluated by managers, peers and self-evaluated. The result is decisive for career progression, obtained by employees who have an average of at least 8 in the three-year period.

Since the redemocratization in Brazil, the subjectivity theme has been an important component in the discussion and formulation of public policies in the country. In most of the mentions, subjectivity is emphasized in the protagonist position of subjects who contribute to the production of health, whether they are managers, workers or users. One of the authors who influenced this debate was Michel Foucault, since his visits to the country in the 1970s, when he spoke intensely with health workers and mental health professionals. Based on it, subjectivity is taken as procedurally produced by practices of coercion and/or freedom from “[…] a certain number of rules, styles, conventions […]” present in culture (Foucault, 2004, p. 291). Foucault’s production, from 1977 onwards, began to problematize power relations as being not only derived from coercive practices, but also from more autonomous practices of freedom, taking the subjectivity theme as its ultimate research axis.

The objective of this study was to investigate the effects of evaluation practices on the subjectivation processes experienced by workers of the Family Health Teams (Equipes de Saúde da Família - EqSP) in Primary Health Care, in a health district in the municipality of Belo Horizonte. For this, we sought to know aspects of the evaluation instruments considered by workers as gains, and those considered losses, as well as to analyze the effects of subjectivation on professionals.

Method

We conducted 16 semi-structured interviews in two Basic Health Units (BHU) in the Barreiro health district, between August 2017 and March 2018. The script addressed topics such as the health assessment culture, which evaluation instruments they knew, aspects positive and negative effects of each instrument. The criterion for choosing the units was the presence of EqSF who had been working in the units for a longer time, with a lower turnover of professionals. The interviews took place with eight nurses, two doctors, two nursing technicians, two community health agents (CHA) and two managers. Nurses from all teams were interviewed as they were in charge for the main aspects of managing the work of their teams, including the handling of data related to the National Program for Improving Access and Quality of Primary Health Care - PMAQ-AB. Therefore, the nurses interviewed were the ones who knew the most about the PMAQ-AB, followed by the physicians. The CHAs knew little about it and the nursing technicians only recognized its existence, as they did not participate in the evaluation cycles. As PE was applied individually, it was well known by everyone. BHU A had five teams and B, three teams. We also interviewed two members of the local health committees, both of whom had completed high school. However, both were completely unaware of the ongoing evaluations, even though they understood their operation in their professional experience, and were therefore discarded.

Respondents were named according to profession, interview order and BHU where they worked. For the profession, we used the letter ‘E’ for nurses, ‘A’ for community health agents, ‘M’ for physicians, ‘T’ for nursing technicians and ‘G’ for managers. To identify the BHU we use ‘A’ to refer to the first one we visited and ‘B’ to the second. For example, E1B is the first nurse interviewed at the second UBS.

We carried out a deductive thematic analysis (Souza, 2019), based on the literature and on the research objectives: the gains resulting from the evaluation practices, their problematic aspects and the subjectivation processes resulting from the evaluations.

The research was approved by the Research Ethics Committee of the Pontifical Catholic University of Minas Gerais with CAAE: 58882516.2.0000.5137.

Gains from evaluations

In general, respondents pointed out benefits from the use of evaluation instruments both at the personal level and regarding work performance. Professional changes were related to relationships with users and coworkers.

The main characteristic highlighted was the condition of identifying what is really important at work and thus promoting changes in this regard. Almost all professionals from all categories pointed out that evaluations help redirect individual and team work. By stating that “[…] evaluating what we have been doing, the way we have been doing it to try to improve... for me... personally it is always been that, right? Because we are evaluating what we did, what we did not do... where we... failed to try to fix it and are offering” (E1B), the nurse points out that the PMAQ indicators produced new error parameters that professionals used to redirect their actions.

The same happened after PE. The workers learned the opinions of coworkers and managers who evaluated them, paying attention to certain behaviors that could be improved. For a CHA, “[…] when it is done with the correct professional, I think it is valid because then you know how you are, what you can improve, right?” (A1A).

The theme of improvement in work processes was the central point raised by the workers. Both the PMAQ and the PE helped to reorient the professional actions. According to Power (2000), audit management, of which evaluation is a part, aims to realign organizational life. In the present case, the first instrument did so through numbers and indicators, and the second based on the opinion of coworkers and managers, creating “[…] an opportune moment for reflection. To stop, to think, to really evaluate, to see what [...] ponder, try to improve, understand?” (E1A).

Regarding the gains brought by the evaluation, elements of improvement in the team work processes were pointed out. According to half of the respondents, the PMAQ brought several gains to the EqSF. One of the areas of advancement provided by the evaluation was risk stratification, improving attention to users. “The PMAQ brought this, to stratify, better see which one is most in need. [...] So the PMAQ came to help in this sense, in this sense of risk stratification” (E5A). For managers, the PMAQ made it easy to “[…] cut out within this territory what it points out, right,... so it helped to organize some actions” (G1A) and “[…] it came to assess whether the team has control over the their patients” (G1B).

Seven of the interviewees, including one of the managers, pointed out some improvement in the attention to users, as one of the nurses says: “[…] because of this we can ‘oh, I’m sinning here, I’m not giving this attention to this type of care’ we end up concentrating and strategizing” (E2A). However, it is worth noting some adjectives used by professionals that indicate a certain blaming effect present in evaluation practices, such as ‘error’ and ‘sin’.

Another nurse exemplifies the changes promoted by the PMAQ at work, such as the greater concern with the continuity of treatments and with the recording of activities. Due to the absence of the pregnant woman to the consultation, she asks the CHA for a family visit. “Then I go into the patient’s medical record and write: ‘the CHA went to the patient’s house and it was seen that she is doing prenatal care and consulted on such day’ to at least have a record (E3A).

With the risk stratification proposed in the PMAQ, the teams were able to identify severe cases that went unnoticed, creating strategies to address them. As a result, attention to users was intensified, generating active searches and recording more accurate information. One of the nurses even created her own instrument to control the monitoring of pregnant women: “I was improving it according to my knowledge... my knowledge of these evaluation indicators... so today I say that it is much better than the start” (E1B).

Even to a lesser extent, PE also generated changes in activity. It was the case of a physician who realized that when trying to help some users, attending last-minute favors, she harmed others who were scheduled. Seeing this, she changed her posture.

[…] the way so many peers evaluated me and how the management evaluated me... it gave me another perspective [...] here there is the same schedule of appointments, a patient arrives and knocks on the door and ask a favor I do. I’m not being fair to those who came at the right time and waited, waited for the day of the appointment. So, I liked this other look that was brought to me, and I’ve been exercising that with patients, you know? I even thought it was complicated or that they would think it was bad. […] now I’m negotiating with patients: ‘This is not the way’, ‘this is not the way, it has to be like this’ (M1B).

The rhetoric of negotiation, important in Primary Health Care, sometimes clashes with a more vertical and prescriptive approach to acting, as we see in the examples offered by the physician: ‘it’s not’, ‘you have to’. However, this does not invalidate the movement that the physician sought to carry out based on the evaluation of colleagues. The evaluation processes generated significant changes in user care at the two BHU. As pointed out by Abelson, Humphrey, Syrowatka, Bidonde and Judd (2018), evaluations have the potential to promote the restructuring of the public health system. The authors claim that the results of the evaluations promote more comprehensive and transparent approaches to work management.

The last benefit provided by the evaluation instruments we identified was in relation to the work team. Some nurses highlighted specific changes in this regard. The first was the possibility of a more incisive demand from the team, as pointed out by a nurse: “[…] with the PMAQ, with the indicators, we are more attentive to this, even requesting the team for some things that, over time, become automatic” (E4A). Another nurse understands that the common need between the teams could unite them to improve the indicators: “[…] here we are five teams in the same structure. If more than one team is in need of help, why not join three teams and go to the church hall and do a collective action to improve this indicator, right?” (E2A). As for PE, only one nursing technician reported an improvement in her relationship with the team because of the feedback received. She states that the assessment brought improvements in “[…] interaction because I didn’t like to interact with people [...] in this sense I observed, I had to improve my performance” (T1A).

We found, therefore, that the evaluations promote changes not only in the sense of demanding efforts to reach a new level of effectiveness, but also to enable the promotion of actions that improve the attention to the user. We follow the reasoning of Triantafillou (2017), when stating that evaluations are not restricted to measuring, but are aimed at transforming. This transformation affects not only work processes, but subjectivity itself. For the author, they produce new visibilities and normative structures that allow comparisons to be made that encourage constant organizational change and the practices that are measured.

Problematic Aspects of evaluations

The criticisms made to the evaluations happened in greater number than the speeches about the gains obtained, analyzed in the previous section. Criticism was directed towards the instruments’ indicators, the evaluation processes, the difficulty of putting into practice the results obtained and the non-compliance with the agreements by the municipal managers.

The most emphatic criticism pointed out by eleven of the interviewees, including one of the managers, is that neither of the two instruments was able to account for the reality they experience in their daily work. They complained that the PMAQ indicators were only concerned with the numbers generated by the calls. “But from what we are seeing now, there is a negative result, because I think there was really a demand simply for... numbers, and we are not having a return to the quality of care... for service” (E5A). This quantitative bent of evaluation instruments can reduce the task of assessing to a simple measurement of what can be quantified (Dujarier, 2015). For this reason, Duffy (2017, p. 136-137) understands that the evaluation should clarify the limits of the instrument it uses, “[…] not seeking to produce a definitive measurement, [...] but should seek to identify what is immeasurable or what exceeds the measurement”.

A nurse exemplified the case they accompanied of a homeless person. To be well evaluated in prenatal care, all teams needed to register the procedure six times in all pregnant women who were registered in the unit. As the pregnant woman, in this case, had no address, they were only able to provide the first assistance. “She wasn’t even from our area and we offered care to her right away, we welcomed her”. Unable to continue monitoring, they were harmed in the assessment. “I couldn’t even do an active search because I didn’t even know where she was living, because she lived on the street and if you look at the indicator, it’s bad for that”. (E3A)

Another problem was the pregnant women who made use of supplementary health. They were considered as a population to be assisted by the team, but resulted in a negative assessment for not being monitored at the unit. One of the physicians points out this situation: “In the neighborhood you have pregnant women who... have an insurance plan, they have another treatment, right? [...]. So, it enters as a negative factor in our numerator without this being within our reach.” (M1A)

With the exclusive quantitative focus, the indicators did not cover the day-to-day challenges. A nurse told that; “[...] we have a patient that we have a hard time for her coming to adhere to, so we do a great task”. And when they finally managed to access the patient, they “[…] do everything at once, that won’t generate any indicator, but we managed to do it, we had the opportunity and even if it’s not marked and everything, we got something” (E3A). There is no indicator in the PMAQ that captures difficult-to-handle cases, which in general require hard work by the team. The three situations demonstrate that the strictly numerical evaluation, by not covering the difficulties, the mishaps, the extra efforts employed by the teams, in short, the complexity involved in health care, harms the team results.

As for PE, professionals did not feel that their professional performance was properly evaluated. Its focus was more directed towards interpersonal relationships, as one of the physicians says: “Sometimes the person has good productivity, good performance, but they are not a nice person to others, right, who will evaluate” (M1A). The professional effectiveness and their efforts to improve at work were also not considered: “I think there might be a question related to what you developed differently in the unit” (E4A); “[…] something that I think should also be evaluated, which I’m here remembering... if the professional who participates in the Congress, if takes a distance learning course” (M1B).

In both evaluations, the indicators did not address the entire reality of daily work, since not all actions in the labor activity are measurable (Ferreira Neto, 2017). However, in the cases mentioned, the extra efforts were not only ignored, but also considered incomplete work.

Half of the respondents, including the two managers, felt that some of the indicators in the evaluation instruments were either difficult to understand or not very specific. According to one of the managers, the PMAQ had “[…] an excessive number of indicators [...] an evaluation criterion without the incorporation and subjectivation of the worker to understand these indicators” (G1A). The other manager pointed out that “Some kind of name that was there, sometimes very technical: ‘Do you do micro nebulization?’ As for PE, one of the nurses stated that “some [questions] that are very broad, you know? And sometimes it doesn’t fit, sometimes something that I would score, for example, lower for a coworker or for myself, then it’s so broad that you can’t” (E3A).

The problems perceived made professionals question the validity of the results of the two instruments. For one of the physicians, the PMAQ needs to be “[…] improved, so you don’t be... that you don’t have false... eh... falsely bad or falsely positive indicators” (M1A). The other physician pointed out that “these instruments [...] they are not... reliable, let’s put it this way” (M1B). We understand that a good evaluation instrument must carry with it the commitment to its continuous review and improvement. This is particularly important in the case of the PMAQ and we expect that this and other research will collaborate in this regard. The PMAQ collaborated to create new municipal evaluation initiatives, such as those already mentioned in the Territorial Care's Four-monthly Monitoring and Management, which use its data and indicators, and add new elements. In relation to PE practiced by PBH, we have more structural problems both in the instrument format and in the process of execution by peers and managers. Ultimately, the list of 10 items with a score to be assigned creates a measure of opinion, an eminently subjective indicator. Some workers also consider the questions to be broad and vague. “There are some that are very broad, you know?” (E3A)

In the PE, the evaluation by peers or coworkers who do not know each other interfered with the reliability of the results. In addition, as it is linked to career development, two thirds of respondents indicated that people do not score their coworkers below the minimum necessary, in order not to harm them. There is, in most cases, a corporate defense to guarantee everyone career advancement every 3 years. As a nurse summarizes: “Give eight for those who need it. Give ten to those who deserve it. Understood?” (E3B). Among the interviews there was an exception. Due to interpersonal conflicts, one nurse felt harmed by her coworkers in her evaluation: “relationships at work are conflictual and people take to this performance assessment not the professional performance itself, but the performance of the relationship I have with this professional”. (E4B) This situation, even if a minority one, is confirmed by the physician from the other unit:

You can take all the evaluations, it’s between nine and ten for everyone, or sometimes it happens that an evaluation falls into the hands of a disaffected person or something like that, go there and blow up the evaluation with 7.5. […] I don’t know if I really agree with this type of evaluation, especially for salary progression (M1A).

Because of these issues, several workers suggested delinking PE from career progression, assuming that this would allow for a more honest peer review, as we saw in the previous speech.

With respect to the PMAQ, a third of respondents highlighted the long cycles of application of the instrument as a problem in the process. This made them lose their direction over time, as a nurse points out. “They don’t give much... ‘north’ like that to us. They come... and ask how our control is, what do we have... and I think there should be something more constant, not just once in a while” (E2B). This aspect is one of the factors of the main problem pointed out for the PMAQ: its applicability in everyday life.

Most technical professionals pointed out that the main hindering element for the application of PMAQ results in daily life is the model of organization of work in primary health care in the municipality, which devoted a lot of time to care for acute cases. The professionals complained that it was “[…] very cute to talk about how the PMAQ would be, but with this door wide open, it’s getting harder every day” (E3A). This overloaded them: “[…] the clash on a daily basis, I think it’s still a bit suffocating, you know? It is suffocating” (M1A).

Professionals considered that the role of primary care is health promotion and prevention. Thus, the need to attend to acute cases conflicted with the professional conception. This perception is similar to the results of Gomes, Cotta, Araújo, Cherchiglia and Martins (2011), who analyzed the social representations of professionals in relation to primary health care, which they considered restricting their role to health promotion and prevention. An analysis of the results of the first PMAQ, carried out in 2012, indicated the incipience of prevention and health promotion actions in primary health care across the country (Medina et al. 2014), thus corroborating the perception of respondents. Nevertheless, Norman and Tesser (2015) understand that historically we have worked with a false dichotomy between spontaneous demand and programmed demand, the latter being historically marked by a preventive character. From the authors’ perspective, the most important thing would be to guarantee the access of the enrolled population to the teams, in meetings that were not marked by the complaint-conduct pattern, but by comprehensive care. For this to occur, it would be necessary to reduce the population enrolled by EqSF, as occurs in countries that have universal assistance. This perspective may challenge the perception that integrality in its prevention and promotion components is impeded by the pressure of ‘spontaneous demand’, but we understand that this will need to involve not only a review of the team work process, but an investment in increasing the number of EqSF and the consequent decrease in its enrolled population.

Finally, the last problem pointed out concerns the commitment made by municipal managers. Almost all the professionals and one of the managers of the two UBS highlighted the lack of transparency on the part of the PBH regarding the transfer of the resource promised by the Ministry of Health to the teams that had good results in the PMAQ. “[…] they said that... this money from the PMAQ was going to come to the health center... to share it with the team employees... and that money never arrived” (E3B). This even led to a boycott by professionals, who refused to join the third cycle of the PMAQ⁴. The professional statements reinforce what the literature points out. The effectiveness of an instrument and its application in the evaluated activities depend not only on the material, but also on the environment in which it is inserted. According to Hood (2012), number-based management improves or worsens the effectiveness of a service depending on the context and the objectives it serves. Evaluation alone does not generate the desired changes if the environment is not conducive to transformation.

The problems with evaluations have multiple factors that do not concern only the instruments. In addition to the criticism directed at the indicators themselves, the evaluation process and its link to financial gains, as explained above, were also criticized by the interviewees.

Subjectivation processes produced by evaluations

Data analysis revealed the effects of subjectivation that the evaluation practices produced on the interviewees. The main effects were a reflective knowledge about what is important at work, associated with the development of strategies to improve work actions; feelings of fear, guilt and frustration in relation to the result of one’s performance; the collective action to refuse to adhere to the new PMAQ-AB cycle, given the non-fulfillment of the commitment to transfer resources to the units and workers.

Respondents emphasize that they adopted the material as a reference and compared it with the activity they performed and with the attitude they had. With this data, they started to think about their attitudes differently and better direct their actions individually and as a team. A nurse demonstrated how her team worked in meetings to understand the results: “[…] we have a bigger discussion; we look at indicator by indicator [...] so we see where there are flaws” (E3A).

There has been an ‘introjection of the evaluation gaze’, as recommended in a document from the Ministry of Health (Humaniza SUS, 2004). Based on the new perceptions promoted by the evaluations, workers felt impelled to promote changes in their actions. Even with the criticism related to the difficulty of putting into practice what they learned with the processes, half of the interviewees said they had changed their attitudes at work. With the results of the PMAQ, the changes focused primarily on managing teamwork. Nurses were protagonists in this process. “Some things that we normally don’t demand from... the team, that from some... some items, right, that were requested in the PMAQ, we end up inserting in our day-to-day work” (E4A). Concomitantly, there was a reorganization of work and data. “So, today I prepare the team so that if the mayor suddenly says ‘everyone has to answer the PMAQ’, I already have these indicators kind of ... organized, understand?” (E2A).

Although performance evaluation was considered unreliable because of its economic bias, it also promoted changes in the professional attitude. Two examples, already seen before, were the physician who changed her attitude with the patients in relation to the hours of service (M1B) and the nursing technician who changed her attitude with the team (T1A). The evaluations promoted an increase in the knowledge of professionals about their own performance at work. According to the results and expected postures, attitudes were modified. Research carried out in a municipality in the interior of the state of Rio Grande do Norte identified that the main changes after the implementation of the PMAQ-AB occurred in the organization of work, infrastructure and organization of records (Feitosa et al., 2017).

The second effect of subjectivation produced was feelings of fear, guilt and frustration in relation to the result of the performance. Expressions such as ‘failure’, ‘error’ and even ‘sin’ were present in the conversations with the interviewees.

Regarding the effects of subjectivation, Dujarier (2015) understands that management by comparing evaluation results can instill fear of punishment. Because they feel cornered, the worker tends to take positions to sabotage the process. In the case of one of the nurses, upon receiving a negative evaluation in the PMAQ, she began to suffer and decided not to participate in the next cycle of evaluations. She took a stand saying: “I think not signing the next cycles is one way for us...’entering the den’. They can’t just keep throwing stones at me. [...] I stay in the den and keep throwing stones from inside too” (E4B).

More than half of the interviewees, including one of the managers, pointed out some negative feeling arising from the evaluation. The main feeling expressed directly by them was frustration, present in the speech of nine of them.

One aspect pointed out by seven professionals was the difficulty of putting into practice what they had learned with the PMAQ. Health professionals understand that the organization of the service at the UBS stifles the work and removes the professional autonomy, generating frustration and demotivation. “We get unmotivated... right?” (E3B). They felt limited to carry out the proposal:

The negative is that you are not always able, or you have instruments, or you are able to change, improve that situation. Sometimes you realize that a flaw exists, but you won’t always be able to fix it. [...] We don’t have the autonomy to change the flow inside here (E1A).

The third effect of subjectivation was the collective refusal, mediated by the Union, to join the new PMAQ-AB Cycle, if there were no guarantees of financial transfer to the units and workers. “So, people... they don’t want to answer PMAQ anymore’. (E5A). “Because what’s been happening in the City Hall about it, I participated in a PMAQ meeting now, many teams don’t want to join the evaluation anymore, you know?” (E2B).

The lack of financial transfer from the PMAQ was greatly felt. All made an effort to reach the targets required by the indicators, with the promise that the UBS would receive some resources for improvements in infrastructure and in the acquisition of materials. Contact with colleagues who worked in other municipalities and received transfers caused anger among the teams. “Belo Horizonte, which is bigger, whoever was in management did not pass it on to us. [...] In fact, people were frustrated, because you try to do a work and give your best and suddenly...” (A1A). All made an effort to reach the targets required by the indicators, with the promise that the UBS would receive some resources for improvements in infrastructure and in the acquisition of materials.

Dujarier (2015) suggests that there are two vectors in subjectivation processes. The first would be a servile subjectivation, shaped by fear. The second would be the emergence of strategic and autonomous postures, which can undo the submission’s bonds. Even though the refusal to adhere and the consequent non-federal funding can be considered a reckless measure, it was through it that the Health Department carried out, in 2017, an agreement that had been negotiated since 2015. As a reaction to certain evaluation processes, the workers, who are always the focus of evaluations, tend, over time, to find strategies to circumvent the process, breaking with adherence to game rules (Dujarier, 2015).

We cannot ignore that evaluation is an instrument of government, management, and in this relationship, there are both moments of alignment and moments of confrontation. Both moments can have autonomy contours. As Foucault states, the critical stance implies that when listening from the other ‘obey’, one must assess whether there is a fair idea in this. The autonomous reflection can lead to the refusal of order, or to obedience, which, in this case, “[…] will be founded on the autonomy itself” (Foucault, 1978, p. 6). That is, not all alignment is always submission, just as not all confrontation is always autonomy. Thus, more autonomous or more submitted processes of subjectivation will depend on individual and collective processes experienced by workers in the face of evaluation practices. Duffy (2017), based on the contributions of Foucault and Deleuze, states that subjective transformation is not apart from regulation, as the so-called ‘self-practices’ involve the autonomous handling of norms, styles and conventions present in culture.

Final considerations

The evaluation has become a reality of the Unified Health System. Although most of the literature states that the evaluation promotes constraints on the subjectivities evaluated, we analyze that it can also be a material that produces subjectivities engaged in improving the care of SUS users, and improve the team work process.

The PMAQ and AD fueled subordinations, engagements and resistance. They promoted reflections on the attitudes and actions of the employees, leading them to new positions regarding the work. With the PMAQ, mainly, the work was reorganized to better serve UBS users. But, in addition to team engagement and planning, evaluations also generated conflicts, frustrations and blame.

This article has limitations as to its possibility of generalization, due to the regional character of the study, however it adds new reflections on the role of evaluations in health institutions. The evaluation proves to be an instrument that enhances new actions, improving the service provided to the population and triggering the professionals’ subjectivation processes. On the other hand, we also emphasize that it does not work in isolation, it is necessary contexts that welcome transformations, that comply with agreements and that it is periodically revised for the constant construction of new possibilities. Assuming that evaluation is only transformative when the institutional environment is conducive to it, as pointed out by Hood (2012), can indicate an important limit to the effectiveness of evaluation processes. Despite this, we understand that the evaluation culture itself can sensitize its surroundings to the possibility of transformation, which expands its power as an intervention.

Referências

• Abelson, J., Humphrey, A., Syrowatka, A., Bidonde, J. & Judd, M. (2018). Evaluating patient, family and public engagement in health services improvement and system redesign. Health Quarterly, 21, 61-67. DOI: https://doi.org/10.12927/hcq.2018.25636
  » https://doi.org/10.12927/hcq.2018.25636
• Carvalho, A. L., & Shimuzu, H. E. (2017). A institucionalização das práticas de monitoramento e avaliação: desafios e perspectivas na visão dos gestores do Sistema Único de Saúde. Interface, 21(60), 23-33. DOI: https://doi.org/10.1590/1807-57622015.0817
  » https://doi.org/10.1590/1807-57622015.0817
• Duffy, D. (2017). Evaluation and governing in the 21th century: disciplinary measures, transformative possibilities. Londres, UK: Palgrave MacMillan.
• Dujarier, M. A. (2015). A automatização do julgamento emitido sobre o trabalho: medir não é avaliar. In G. Balandier (Org.). O que avaliar quer dizer? (A. Danesi, Trad.). São Paulo, SP: Fasp-Unifesp.
• Feitosa, R. M., Paulino, A., Lima-Júnior, J. O., Oliveira, K. K., Freitas, R. J., & Silva, W. F. (2017). Mudanças ofertadas pelo programa nacional de melhoria do acesso e da qualidade da atenção básica. Saúde e Sociedade, 25(3), 821-829. DOI: https://doi.org/10.1590/s0104-12902016151514
  » https://doi.org/10.1590/s0104-12902016151514
• Felisberto, E., Freeze, E., Bezerra, L., Alves, C. & Samico, I. (2010). Análise da sustentabilidade de uma política de avaliação: o caso da atenção básica no Brasil. Cadernos de Saúde Pública, 26(6), 1079-1095. DOI: https://doi.org/10.1590/S0102-311X2010000600003
  » https://doi.org/10.1590/S0102-311X2010000600003
• Felix, G., Bertolin, J., & Polidori, M. (2017). Avaliação da educação superior: um comparativo dos instrumentos de regulação entre Brasil e Portugal. Avaliação, 22(1), 35-54. DOI: https://doi.org/10.1590/s1414-40772017000100003
  » https://doi.org/10.1590/s1414-40772017000100003
• Ferreira-Neto, J. L. (2017). Cultura de avaliação e efeitos de subjetivação na saúde. Psicologia em Estudo , 22(4), 539-549. DOI: https://doi.org/10.4025/ psicolestud.v22i4.36676
  » https://doi.org/10.4025/ psicolestud.v22i4.36676
• Foucault, M. (1978). O que é a crítica? Crítica e Aufklärung (G. Borges, Trad.). Recuperado de http://michel-foucault.weebly.com/uploads/1/3/2/1/13213792/critica.pdf
  » http://michel-foucault.weebly.com/uploads/1/3/2/1/13213792/critica.pdf
• Foucault, M. (2004). Ética, sexualidade, política Rio de Janeiro, RJ: Forense Universitária.
• Furtado, J., & Vieira-da-Silva, L. (2014). A avaliação de programas e serviços de saúde no Brasil enquanto espaço de saberes e práticas. Cadernos de Saúde Pública, 30(12), 2643-2655. DOI: https://doi.org/10.1590/0102-311x00187113
  » https://doi.org/10.1590/0102-311x00187113
• Gomes, K. O., Cotta, R. M. M., Araújo, R. M. A., Cherchiglia, M. L., & Martins, T. C. P. (2011). Atenção Primária à Saúde - a “menina dos olhos” do SUS: sobre as representações sociais dos protagonistas do Sistema Único de Saúde. Ciência e Saúde Coletiva, 16(1), 881-892. DOI: https://doi.org/10.1590/S1413-81232011000700020
  » https://doi.org/10.1590/S1413-81232011000700020
• Hood, C. (2012). Public management by numbers as a performance-enhancing drug: two hypotheses. Public Administration Review, 72(1), 585-592. DOI: https://doi.org/10.1111/j.1540-6210.2012.02634.x
  » https://doi.org/10.1111/j.1540-6210.2012.02634.x
• Humaniza SUS: Política Nacional de Humanização. (2004). Brasília, DF: Ministério da Saúde. Departamento de Atenção Básica.
• Martuccelli, D. (2015). Crítica da filosofia da avaliação. In G. Balandier (Org.). O que avaliar quer dizer? (A. Danesi, Trad.). São Paulo, SP: Fasp-Unifesp .
• Medina M. G., Aquino R., Vilas Bôas A., Mota E, Pinto-Júnior, E., Luz, L. A., … Pinto, I. C. M. (2014). Promoção da saúde e prevenção de doenças crônicas: o que fazem as equipes de Saúde da Família? Saúde em Debate, 38(esp.), 69-82. DOI: https://doi.org/10.5935/0103-1104.2014S006
  » https://doi.org/10.5935/0103-1104.2014S006
• Menicken, A., & Miller, P. (2012). Accounting, territorialization and power. Foucault Studies, 13, 4-24. DOI: https://doi.org/10.22439/fs.v0i13.3503
  » https://doi.org/10.22439/fs.v0i13.3503
• Norman, A., & Tesser, C. (2015) Acesso ao cuidado na Estratégia Saúde da Família: o equilíbrio entre demanda espontânea e prevenção/promoção da saúde. Saúde e Sociedade, 24(1), 165-179. DOI: https://doi.org/10.1590/S0104-12902014000300011
  » https://doi.org/10.1590/S0104-12902014000300011
• Pacheco R. S. (2010). A agenda da Nova Gestão Pública. In M. Loureiro, F. Abrucio, R. Pacheco (Org.). Burocracia e política no Brasil: desafios para a ordem democrática no século XXI Editora FGV.
• Pinto, H. (2014). Múltiplos olhares sobre e a partir do Programa de Melhoria do Acesso e Qualidade (Dissertação de Mestrado). Escola de Enfermagem, Programa de Pós-graduação em Saúde Coletiva, Universidade Federal do Rio Grande do Sul, Porto Alegre.
• Power, M. (2000). The audit society - second thoughts. International Journal of Auditing, 4(1), 111-119. DOI: https://doi.org/10.1111/1099-1123.00306.
  » https://doi.org/10.1111/1099-1123.00306
• Saúde mais Perto de Você. Acesso e Qualidade. Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica (PMAQ): Manual Instrutivo (2012). Brasília, DF: Ministério da Saúde. Departamento de Atenção Básica.
• Sindibel. (2017). Sindicato dos servidores públicos municipais de Belo Horizonte. Sindibel Informa (panfleto distribuído aos servidores da saúde).
• Souza, L. K. (2019). Pesquisa com análise qualitativa de dados: conhecendo a Análise Temática. Arquivos Brasileiros de Psicologia, 71(2), 51-67. DOI: https://doi.org/10.36482/1809-5267.ARBP2019v71i2p.51-67
  » https://doi.org/10.36482/1809-5267.ARBP2019v71i2p.51-67
• Triantafillou, P. (2017). Neoliberal power and public management reforms Manchester, UK: Manchester Press.

• 4
  Only in September 2017, when an increase from 15% to 20% of the resource for Performance Compensation was negotiated, the Union oriented the civil servants to adhere to the contract (Sindibel, 2017).

Publication Dates

• Publication in this collection
  16 Mar 2022
• Date of issue
  2022

History

• Received
  08 June 2019
• Accepted
  03 Sept 2020