Structural and procedural conditions in National Immunization Program Information System establishment

Condiciones estructurales y procesales en la implementación del Sistema de Información del Programa Nacional de Inmunización

ABSTRACT

Objectives:

to analyze structural and process conditions in National Immunization Program Information System establishment.

Methods:

a cross-sectional study conducted in 307 vaccination rooms in the state of Minas Gerais in 2017. For data collection, a multidimensional questionnaire was used. Descriptive data analysis was performed.

Results:

vaccination rooms have basic inputs necessary for System establishment. The greatest problems relate to professional practice. Low enrollment of population, failures in the active search for absentees, vaccine scheduling and absence of reports to monitor vaccination coverage were identified. Training was considered insufficient and ineffective.

Conclusions:

Immunization Information System is an essential technological innovation for the management of immunization actions. However, the production of timely records and the use of information are still challenges. Investments in training are required to ensure System’s management and operationalization activities.

Descriptors:
Immunization Programs; Information Technology; Health Information Systems; Nursing; Evaluation Studies

RESUMEN

Objetivos:

analizar las condiciones estructurales y de proceso en la implantación del Sistema de Información del Programa Nacional de Inmunización.

Métodos:

estudio transversal, realizado en 307 salas de vacunación, en el Estado de Minas Gerais, en 2017. Para la recolección de datos, se utilizó un cuestionario multidimensional. Se procedió a un análisis descriptivo de los datos.

Resultados:

las salas de vacunación tienen insumos básicos necesarios para la implantación del Sistema. Los mayores problemas se refieren a las prácticas de los profesionales. Se identificaron bajo registro de la población adscrita, fallas en la búsqueda activa de faltosos y en la programación de vacunas y ausencia de informes para monitorear la cobertura vacunal. Los entrenamientos se consideraron insuficientes y poco efectivos.

Conclusiones:

el Sistema de Información de Inmunización es una innovación tecnológica esencial para la gestión de las acciones de inmunización, pero todavía son desafíos, la producción de registros oportunos y el uso de la información. Las inversiones en capacitación son necesarias para garantizar las actividades de gestión y operacionalización del Sistema.

Descriptores:
Programas de Inmunización; Tecnología de la información; Sistemas de Información em Salud; Enfermería; Estudios de Evaluación

RESUMO

Objetivos:

analisar as condições estruturais e de processo na implantação do Sistema de Informação do Programa Nacional de Imunização.

Métodos:

estudo transversal realizado em 307 salas de vacinação em Minas Gerais em 2017. Para a coleta de dados, foi utilizado um questionário multidimensional. Procedeu-se a análise descritiva dos dados.

Resultados:

as salas de vacinação possuem insumos básicos necessários para a implantação do Sistema. Os maiores problemas relacionam-se às práticas dos profissionais. Identificaram-se baixo cadastro da população adscrita, falhas na busca ativa de faltosos, no aprazamento de vacinas e ausência de relatórios para monitorar a cobertura vacinal. As capacitações foram consideradas insuficientes e pouco efetivas.

Conclusões:

o Sistema de Informação de Imunização é uma inovação tecnológica essencial para o gerenciamento das ações de imunização, mas ainda são desafios a produção de registros oportunos e o uso da informação. Investimentos em capacitações são necessárias para garantir as atividades de gestão e operacionalização do Sistema.

Descritores:
Programas de Imunização; Tecnologia de Informação; Sistema de Informação em saúde; Enfermagem; Estudos de Avaliação

INTRODUCTION

Health Information Systems (HIS) are technological innovation strategies that instrumentalize the process of collecting, processing, analyzing and disseminating information, enhancing the management of these data in different health care settings^(1-2). Knowing the importance of appropriation and proper handling of such inputs is fundamental to foster, develop and implement decision-making in health care services⁽³⁾

Health information systems include those that can prioritize and manage immunization data. In the United States (USA), the Immunization Information System (IIS) has been in use since the 1970s⁽⁴⁾. This system aims to plan decision-making regarding vaccination activities by assessing the risk of outbreaks or epidemics, by monitoring the vaccination process individually or by controlling the inventory of immunobiologicals needed by administrators^(5-7).

The Brazilian Unified Health System (SUS - Sistema Único de Saúde) currently has the National Immunization Program Information System (SIPNI - Sistema de Informação do Programa Nacional de Imunização) established in 2010 by the National Immunization Program⁽⁵⁾. This system replaces all local immunization systems, enabling individual registration of each vaccinee⁽⁷⁾. In addition, it brings together, on a single basis, subsystems that provide diverse information, such as the amount of doses applied, vaccine coverage, and Adverse Events Following Immunization (AEFI). There are two versions of SIPNI available, the desktop and the online system (web version)⁽⁸⁾.

However, despite the numerous benefits and being in an advanced stage of establishment in Brazil, some challenges are observed regarding its operationalization, completeness and quality of data collected (duplicity and underreporting). Data that often provides poor quality data well below interoperability standards^(9-11). These issues are compounded by the scarcity of skilled human resources, the lack of information technology, the ineffectiveness of constantly updating SIS and the integration of all these factors^(1,12). This reality is even more alarming in regions where the high demand for services is higher due to the large population contingent, such as the state of Minas Gerais (MG).

Added to the difficulty of operationalizing information systems are two relevant aspects: the first is structural and refers to the resources employed, such as the existence of own information systems in some municipalities, whose programming languages and commands are incompatible with SIPNI. The second is procedural and refers to services well produced using a certain technology, such as SIPNI^(7,13). Both are equally important factors for the effectiveness of the SIPNI as a planning and decision-making tool in relation to vaccination activities in the daily routine of services.

It is in this sense that it becomes important to know the facilitating and hindering elements of the structural and procedural conditions of the SIPNI. These requirements are able to influence the expected results within municipal vaccination rooms. By characterizing such dimensions to the correct handling logic, it is believed that it can provide not only learning about the importance of this technology, but also awakening the professional to problem solving and situational diagnostics.

Thus, it is expected that this work will broaden the scope of knowledge about the gaps that permeate nursing professional practice in the correct SIPNI handling. It is believed that, based on their findings, more plausible strategies can be chosen, especially aiming at professional critical reflection on issues related to immunization. These themes are monitoring vaccination coverage, searching for overdue cards and others, in order to involve the entire multidisciplinary team and integrate this perspective with a logic closer to the reality of these professionals, ultimately improving health indicators.

OBJECTIVES

To analyze structural and process conditions in National Immunization Program Information System (SIPNI - Sistema de Informação do Programa Nacional de Imunização) establishment.

METHODS

Ethical aspects

This study was approved by the Human Research Ethics Committee of Universidade Federal de São João del-Rei (REC/UFSJ).

Design, period and place of study

This is a cross-sectional study that was conducted in the primary health care (PHC) vaccination rooms of the municipalities of the Western Region of Minas Gerais in 2017. The dimensions of structure and process related to the management components of SIPNI, vaccinee registration and movement of immunobiologicals were analyzed. The structure consists of relatively stable characteristics, such as physical and organizational conditions, and the process encompasses the activities and goods produced developed in services^(13-14).

MG region was chosen because its population has presented, in the last ten years, the third highest population growth rate of Minas Gerais (14.93%), reaching 1,364,023 inhabitants, which corresponds to 5.8% of its total⁽¹⁵⁾. In addition, this region is responsible for feedback on a considerable volume of epidemiological data from the state, with 307 vaccination rooms distributed in PHC of the 54 corresponding municipalities. Nursing professionals play a prominent role in this setting because they compete for immunization activities. Usually nursing technicians and/or assistants are supervised by a nurse and perform all activities related to the vaccination room, including those related to SIPNI. In most municipalities, a professional, usually a nurse, is nominated to occupy the position of technical reference in immunization⁽¹⁶⁾.

Population or sample; inclusion and exclusion criteria

The eligible population referred to the nursing professional working in the 307 PHC vaccination rooms of the 54 municipalities mentioned above. Inclusion criteria were professionals who work in vaccination rooms with the SIPNI implanted, with at least one nursing professional who knew how to operate the System. In addition, only those whose participation was voluntary in the research were elected.

Study protocol

A multidimensional questionnaire (structure and process)⁽¹⁷⁾ consisting of objective questions about: (1) demographic social characterization of participants was used for data collection; (2) dimension structure (physical facilities; equipment, human resources, financial resources, regulations); (3) process dimension (activities related to the operationalization and management of the SIPNI, vaccination registration and movement of immunobiologicals). Prior to the collection were clarified to the nursing professionals (nurse, technician or nursing assistant) participating in this research the procedures and stages of the research and then asked to sign the Free and Informed Consent Term (FICT). Data collection was performed in person by the research members, previously trained to apply the questionnaire.

Analysis of results, and statistics

A descriptive data analysis was performed with distribution of absolute and relative frequencies of the studied variables, as well as measurement of median central tendency of quantitative variables. Data were processed and tabulated in Epidata software (version 3.1. Epidata Association, Odense, Denmark) and Epi Info 7.0.

RESULTS

Applying the study eligibility criteria, of the 307 293 vaccination rooms were included in the study (95.4%). Fourteen rooms were excluded for not having SIPNI established. Thus, a total of 293 nursing professionals were interviewed.

Of these, most were female (275; 93.9%) and were 30 or older (228; 77.8%). The median age observed was 36 years. Half of them (147; 50.2%) were nurses. Most of participants had complete high school (146; 49.8%) and more than 10 years since graduation (133; 45.4%).

Regarding the working time of professionals in vaccination rooms, 106 (36.2%) work from 1 to 5 years. The median of this working time was 6 years. SIPNI establishment in vaccination rooms varied between 2011 and 2017, with the majority being established from 2014 (274; 95.3%).

Results showed that among the 293 vaccination rooms, 64.8% (n=199) had SIPNI implanted and 30.6% (n=94) used their own information system (Information System acquired by the municipality to aggregate several) although using the SIPNI database. Of the rooms with SIPNI deployed, 29.6% (n=59) operate the web version (online) and 140 (70.4%) operate the desktop version.

Only 48.5% (n=142) of the vaccination rooms have the professional registered as an operator in the SIPNI in the health unit and the others remain in the municipal health department. As for training, the majority (231; 78.9%) of the professionals were trained, although 33.5% (n=98) of the training were performed only in the presence of a new system update.

SIPNI-compatible computers (231; 78.85%) and printers (233; 79.55%) were found in most vaccination rooms, as well as the existence of basic printing supplies (191; 65.2%). However, while 48.8% of services have stable internet access, a considerable rate reported instability (36.9%) or lack of access (14.3%) (Table 2). The municipality’s computer technician was the main reference used to solve the system’s doubts, followed by the technical reference of the Regional Health and Telehealth Superintendence. The least used communication channels by professionals were SIPNI chat, followed by SIPNI website (DATASUS) and SIPNI video lessons on YouTube. In 100% of the vaccination rooms participating in the study, paper is used to record the activities performed in the daily routine of services.

Table 1
Characteristics of nursing professionals working in vaccination rooms in the municipalities of the Western Region of Minas Gerais, Brazil, 2017

Variável	n (%)
Sex
Male	18 (6.1)
Female	275 (93.9)
Age (years)
Up to 29	65 (22.2)
30 o above	228 (77.8)
Education level
Vocational High School	146 (49.8)
Higher Education	46 (15.7)
Mster's Degree	3 (1)
Specialization	98 (33.5)
Graduation time
< 1 year (2017) 1 to 5 years (2012 - 2016)	2 (0.7) 56 (19.1)
6 to 10 years (2007 - 2011	102 (34.8)
> 10 years (1982 - 2006)	133 (45.4)
Work place
Family Health Strategy	274 (93.5)
Traditional Health Unit	19 (6.5)

Table 2
Characteristics of the structure dimension of SIPNI in the municipalities of the Western Region of Minas Gerais, Brazil, 2017

Structure	n (%)
Existence of computer with operating system SIPNI compliant	231 (78.8)
Printer Existence Available	233 (79.5)
Existence of basic printing supplies	191 (65.2)
Presence of SIPNI manual (online or printed)	192 (65.5)
Use of communication channels	111 (37.9)
Existence of a professional for technical support (Computing)	255 (87.0)
Access to internet in the vaccination room
Stable internet	143 (48.8)
Unstable internet	108 (36.9)
No access to internet	42 (14.3)

Regarding the process dimension, the potential factors that influenced SIPNI operationalization and management stand out. Regarding tool handling, the lowest rates were related to insufficient enrollment of the enrolled population (34; 11.6%), the report to monitor vaccination coverage (47; 16.0%) and the estimated withdrawal rate (73; 24.9%) (Table 3).

Table 3
Characteristics of activities related to the operationalization and management of SIPNI in the municipalities of the Western Region of Minas Gerais, Brazil, 2017

Operationalization/management	n (%)
Registered vaccinators	222 (75.8)
Reporting of the SIPNI registered professional's dismissal to the System operator	205 (70.0)
Registration of vaccines at SIPNI by the professional administering the vaccine	188 (64.2)
Registered population	34 (11.6)
SIPNI versions updated	137 (46.8)
Backup performed	199 (67.9)
Export files sent monthly to the SIPNI municipal coordination	138 (47.1)
Report to monitor vaccination coverage	47 (16.0)
Report of applied doses	90 (30.7)
Relatório para busca ativa de faltosos	122 (41.7)
Use of information generated by SIPNI to control immunobiological inventory	198 (67.6)
Estimated withdrawal rate	73 (24.9)
Disclosure of SIPNI information	144 (49.2)

Registration of vaccines administered in the SIPNI was not always performed by the person administering the vaccine, being made by different professionals besides the nursing staff, such as administrative agent, nursing intern and Community Health Agent. Regarding the professional responsible for updating the SIPNI versions, the results indicated that the updates are made by the municipality’s computer technician, nurse or administrative assistant of the unit.

Among those who back up data, 24.9% use a thumb drive, data cloud, CD, DVD or external hard drive, followed by 8.5% saving the backup to another folder on their computer and 2.1%. use the default SIPNI folder. It was found that some locations (59; 20.1%) do not need to back up the data as they use the SIPNI web version.

Regarding the activities related to vaccinee registration, the results pointed to vaccines and the active search for cardholder delinquents as the least accomplished items (Table 4). It is noteworthy that in the vaccination registration screen the information is filled only in the required fields in 48.1% vaccination rooms (n=141).

Table 4
Characteristics of activities related to SIPNI vaccinee registration in the municipalities of the Western Region of Minas Gerais, Brazil, 2017

Vaccinee registration	n (%)
Registration of vaccinee information (required and not required)	152 (51.9)
Vaccine scheduling	100 (34.1)
Active delinquent card search	140 (47.8)
SIPNI registered batch conference with vaccine batch	267 (91.1)

Regarding activities related to the movement of immunobiologicals. most professionals update the vaccine batches in the SIPNI. However, there are still failures regarding vial registration used in the vaccination room and the filling of immunobiological loss fields (Table 5).

Table 5
Characteristics of activities related to the movement of immunobiologicals in SIPNI in the municipalities of the Western Region of Minas Gerais, Brazil, 2017

Movement of immunobiologicals	n (%)
Updated records of SIPNI vaccine batches	276 (94.2)
Registration of vials received and used in the vaccination room	218 (74.4)
Filling in the immunobiological loss fields	204 (69.6)

DISCUSSION

The analysis of the results identified that, in general, the most limiting elements of SIPNI establishment were related to the process dimension, signaling a need for a general change of the activities developed in the production relations between professionals and vaccinates.

Regarding the preparation of these professionals in the handling of the SIPNI, although part of the interviewees reported that they received training when they entered the service in the vaccination room, they do not feel prepared to fully operationalize the system. The training, when carried out, was due to changes in the new versions, focusing on immunization registration. This setting may occur due to a discontinuity of training, suggesting that the nurse managers of the units have played their role in providing continuing education with professionals. However, such training and training do not last for long periods of time, making professionals desensitized and able to resume their old practices^(16,18).

A systematic review study, to synthesize the facilitating and hindering conditions in electronic the health record introduction identified that where there was adequate technological support and training, the acceptance of the information system was easier. In contrast, the lack of training were barriers to system introduction⁽¹⁹⁾.

Another limiting factor may be related to the unfamiliarity of professionals with technologies and eventual system failures, since the successful immunization information system introduction depends on the active involvement of health professionals^(12,20). The ease of using a technology influences its acceptance. One study found that being able to manipulate technological tools influences the use of a HIS⁽²¹⁾. Thus, conducting information system training is related to the ease of use and usefulness of the system as it enhances people’s ability to handle the technology^(19,21).

Studies conducted in Ghana (2016) and Brazil (2018) showed that in a health management system (DHIMS) introduction, many health facilities still use paper forms, underestimating the use of health technology systems, which may delay reporting and incomplete data^(11,22). However, in the study’s participating vaccination rooms, the use of the paper was justified by the instability of the internet, system slowdowns and the vaccine data sometimes being released into the SIPNI by someone else.

According to COFEN (Conselho Federal de Enfermagem - Nursing Federal Board) Technical Opinion 008/2012, the administration of medication or any procedure performed by the nursing professional when it is registered by another is prohibited. This is in line with what was observed in the study, so that vaccine administration should be delivered to the system by the recipient.

Therefore, the need to introduce information systems in an integrated manner and, nonetheless, sensitize professionals about the correct and complete handling of the SIPNI anchored to more innovative perspectives. There is still a need to change the management of such information, since the available vaccination data are often being manipulated to meet alternatives to improve data quality⁽¹⁰⁾.

This transition should take place from training that encourages reflection and creative learning potential, stimulating adherence and strengthening the importance given to such devices, encouraging greater motivation that overcomes the difficulties inherent in handling such a tool. It is believed that the records can be enhanced and the biases in the administration of the immunobiological can be optimized, optimizing the data entry in the system and making the report generation closer to reality⁽¹⁷⁾.

Therefore, among the challenges encountered in SIPNI establishment, the lack of nurses fully involved with technological innovation may be the precursor of low effectiveness in handling this system, encouraging the need to encourage participation and involvement of nurses in order to instill the responsibility and potential for their proper use. Moreover, they could support, train other nursing professionals (nursing technicians and assistants) and use the information generated to plan actions aimed at the population in its area of coverage. They would also disclose immunization information to both other unit professionals and the population, making them also responsible for increasing immunization coverage.

Regarding the process dimension, despite these identified barriers, the enrollment and registration of vaccines does not appear to be a problem of SIPNI establishment, because the health units investigated, in their majority, presented good general indicators, since 70 % record vaccinee information. This fact reproduces the SIAPI (Brazilian National Immunization Program Assessment Information System - Sistema de Informação de Avaliação do Programa Nacional de Imunizações) which proposed the systematic recording of the number of doses administered by age group and calculation of vaccine coverage. However, the data were generated by doses applied and not by vaccinee individuals, which made it difficult to track the delay/withdrawal of vaccination schedule by individual⁽²³⁾.

The use of information generated by the SIPNI to calculate absenteeism, dropout rate and vaccination coverage according to the study is underused by most of the interviewed vaccination rooms. Knowing that health indicators generated by ISS are an important resource to support managers, especially in the direction of decision making, such underutilization goes in the opposite line of health planning. The reduced use of these indicators has been justified by limiting factors, such as the reduced number of professionals, the lack of training, the poor integration of the various systems and their slowness⁽²⁴⁾.

It is in this sense that the role of local management of each unit becomes important, indicating the real need of the population in its area of coverage and efficient use of data, setting the best strategy for a good health intervention plan. In the same perspective, it is necessary to reassess ISS in order to be able to direct the management activities and thereby favor the quality of SUS services⁽²⁵⁾.

This situation is corroborated by the lack of a health information and informatics policy aimed at the organization of the processes of production, use and dissemination of information and the insufficient training of human resources for information management⁽¹⁾. The main challenge is to make a quality leap in information management and in the appropriation of its product for the construction of a care model compatible with the guiding principles of SUS, in fact contributing to municipalization⁽²⁰⁾.

Despite the obstacles found in the handling of the SIPNI regarding the process dimension, the results regarding the SIPNI structure dimension, in turn, suggested that data were favorable for its establishment. Data were not found problems such as availability of printers, operating systems and technical support, as well as the stability of the internet. Vaccination room structures are being worked out to meet basic operational needs, such as proprietary computer availability in most vaccine and printer rooms for the collective use of professionals.

In addition, in most vaccination rooms there are manuals (online, scanned or printed) for instruction. That is, in the absence of training, these manuals momentarily supply the operational need of the system. In addition, SIPNI has free technical support that provides video lessons to assist in training for the use of the new information system that can be accessed by anyone interested, having the function of assisting in training and answering questions of professionals in the vaccination room.

This reality suggests that health facilities, despite the difficulties encountered by professionals in its operationalization, have presented the basic inputs necessary for SIPNI establishment. Therefore, the idea that failure is more related to light technologies (which relate to relationships) than to hard technologies (related to material resources), as Merhly puts it, is corroborated⁽²⁶⁾.

The results of this study indicate that SIPNI, despite being a technological innovation created to replace the previous ones and add effective decision-making power and not find major structural problems, is still little used. SIPNI was handled in restricted situations as a mandatory data generator for municipalities and not as a support for decision-making.

As a nurse responsible for the vaccination room, the nurse should be in charge of the duties assigned to him, including the monitoring of data and the use of information produced by the SIPNI, understanding it as an essential tool for health management.

Therefore, it is important that new studies in the area explore innovative strategies to raise awareness of such a tool, so that the training of these professionals (including nurses, nursing technicians and assistants) becomes effective and lasts for longer periods of time. Such skills should be reformulated since, although they do not have long-term effects, professionals seem to recognize their importance. The study by Paulino et al.⁽²⁷⁾, for example, identified that nurses seem to be unaware of the impact indicators of continuing education actions, but recognize that training contributes to the necessary changes in daily life and to the integration between staff and staff. the community.

Given the above, the results analysis allowed us to infer that the biggest problems with SIPNI establishment referred to the process dimension, that is, the low enrollment of the enrolled population and the incipient data from the vaccination coverage reports, besides the active search for absentees and the scheduling of vaccines.

Taking into consideration the purposes of SIPNI since its inception in 2010^(7-8,10), in the face of some difficulties that may be considered natural to the implantation process, its relevance stands out, especially due to the importance of generating vaccinee data from an effective and free operating system.

The fact that it is linked to the SIPNI and the guiding principles of SUS makes this system even more relevant, since Brazil is a reference in coverage and information about vaccine data. Such events can be evidenced by good indicators coming from immunization programs, assistance to Palestinian governments, the West Bank and Gaza Strip, and training provision for countries such as Suriname, Angola, the United States, Mexico, Argentina, among others⁽²⁸⁾.

Study limitations and Contributions to the field

One of the limitations of this study is that, because it is a descriptive study, the identified variables could not be associated with the successful SIPNI establishment. The approach used (quantitative) is still added, although it has answered the objective, as it is very normative, it does not qualify the context variables that can explain the findings. In any case, the importance of different methodological strategies for understanding the impediments of this process is highlighted, so that this tool can reveal its potential to develop and implement decision-making in health care services.

Contributions to nursing, health or public policy

Assessment studies are seen with great relevance to public policy introduction, as it subsidizes the technical and scientific development regarding the organization and management of health systems, programs and/or practices. For nursing, the results of this study may support the professionals’ decision-making actions for SIPNI establishment. Recognition of the need for effective and sustainable information systems is necessary to ensure information feedback and monitoring of activities in an agile, efficient and timely manner.

CONCLUSIONS

SIPNI is an important technological innovation in the management of immunization actions, but its use remains a challenge. Nursing professionals, responsible for immunization services in PHC, need knowledge to ensure effective attitudes towards SIPNI management and operationalization activities, thus improving the conduct of vaccination rooms and supporting health surveillance actions. It is hoped that further studies in the area will be able to generate new possibilities to facilitate the resolution of such difficulties, so that the health data of each site are closer to the local reality, finally improving future perspectives on system implantation.

• FUNDING
  Funding Agencies: Minas Gerais Research Support Foundation (FAPEMIG - Fundação de Amparo à Pesquisa de Minas Gerais) and Ministry of Health (MoH) - SUS Research Program (PPSUS - Programa de Pesquisa para o SUS) (APQ-03787-17; APQ-01505-17).

ACKNOWLEDGMENTS

A special thanks to Universidade Federal São João del-Rei (UFSJ); Minas Gerais State Department of Health; the Ministry of Health (MoH); FAPEMIG; Coordination for the Improvement of Higher Education Personnel (CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível Superior).

REFERENCES

• ¹
  Cavalcante RB, Kerr-Pinheiro MM, Guimarães EADA, Miranda RM. Panorama de definição e implementação da Política Nacional de Informação e Informática em Saúde. Cad Saúde Pública[Internet]. 2015;31:960-70. doi: 10.1590/0102-311X00095014
  » https://doi.org/10.1590/0102-311X00095014
• ²
  Holmes ES, Santos SR, Almeida AF, Oliveira JHD, Carvalho GDA, Fonseca LCT, et al. Health information systems in the decision making process in primary care. Int Arch Med [Internet]. 2016 [cited 2017 Feb 10];9. Available from: http://imed.pub/ojs/index.php/iam/article/view/1380/1116
  » http://imed.pub/ojs/index.php/iam/article/view/1380/1116
• ³
  Santos ADFD, Fonseca Sobrinho D, Araújo LL, Procópio CDSD, Lopes EAS, Lima AMDLD, Matta-Machado AT. Incorporation of Information and Communication Technologies and quality of primary healthcare in Brazil. Cad. Saúde Pública. 2017;33(5):e00172815. doi: 10.1590/0102-311x00172815
  » https://doi.org/10.1590/0102-311x00172815
• ⁴
  Muscoplat MH, Rajamani S. Immunization information system and informatics to promote immunizations: perspective from Minnesota immunization information connection. J Biomedical Inform Insights. 2017. Available from: doi: 10.1177/1178222616688893
  » https://doi.org/10.1177/1178222616688893
• ⁵
  Ministério da Saúde (BR). Secretaria Executiva. DATASUS. Plano diretor de tecnologia da informação. Brasília; 2016. p.73.
• ⁶
  Community Preventive Services Task Force. Recommendation for use of immunization information systems to increase vaccination rates. J Pub Health Manag Pract. 2015;21(3):249-52. doi: 10.1097/PHH.0000000000000092
  » https://doi.org/10.1097/PHH.0000000000000092
• ⁷
  Sato APS. Programa Nacional de Imunização: Sistema Informatizado como opção a novos desafios. Rev Saúde Pública. 2015;49:39. doi: 10.1590/S0034-8910.2015049005925
  » https://doi.org/10.1590/S0034-8910.2015049005925
• ⁸
  Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. Coordenação Geral do Programa Nacional de Imunização. Manual do Sistema de Informação do Programa Nacional de Imunizações SIPNI. Brasil; 2014.
• ⁹
  Curran EA, Bednarczyk RA, Omer S B. Evaluation of the frequency of immunization information system use for public health research. Human vaccines & immunotherapeutics. 2013;9(6):1346-50. doi: 10.4161/hv.24033
  » https://doi.org/10.4161/hv.24033
• ¹⁰
  Teixeira AMS, Domingues CMAS. Monitoramento rápido de coberturas vacinais pós- campanha de vacinação no Brasil: 2008, 2011 e 2012. Epidemiol Serv Saúde. 2013;22(4):565-78. doi: 10.5123/S1679-49742013000400003
  » https://doi.org/10.5123/S1679-49742013000400003
• ¹¹
  Cavalcante RB, Vasconcelos DD, Gontijo TL, Guimarães EA de A, Machado RM, Oliveira VC. Informatização da atenção básica a saúde: avanços e desafios. Cogitare Enferm. 2018;23(3):e54297. doi: 10.5380/ce.v23i3.54297
  » https://doi.org/10.5380/ce.v23i3.54297
• ¹²
  Silva AR, Oliveira TM, Lima CF, Rodrigues LB, Bellucci JN, Carvalho MGO. Sistemas de informação como instrumento para tomada de decisão em saúde: revisão integrativa. Rev Enferm UFPE. 2016;10(9). doi: 10.5205/reuol.9571-83638-1- SM1009201634
  » https://doi.org/10.5205/reuol.9571-83638-1- SM1009201634
• ¹³
  Champagne F. A Análise da Implantação. In: Brousselle A. (org). Avaliação em saúde: conceitos e métodos. Rio de Janeiro: Editora Fiocruz; 2ª reimpressão, 2016. p.217-38.
• ¹⁴
  Donabedian A. Basic approaches to assessment: structure, process and outcome. In: Donabedian A. Explorations in Quality Assessment and Monitoring. pp. 77-125, Ann Arbor, Michigan: Health Adiministration Press; 1980.
• ¹⁵
  Malachias I, Leles F AG, Pinto MAS. Plano diretor de regionalização da saúde de Minas Gerais (PDR/MG). Belo Horizonte: Secretaria de Estado de Saúde de Minas Gerais, 2011.
• ¹⁶
  Oliveira VC, Gallardo MDPS, Arcêncio RA, Gontijo TL, Pinto IC. Avaliação da qualidade de conservação de vacinas na Atenção Primária à Saúde. Ciênc Saúde Coletiva. 2014;19(9):3889-98. doi: 10.1590/1413-81232014199.12252013
  » https://doi.org/10.1590/1413-81232014199.12252013
• ¹⁷
  Silva BS, Coelho HV, Cavalcante RB, Oliveira VC, Guimarães EAA. Evaluation study of the National Immunization Program Information System. Rev Bras Enferm. 2018;71(Suppl 1):615-24. [Thematic Issue: Contributions and challenges of nursing practices in collective health]. doi: 10.1590/0034-7167-2017-0601
  » https://doi.org/10.1590/0034-7167-2017-0601
• ¹⁸
  Oliveira VC, Rennó HMS, Santos YR, et al. Educação para o trabalho em sala de vacina: percepção dos profissionais de enfermagem. Rev Enferm Cent-O Min. 2016;6(3):2331-41. doi: 10.19175/recom.v6i3.1180
  » https://doi.org/10.19175/recom.v6i3.1180
• ¹⁹
  Carrie AM, Grenier S, Duplantie J, et al. Comparison of user groups' perspectives of barriers and facilitators to implementing electronic health records: a systematic review. BMC Med. 2011;9(46). doi: 10.1186/1741-7015-9-46
  » https://doi.org/10.1186/1741-7015-9-46
• ²⁰
  Guimarães EAA, Carmo RF, Loyola Filho AI, Meira AJ, Luz ZMP. O contexto organizacional do Sistema de Informações sobre Nascidos Vivos segundo profissionais de saúde do nível municipal. Rev Bras Saúde Materno Infantil. 2014;14(2):165-72. doi: 10.1590/S151938292014000200006
  » https://doi.org/10.1590/S151938292014000200006
• ²¹
  Santos, MC, Marin, HF. Análise do uso de um sistema informatizado por gestores hospitalares. Acta Paul Enferm. 2018;31(1):1-6. doi: 10.1590/1982-0194201800002
  » https://doi.org/10.1590/1982-0194201800002
• ²²
  Adokiya MN, Awoonor-Williams JK, Beiersmann C, Müller, O. Evaluation of the reporting completeness and timeliness of the integrated disease surveillance and response system in northern Ghana. Ghana Med J. 2016;50(1):3-8. doi: 10.4314/gmj.v50i1.1
  » https://doi.org/10.4314/gmj.v50i1.1
• ²³
  Nóbrega AA, Teixeira AMS, Lanzieri TM. Avaliação do sistema de informação do Programa de Imunizações (SI-API) [Internet]. 2010[cited 2018 Jan 02]. Available from: http://www.cadernos.iesc.ufrj.br/cadernos/images/csc/2010_1/artigos/Modelo%20Livro%20UFRJ%2014-a.pdf
  » http://www.cadernos.iesc.ufrj.br/cadernos/images/csc/2010_1/artigos/Modelo%20Livro%20UFRJ%2014-a.pdf
• ²⁴
  Moreno RA. Interoperabilidade de Sistemas de Informação em Saúde. Journal of Health Informatics [Internet]. 2016[cited 2018 Jan 02];8(3):I. Available from: http://www.jhi- sbis.saude.ws/ojs-jhi/index.php/jhi-sbis/article/viewFile/502/268
  » http://www.jhi- sbis.saude.ws/ojs-jhi/index.php/jhi-sbis/article/viewFile/502/268
• ²⁵
  Lima KWSD, Antunes JLF, Silva ZPD. Percepção dos gestores sobre o uso de indicadores nos serviços de saúde. Saúde e Sociedade [Internet]. 2015;24:61-71. doi: 10.1590/S0104-12902015000100005
  » https://doi.org/10.1590/S0104-12902015000100005
• ²⁶
  Merhy EE. Saúde: a cartografia do trabalho vivo. 2ª ed. São Paulo: Hucitec; 2005. Minas Gerais. Secretaria de Estado de Saúde de Minas Gerais. IN: Adscrição e população dos municípios por macrorregiões e microrregiões de saúde. Belo Horizonte: Secretaria de Estado de Saúde de Minas Gerais, 2014. 24 p.
• ²⁷
  Paulino VCP, Bezerra ALQ, Branquinho NCSS, Paranaguá TTB. Ações de educação permanente no contexto da estratégia saúde da família. Rev Enferm UERJ [Internet]. 2012[cited 2018 Jan 02];20(3);312-6. Available from: http://www.facenf.uerj.br/v20n3/v20n3a15.pdf
  » http://www.facenf.uerj.br/v20n3/v20n3a15.pdf
• ²⁸
  Ministério da Saúde (BR). Secretaria de Vigilância em Saúde. PNI- Referência Mundial. Programa Nacional de Imunização, 2003.

Edited by

EDITOR IN CHIEF: Antonio José de Almeida Filho

ASSOCIATE EDITOR: Fátima Helena Espírito Santo

Publication Dates

• Publication in this collection
  01 June 2020
• Date of issue
  2020

History

• Received
  10 Apr 2019
• Accepted
  09 July 2019