Validation of indicators for monitoring the quality of prenatal care

Migoto, Michelle Thais; Oliveira, Rafael Pallisser de; Freire, Márcia Helena de Souza

Michelle Thais Migoto

Universidade Federal do Paraná. Curitiba, PR, Brasil.

https://orcid.org/0000-0002-8546-8694

Rafael Pallisser de Oliveira

Universidade Positivo. Curitiba, PR, Brasil.

https://orcid.org/0000-0002-6997-399X

Márcia Helena de Souza Freire

Universidade Federal do Paraná. Curitiba, PR, Brasil.

https://orcid.org/0000-0003-3941-3673

Corresponding author: Michelle Thais Migoto. E-mail: michellemigoto@gmail.com

AUTHOR’S CONTRIBUTIONS

Study design. Michelle Thais Migoto. Marcia Helena de Souza Freire

Data collection or production. Michelle Thais Migoto.

Data analysis. Michelle Thais Migoto. Rafael Pallisser de Oliveira.

Interpretation of results. Michelle Thais Migoto. Rafael Pallisser de Oliveira. Marcia Helena de Souza Freire

Writing and critical review of the manuscript. Michelle Thais Migoto. Rafael Pallisser de Oliveira. Marcia Helena de Souza Freire

Approval of the final version of the manuscript. Michelle Thais Migoto. Rafael Pallisser de Oliveira. Marcia Helena de Souza Freire

Responsibility for all aspects of the content and integrity of the published article. Michelle Thais Migoto. Rafael Pallisser de Oliveira. Marcia Helena de Souza Freire

ASSOCIATED EDITOR
https://orcid.org/0000-0001-7366-077X
Beatriz Rosana Gonçalves de Oliveira Toso

SCIENTIFIC EDITOR
https://orcid.org/0000-0002-1522-9516
Ivone Evangelista Cabral

Indicators	AR of domain clarity	CVR	AR of domain relevance	CVR	CVI of item clarity	CVI of item relevance
B.1 Health professionals	72.70	0.45	72.70	0.45	0.82	0.73
B.2 Health professionals: physicians	81.80	0.64	90.90	0.82	0.91	0.82
B.3 Health professionals: obstetric nurses	90.90	0.82	90.90	0.82	1.00	0.91
B.4 Health professionals: pediatricians	81.80	0.64	100.00	1.00	1.00	0.91
B.5 Health professionals: pediatric or neonatal nurses	90.90	0.82	100.00	1.00	1.00	0.91
B.6 Health establishment: healthcare unit	63.60	0.27	90.90	0.82	0.91	0.91
B.7 Health establishment: mobile emergency care service	72.70	0.45	81.80	0.64	0.82	0.82
B.8 Expenditures	54.50	0.09	81.80	0.64	0.73	0.82
B.9 Expenses	81.80	0.64	81.80	0.64	0.82	0.82
B.10. Investments	81.80	0.64	90.90	0.82	0.82	0.82
B.11 Cost	72.70	0.45	81.80	0.64	0.82	0.82
C.1 Coverage of the health information system for primary care.	100.00	1.00	100.00	1.00	0.91	0.91
C.2 Training in the health information system for primary care.	81.80	0.64	90.90	0.82	0.82	0.91
C.3 Health information system completeness for primary care.	90.90	0.82	90.90	0.82	0.82	0.91
D.1 Pregnant woman with prenatal care	90.90	0.82	90.90	0.82	1.00	1.00
D.2 Early initiation of prenatal care	90.90	0.82	90.90	0.82	1.00	1.00
D.3 Pregnant woman with stratified risk	90.90	0.82	81.80	0.64	1.00	1.00
D.4 Number of prenatal visits	90.90	0.82	81.80	0.64	1.00	1.00
D.5 Number of prenatal + puerperium visits	81.80	0.64	90.90	0.82	1.00	1.00
D.6 Number of prenatal visits + all exams	81.80	0.64	90.90	0.82	1.00	1.00
D.7 Number of prenatal + puerperium visits + all exams	81.80	0.64	90.90	0.82	1.00	1.00
D.8 Number of prenatal + puerperium visits + all exams + diphtheria and tetanus vaccination + HIV test	81.80	0.64	90.90	0.82	1.00	1.00
D.9 Pregnant women immunized with diphtheria and tetanus vaccine.	90.90	0.82	81.80	0.64	1.00	1.00
E.1 Congenital syphilis	81.80	0.64	90.90	0.82	0.82	1.00
E.2 Neonatal tetanus	90.90	0.82	90.90	0.82	1.00	1.00
E.3 Vertical HIV transmission	90.90	0.82	90.90	0.82	1.00	1.00
E.4 Pregnant women accompanied by the supplementary health system (health plans)	81.80	0.64	81.80	0.64	0.73	0.91
E.5 Abortion treatment	72.70	0.45	90.90	0.82	0.82	0.91
E.6 Teenage pregnancy	81.80	0.64	81.80	0.64	0.91	1.00
E.7 Early diagnosis of congenital anomalies	90.90	0.82	90.90	0.82	0.91	1.00
F.1 Maternal mortality ratio	100.00	1.00	100.00	1.00	1.00	1.00
F.2 Early neonatal mortality rate	100.00	1.00	100.00	1.00	1.00	1.00
F.3 Late neonatal mortality rate	100.00	1.00	100.00	1.00	1.00	1.00
F.4 Neonatal mortality rate	100.00	1.00	100.00	1.00	1.00	1.00
F.5 Perinatal mortality rate	90.90	0.82	100.00	1.00	0.91	1.00
F.6 Fetal mortality rate	100.00	1.00	100.00	1.00	0.91	1.00
F.7 Infant mortality rate	100.00	1.00	100.00	1.00	1.00	1.00
F.8 Abortion rate	100.00	1.00	100.00	1.00	0.91	1.00

Indicators	Judges’ suggestions in terms of clarity and relevance of the item to the domain	Adaptations
B.1	- The item label does not refer to its contents for calculation.	Item deleted.
	- Work individually by quantifying gynecologist and obstetric nurse.
	- In primary care, the doctor and general nurse guarantee care, but this item does not contemplate the other levels of care.
	- Most pregnant women are at habitual or intermediate risk and are not seen by specialists.
B.2	- The most accurate information would be to supplement the name of the indicator for gynecologists and obstetricians.	Adjusted the name of the indicator to gynecologist and obstetrician.
B.2	- The indicator does not include the three levels of care.
B.4	- In neonatal units, the physician needs to be a neonatologist.	Adjusted the name and definition of the indicator by including the neonatologist.
B.4	- The indicator does not include the three levels of care.
B.6	- Include the family health strategies in the evaluation.	Adjusted the name to primary health care to include all available types of units and the definition of the indicator by including the word number.
	- It is not clear why the line of care measures the number of primary health care units.
	- Indicator is unclear.
B.7	- Describe better what you need and if you want to find it.	Adjusted the name to health establishment - emergency care network, and the definition of the indicator according to the network premises.
B.7	- Use the definition from the State Health Plan 2020-2023.
B.8	- Text is not clear, it could be replaced by the total effective cost.	Adjusted the indicator’s name to revenues collected for financing the Maternal and child health care line and its definition specifying the disbursement by each sphere of government.
	- Clarify what is referred to as expense acquisition.
	- Specify the spheres of government, considering that the funding is tripartite, as defined in the Constitution and the State Health Plan 2020-2023.
B.9	- Separate funding figures for each sphere of government.	Adjusted definition considering hospital and outpatient care specific to obstetric and neonatal care.
	- The financing of medium and high complexity is a tripartite responsibility. Assistance is funded by the National Health Fund, State Health Fund, and Municipal Health Funds. Use the definition of the State Health Plan 2020-2023.
	- Consider that the transference occurs through contracts to philanthropic entities, consortiums, city halls, and municipal funds.
	- Complex to include in an evaluation process.
B.10	- The financing of medium and high complexity is a tripartite responsibility. Use the definition of the State Health Plan 2020-2023.	Adjusted definition considering investments for the expansion of obstetric and neonatal care.
B.10	- Specify what is invested in obstetric and neonatal care.
B.11	- The evaluation was very broad with this item.	Item deleted after having considered the changes in items B.8 to B.10.
B.11	- The financing of medium and high complexity is a tripartite responsibility. Use the definition of the State Health Plan 2020-2023.
C.2	- Specify the number of professionals who have taken the training or would need to take it.	Adjusted the item’s definition to the number of professionals who received training.
D.5	- Make the variables involved in the calculation clearer.	Adequate the name and definition of the indicator considering only puerperium.
D.5	- Since prenatal care was quantified in the previous item, keep this indicator to puerperal consultations only.
D.6	- Make the variables involved in the calculation clearer.	Adequate the definition, including the types of examinations.
D.6	- It was not clear what type of exams, include the laboratory and/or imaging exams.
D.7	- Make the variables involved in the calculation clearer.	Adequate the definition, including the types of examinations.
D.7	- It was not clear what type of exams, include the laboratory and/or imaging exams.
D.8	- Make the variables involved in the calculation clearer.	Adequate the definition, including the types of examinations.
D.8	- Make it clear they are laboratory and imaging exams.
D.9	- Make the variables involved in the calculation clearer.	Suitable name and definition, including all recommended vaccines.
D.9	- Include immunization with influenza, diphtheria and tetanus, and hepatitis B, which are also vaccines recommended for pregnant women.
E.1	- Separate syphilis diagnosis from syphilis deaths.	Adequate the definition for syphilis diagnosis, considering only Congenital Syphilis.
E.4	- Make the variables involved in the calculation clearer.	Adjusted the definition, changed the calculation base to the numerator and CNES source, and the denominator by live births from the SINASC source.
E.4	- Denominator adjusted for live births, with SINASC source.
E.5	- Relate to the number of pregnant women.	Adequate to the title and definition of the item.
E.5	- Review clarity in the definition regarding the calculation.	Adequate to the title and definition of the item.
E.6	- Make the variables involved in the calculation clearer.	Adequate definition and calculation basis per thousand live births.

Scale statistics	Entries B.2 to B.10	Activities C.1 to C.3	Outputs D.1 to D.9	Results E.1 to E.7	Impact F.1 to F.8
Cronbach’s alpha coefficient	0.957	0.863	0.980	0.862	0.957
Average	63.09	21.09	68.36	51.45	61.64
Variance	125.491	10.891	41.255	28.673	26.255
Standard deviation	11.202	3.300	6.423	5.355	5.124

Scale statistics	Structure	Process	Results
Cronbach’s alpha coefficient	0.936	0.980	0.943
Average	84.18	68.36	113.09
Variance	147.164	41.255	95.291
Standard deviation	12.131	6.423	9.762

STRUCTURE
Domains	Name	Definition	Numerator	Source	Denominator	Source
B. ENTRY	B.2 Health professionals - gynecologists and obstetricians	Number of active gynecologists and obstetricians in a given region divided by the total population and multiplied by 100.	N	CNES	Inhabitants	IBGE
	B.3 Health professionals - obstetric Nurses	Number of active obstetric nurses in a given region divided by the total population and multiplied by 100.	N	CNES	Inhabitants	IBGE
	B.4 Health professionals - pediatricians and neonatologists	Number of active pediatricians and neonatologists in a given region divided by the total population and multiplied by 100.	N	CNES	Inhabitants	IBGE
	B.5 Health professionals - pediatric and neonatologist nurses	Number of active pediatric nurses or neonatologists in a given region divided by the total population and multiplied by 100.	N	CNES	Inhabitants	IBGE
	B.6 Health care establishment - primary health care	Number of units that provide basic and integral care to a population, in a scheduled or unscheduled manner, in basic specialties, and may offer dental care and other higher-level professionals. The assistance must be permanent and provided by a general practitioner or specialist in these areas. It may or may not offer: auxiliary services of diagnosis and therapy (SADT) and 24-hour emergency care. Divided by the total population of the region and multiplied by 100.	N	CNES	Inhabitants	IBGE
	B.7. Health care establishment - emergency care network	Number of land, air, or waterway vehicles used to receive the user, classify the assistance risk according to the degree of urgency, establish the definitive diagnosis, apply the therapeutic measures of the assistance point, and refer the user for therapeutic continuity. Divided by the total number of inhabitants in the region and multiplied by 100.	N	CNES	Inhabitants	IBGE
	B.8. Revenues collected for financing the maternal and child health care line	Sum of financial resources (in reais) disbursed by the municipalities, state, and Union for the payment of expenses, costs, and investments of the Maternal and Child Health Care Line. Considering for municipalities and states, 15% and 12% of tax collection, and for the Union the value of the previous year plus the variation of the Gross Domestic Product respecting the Annual Budget Law (consider the EC 95 of 2016 that freezes spending for 20 years).	R$	Contractualization	Inhabitants	IBGE
	B.9. Expenses	Financial resource disbursed (in Brazilian Reais) by the Health Funds (municipal, state, and federal) for the maintenance of outpatient and hospital care services (contracts with philanthropic entities, consortiums, and municipalities) aimed at providing obstetric and neonatal care.	R$	Contractualization	Inhabitants	IBGE
	B.10. Investments	Financial resource disbursed (in Reais) by Health Funds (municipal, state, and federal) to expand the provision of outpatient and hospital care services for obstetric and neonatal care. Example: purchase of equipment, expansion of physical structure, acquisition of real estate, etc.	R$	Contractualization	Inhabitants	IBGE
C. ACTIVITIES	C.1. SISAB Coverage	Number of pregnant women registered in the information system for prenatal care, SISAB, by the municipality, divided by the number of live births and multiplied by 100.	N	SISAB	Live Births	SINASC
	C.2 SISAB Training	Number of health professionals who perform prenatal consultations and who received training in the format of training to raise awareness of the importance of using the information system to record standardized information according to the protocol established for the collection of data on the care provided. Divided by the total number of professionals who perform this service and multiplied by 100.	N	Contractualization	Total professionals	CNES
	C.3. SISAB Completeness	Completeness of the variables entered into the information system according to the variables collected that reflect the behaviors performed during prenatal care, divided by the total number of pregnant women and multiplied by 100.	N (filled in)	SISAB	Total pregnant women	SISAB
PROCESS
D. EXIT	D.1 Pregnant women with prenatal care	Proportion of pregnant women who were attended for prenatal care, regardless of the number of consultations, divided by the total number of pregnant women and multiplied by 100.	N	SISAB	Total pregnant women	SISAB
	Early start of prenatal care	Proportion of pregnant women who had their first prenatal visit before the 12th week of gestational age, divided by the total number of pregnant women and multiplied by 100.	N	SISAB	Total pregnant women	SISAB
	D.3. Pregnant woman with stratified risk	Proportion of pregnant women who were assessed during prenatal visits and had their risk-stratified appropriately as usual risk, intermediate risk, or high risk by the Basic Health Unit, divided by the total number of pregnant women and multiplied by 100.	N	SISAB	Total pregnant women	SISAB
	D.4 Number of prenatal consultations	Proportion of pregnant women in relation to the number of prenatal consultations performed during the gestational period, according to the adequacy: 2 consultations in the first trimester: 2 consultations in the second; 3 consultations in the third; and one consultation in the puerperium. Divided by the total number of pregnant women and multiplied by 100.	N	SISAB	Live births	SINASC
	D.5 Number of puerperal consultations	Proportion of women who had a postpartum puerperal visit, divided by the total number of pregnant women and multiplied by 100.	N	SISAB	Total pregnant women	SISAB
	D.6 Number of prenatal appointments + All Tests	Proportion of pregnant women with the appropriate number of prenatal visits and performance of all laboratory and imaging tests, divided by the total number of pregnant women and multiplied by 100.	N	SISAB	Total pregnant women	SISAB
	D.7 Number of Prenatal + Puerperium visits + All Exams	Proportion of pregnant women according to the appropriate number of prenatal and puerperium visits and the performance of all laboratory and imaging exams.	N	SISAB	Total pregnant women	SISAB
	D.8. Number of prenatal + puerperium + all exams + diphtheria and tetanus vaccination + hiv test	Proportion of pregnant women who had the appropriate number of prenatal visits, puerperium visits, had all laboratory and imaging exams, had antitetanic vaccination, and underwent HIV testing. Divided by the total number of pregnant women and multiplied by 100.	N	SISAB	Total pregnant women	SISAB
	D.9 Immunized pregnant women	Proportion of pregnant women who have updated their vaccination schedule with all the recommended vaccines such as anti-tetanus, influenza, and hepatitis B. Divided by the total number of pregnant women and multiplied by 100.	N	SISAB	Total pregnant women	SISAB
RESULTS
E. RESULTS	E.1 Congenital syphilis rate	Number of live births with a diagnosis of congenital syphilis, divided by total live births, multiplied by 100,000 live births.	N	HIS	Live births	SINASC
	E.2 Neonatal tetanus	Proportion of live births with a diagnosis of neonatal tetanus, divided by the total number of live births and multiplied by 100.	N	HIS	Live births	SINASC
	E.3. HIV vertical transmission	Proportion of live births diagnosed with HIV by vertical transmission, divided by the total number of live births and multiplied by 100.	N	HIS	Live births	SINASC
	E.4 Pregnant women followed by the supplementary health system	Proportion of pregnant women who receive prenatal care through the Supplementary Health System (health plan), divided by the total number of live births and multiplied by 1,000 live births.	N	CNES	Live births	SINASC
	E.5 Treatment of early fetal loss	Rate of hospital admissions for treatment of pregnant women in early fetal loss condition, with gestational age less than 22 weeks, divided by the number of live births plus fetal deaths and multiplied by 1,000 live births.	N	HIS	Live births + fetal deaths	SINASC SIM
	E.6 Teenage pregnancy	Proportion of adolescents, aged under 20, who had a confirmed pregnancy diagnosis, multiplied by the total number of live births and multiplied by 1,000 live births.	N	SINASC	Live births	SINASC
	E.7 Early diagnosis of congenital anomalies	Proportion of early diagnosis, during the gestational period, of fetuses with presence of congenital anomalies confirmed by imaging examinations, divided by the total number of live births and multiplied by 100.	N	SISAB	Live births and fetal deaths	SINASC SIM
F. IMPACT	F.1 Maternal mortality ratio	Rate that expresses the maternal deaths related or aggravated by the gravidic-puerperal process, pregnancy, childbirth and up to the 42nd day postpartum. Regardless of the duration of pregnancy or the place where it occurred, per 100,000 live births.	N	YES	Live births	SINASC
	F.2. Early neonatal mortality rate	Rate that expresses the death of live births up to the 6th full day of life, born with a gestational age of 22 weeks or more per 1,000 live births.	N	YES	Live births	SINASC
	F.3 Late neonatal mortality Rate	Rate that expresses the death of live births, between the 7th and 28th day of life, born with a gestational age of 22 weeks or more per 1,000 live births.	N	YES	Live births	SINASC
	F.4. Neonatal mortality rate	Rate that expresses the death of live births until the 28th day of life, born with gestational age from 22 weeks per 1,000 live births.	N	YES	Live births	SINASC
	F.5. Perinatal mortality rate	Rate that expresses fetal death from the 22nd week of gestation, up to the 6th full day of life of births with gestational age 22 weeks and older per 1,000 live births.	N	YES	Live births	SINASC
	F.6. Fetal mortality rate	Rate that expresses fetal deaths from the 22nd week of gestation onwards, occurring intrauterine per 1,000 live births.	N	YES	Live births and fetal deaths	SINASC SIM
	F.7. Infant mortality rate	Rate that expresses the death of children up to 1 year old, or 365 days, per 1,000 live births.	N	YES	Live births	SINASC
	F.8. Abortion rate	Rate that expresses the mortality of fetuses with gestational age less than 22 weeks, or with weight less than 500g, or height less than 25 cm per 1,000 live births.	N	YES	Live births and fetal deaths	SINASC SIM

[1] Corresponding author: Michelle Thais Migoto. E-mail: michellemigoto@gmail.com

Brasil

Brasil

Abstract

Objective

Method

Results

Conclusion

Implications for Practice