Methods
In this study we assess available indicators currently used for health and non-health sector policy and planning in government programs. They thus have a defined monitoring purpose which is re-assessed for suitability to the CoC framework.
Indicator extraction
We used the routine district health information system (DHIS) to extract relevant health system indicators. The DHIS monitors health programmes, track patients and map service availability in the health system in SA [6]. The National Indicator Data Set (NIDS) within the DHIS contains indicators of service inputs, processes, outputs and outcomes (where relevant) extracted for this study, for the reference period April 2017–March 2019 [7]. For social determinants of health/intersectoral factors as outlined in Fig. 1, we assessed datasets on the Statistics SA Nesstar portal and selected the General Household Survey (GHS) (reference year 2017) as the most suitable source. The GHS is annually collected and contains data on all intersectoral factors, which are used in policy and planning in SA [8, 9]. All GHS data can be obtained from the DataFirst Portal of the University of Cape Town in SA [10].
Indicator evaluation
The health service indicators from the DHIS were evaluated for suitability to the framework based on their current monitoring purposes and recommendations from existing guidelines. These guidelines included:
Annual performance plans of the Department of Health in SA.
Guidelines for maternity care in South Africa.
The strategic objectives of the global network to improve Quality, Equity and Dignity in maternal, newborn and child health [11].
Resources exploring the WHO Quality of Care Framework for maternal and new-born health [12, 13].
Global Review of Key Interventions related to reproductive, maternal, newborn and child health [14].
Guidelines for positive birth experience with a focus on monitoring Intrapartum care [15].
Quality of care at primary (Ideal Clinic Realization and Maintenance Program) and hospital (National Core Standards) level in SA [16, 17].
For social determinants of health, we relied on literature focusing on the relationship between interventions and maternal health outcomes. We also relied on recommendations by the WHO and Commission on Social Determinants of Health [18], conceptual framework of the social determinants of health [19], and frameworks for practice at country level [20]. The evaluation of indicators also revealed outstanding gaps in measuring interventions on the framework, which we describe in this study.
Results
Indicator set
In Fig. 2 we present a set of 38 indicators that were extracted and evaluated from the DHIS and GHS (27 indicators from the health system and 11 for the intersectoral factors). The figure also describes measurement gaps per intervention package of the CoC. As Fig. 2 shows, indicators are available for most of the intervention packages on the CoC framework. The exceptions were danger sign recognition and emergency preparedness, healthy behaviour promotion and indicators for emergency pregnancy care. The lack of indicators demonstrates unavailable services and/or poor monitoring by the health system. Sometimes indicators are available that do not directly measure maternal health outcomes. Figure 2 shows proxies such as food fortification compliance rates (Indicator 3) used by the health system at community level. Other proxies include Ideal Clinic status (Indicator 20) and national core standards (Indicator 21), which are summary measures of quality of care at facility level. Where only proxies are available, we recommend health information system improvements to measure and integrate measures that are more directly related to maternal health outcomes.
Even where indicators are available, measurement improvements can be made in order to monitor each intervention package comprehensively. Each intervention package consists of more than one intervention, as shown in Fig. 1. For example, while Reproductive Care at district hospital may include timely termination of pregnancy indicators, there is a gap in monitoring post-abortion care and treatment of complicated STIs as part of the package. Thus, more research is needed to assess the extent to which the health system provides services within each intervention package of the CoC. New health system interventions, such as Human Papillomavirus (HPV) vaccinations for school going girls and health promotion through mobile phones (MomConnect program), should be monitored through the DHIS. This will improve the comprehensiveness of the data set and ease of monitoring the CoC in the health system.
Quality of care was an under-measured aspect of interventions such as antenatal care visits, normal and Caesarean deliveries, and postnatal visits. There is a need for intervention specific qualities of care indicators, as exemplified by retest rates for HIV positive clients during antenatal care (Indicator 11, Fig. 2). The health system runs parallel quality of care systems for maternal health, particularly the confidential maternal mortality audits [21]. We recommend the establishment of routine measures from these sources for integration into the DHIS. Routine quality of care monitoring should also include reporting of safety incidents and experience of care surveys disaggregated by population groups.
While indicators are available for the intersectoral factors in the framework, we observed gaps in monitoring indoor air pollution, maternal nutrition counselling, and women’s empowerment for decision making and demand for healthcare. Like health system interventions, each intersectoral factor could be measured by more than one indicator. For instance, in the water and sanitation intervention package, the GHS had variables that could assess safety of water, infrastructure and basic sanitation (Indicators 28–30). While educational achievement indicators may be straightforward, factors such as empowerment and nutrition are more multifaceted. Thus, a variety of indicators can be isolated for their measurement, depending on data availability.
In summary, Fig. 2 is the indicator tool which provides a description of available indicators and gaps that need to be addressed to monitor the CoC for maternal health. The gaps identified should not preclude use of the tool to assess the nature and extent of provision of services along the CoC for maternal health in future studies. The improvement and validation of indicators in maternal health should be a continuous process, tied to evolving policies and information system improvements [22].
Adequacy assessment
In Fig. 3 indicators are grouped according to adequacy dimensions, and the information in Fig. 2 used to subjectively assess the level to which intervention packages can be measured by available indicators. We assign “partial” (orange) measurement if indicators are available but there are measurement gaps identified. When assigned “no” (red) if no indicators or proxies were identified from the data sources. And we assigned “yes” (green) if, according to literature and existing guidelines, there are indicators available to measure the intervention package. Availability of indicators for an intervention package does not preclude future rigorous validation processes and iterations; this is a normal process within the health information system that is encouraged.
All dimensions of the CoC can be measured by current indicators, although gaps remain within specific intervention packages. Data gaps were most prevalent in the care access and utilization dimension, where 40% (6/15) of intervention packages had no indicators available. Dimensions of quality and linkages of care can only be partially measured; while only 40% (2/5) of social determinants of health domains have available indicators. In general, the GHS and the DHIS provide indicator data for measurement and monitoring of the majority (74%) of CoC intervention packages (17/23) as defined by the framework in Fig. 1.
