- Research note
- Open access
- Published:
The effect of maternity waiting homes on perinatal mortality is inconclusive: a critical appraisal of existing evidence from Sub-Saharan Africa
BMC Research Notes volume 14, Article number: 86 (2021)
Abstract
Objectives
To assess the appropriateness of the statistical methodology used in a recent meta-analysis investigating the effect of maternity waiting homes (MWHs) on perinatal mortality in Sub-Saharan Africa.
Results
A recent meta-analysis published in BMC Research Notes used a fixed-effect model to generate an unadjusted summary estimate of the effectiveness of MWHs in reducing perinatal mortality in Africa using ten observational studies (pooled odds ratio 0.15, 95% confidence interval 0.14–0.17). The authors concluded that MWHs reduce perinatal mortality by over 80% and should be incorporated into routine maternal health care services. In the present article, we illustrate that due to the contextual and methodological heterogeneity present in existing studies, the authors’ conclusions about the effectiveness of MWHs in reducing perinatal mortality were likely overstated. Additionally, we argue that because of the selection bias and confounding inherent in observational studies, unadjusted pooled estimates provide little causal evidence for effectiveness. Additional studies with robust designs are required before an appropriately designed meta-analysis can be conducted; until then, the ability to draw causal inferences regarding the effectiveness of MWHs in reducing perinatal mortality is limited.
Introduction
There is renewed interest in maternity waiting homes (MWHs) as a strategy to increase facility-based obstetric care and reduce maternal and perinatal mortality. MWHs provide temporary accommodation near a health facility prior to birth for women with high-risk pregnancies and/or living far away from health facilities [1]. Several African and Asian countries are investing in MWH scale-up as part of their national health strategies [2,3,4,5].
While observational studies have reported some benefits [6,7,8] there is still insufficient evidence that MWHs reduce mortality [3, 9] or impact newborn outcomes [10]. The quality of available evidence is also low, yet a recently published meta-analysis has drawn strong conclusions in favour of MWHs. An 82.5% reduction in perinatal mortality was attributed to MWH use and consequently, the authors recommend that “all pregnant women be admitted to MWHs before delivery” [11]. This review has been cited repeatedly to advocate MWH use [12,13,14,15,16,17,18,19] despite limitations in the original studies and the review. In this research note, we critically assess the analytic approach employed in this meta-analysis [11] and discuss important considerations when pooling observational data on complex interventions such as MWHs.
Main text
Features of the recent meta-analysis on MWHs and perinatal mortality
The meta-analysis by Bekele and colleagues included ten observational studies from six countries [7, 8, 20,21,22,23,24,25,26,27] after 31% (n = 73/236) were excluded because full texts were unavailable [11]. Most of these studies included women who delivered at hospitals offering some level of comprehensive emergency obstetric care [7, 8, 20, 21, 23, 26, 27]. The number of perinatal deaths abstracted for MWH users and women admitted directly to hospitals were reported [11], but there were abstraction errors for two studies [21, 27] and some overlap in data from two studies conducted at Attat Hospital in Ethiopia [8, 24]. Three studies [7, 8, 23] reported stillbirths but not early neonatal deaths and in two others it was difficult to distinguish outcomes for MWH users and non-users [22, 26]. The authors used a fixed-effect model to generate an unadjusted pooled odds ratio estimating the association between MWH use and perinatal mortality. The authors reported conducting sub-group analyses by study design due to the high degree of heterogeneity detected (I2 = 97%), but no sub-group estimates were reported or discussed [11].
Methodological considerations
Choice of model for meta-analysis of complex interventions
Decisions about which statistical model to use in a meta-analysis depends on the type of effect expected and the goal of the analysis [28]. Using a fixed-effect model conveys the belief that there is one common true effect size estimated by all individual studies, and that differences in observed effect sizes are a result of sampling error [28,29,30]. When a fixed-effect model is used, the goal is not to extrapolate findings beyond the included set of studies [28, 31]. In contrast, random-effects models are suitable when a distribution of true effects exists, and included studies represent a random sample of possibilities; in this case, findings may be generalized to other similar scenarios [29].
Heterogeneity is the variability in true effects underlying different studies [32, 33]. The I2 statistic (indicates the proportion of variance in observed effects due to variance in true effects and is a “measure of inconsistency”) [32, 33] is often used to decide whether sufficient heterogeneity exists to run a random-effects model but this is not recommended as it has low power [28]. What may be more useful is to assess whether it is likely that studies included are “functionally identical” [29] as assumed under a fixed-effect model. Widespread differences in participant characteristics, intervention designs, settings and outcomes, make the absence of heterogeneity unlikely [28, 33, 34]. Public health interventions are even less likely to be homogenous; they often have interacting components targeting multiple groups, accommodate flexible delivery, and are embedded within complex systems [35]. Given the considerable variation in MWH implementation [36] random-effects models are likely more suitable for meta-analyses involving MWHs.
Alone, however, the estimated mean effect provides an incomplete picture [37] as how effect sizes vary under different conditions and populations is often of interest [38]. With sufficient numbers of studies, sub-group analysis within a few important, pre-specified subgroups (to avoid issues with multiple testing) [28, 39] is one way to explore heterogeneity. Results need to be interpreted cautiously due to the observational nature of the analysis [30].
Finally, in fixed-effect models, larger studies are weighted more heavily [30] as they have smaller sampling error and higher precision. The pooled estimate reported by Bekele et al. [11] was, thus, largely influenced by one study [8] (weight: ~ 74%). In random-effects models, each study provides unique information about the distribution of true effect sizes, therefore weighting is more equivalent [29].
Methodology for the present study
In light of the methodological considerations outlined above, we sought to critically assess the methodology employed by Bekele and colleagues, and explore whether heterogeneity may be better accounted for using a random-effects model. For illustrative purposes, we re-abstracted information from the seven studies [7, 8, 20, 21, 23, 25, 27] from the review that had appropriate data available, as well as three additional eligible studies [40,41,42] identified from reference lists (Table 1). We calculated a summary estimate in Review Manager version 5.4 using a random-effects model for stillbirths and perinatal mortality separately, using unadjusted outcome events reported for MWH users and women directly admitted to hospital.
To explore heterogeneity, we conducted sub-group analysis for stillbirths to demonstrate how country and type of managing authority may change effect estimates. While no definitive conclusions can be made, the results provide insight into sources of heterogeneity.
Random effects model findings and implications
The pooled estimates are suggestive of an association between MWH use and lower stillbirths (pooled Risk Ratio [RR] = 0.39, 95% Confidence Interval [CI]: 0.19 to 0.80; nine studies; 43,385 participants) and to a lesser extent lower perinatal mortality (pooled RR = 0.69, 95% CI: 0.52 to 0.93; six studies; 8,492 participants) (Fig. 1). The comparative similarity in weights calculated for stillbirths point to higher between-studies than within-study variance [29]; this is also reflected in the high values of I2 (I2 = 93%, indicating 93% of the total variation is attributable to heterogeneity [33]) and τ2 (τ2 = 0.97).
The lower I2 values suggest that there is more consistency among studies conducted in Ethiopia (I2 = 86%) and even more among those conducted in other countries (I2 = 35%) than when all studies are considered together (I2 = 93%) (Table 2).
Overall, the reduction in the between-study variance for country sub-groups (τ2 = 0.10—0.28 subgroups versus τ2 = 0.97 all studies) suggests that between-country contextual differences could be one source of heterogeneity. The between-study variance was also lower when the type of managing authority was considered. There was more consistency among studies with government-run facilities (I2 = 42% τ2 = 0.47) than overall (I2 = 93% τ2 = 0.97). While the test for subgroup differences was not statistically significant, the existence of heterogeneity due to managing authority cannot be ruled out.
Conclusion
Given the complexity of MWH interventions and the variation in contextual factors, heterogeneity must be appropriately addressed when conducting meta-analysis on MWH effects. More robustly designed studies with adequate reporting are needed to enable exploration of heterogeneity in effects. Careful consideration of the quality of evidence and specific conditions required to improve outcomes for women and babies is required before implementing further scale-up of MWHs.
Limitations
Firstly, meta-analyses produce “observational” results even if randomized controlled-trials (RCTs) are included as random allocation is not preserved [43]. Observational studies, where assignment to comparison groups is not random, are considered to be at even higher risk for selection bias and confounding than RCTs [34]. While a random effects model is more suitable for MWH studies, the pooled estimates presented here may still be compromised by bias and confounding inherent to observational designs. Future analyses may consider meta-regression to assess the effect of study-level covariates on effect sizes [28] when at least ten studies are available [30]. If available, adjusted analyses with comparable adjustment variables can also be used to generate adjusted pooled estimates. Ideally, however, additional individual studies using robust designs are required for results from meta-analyses to be more informative. RCTs are generally accepted as providing the highest quality evidence [34] if well designed, conducted and reported. Where it is not feasible or ethical to conduct trials, longitudinal studies with careful participant selection, adequate confounder information, sufficient follow-up levels that analyse data suitably may be acceptable alternatives. Availability of additional studies would also improve estimates of between-study variance (τ2) which tend to be imprecise with fewer available studies [28]. Precision, in random effects models, is enhanced by the number of studies included, not study sample sizes [29].
Secondly, while there is an urgent need to improve methodological reporting in primary studies as illustrated in Table 1, there is an equal necessity to provide more details about MWH models themselves. Specifically, information on referral criteria and practices, community outreach activities to raise awareness and facilitate women’s access to MWHs, duration of stay and gestational age at admission, accommodation services available at MWHs, associated costs, level of monitoring of MWHs by health workers, the stage of labour when women are transferred to the health facility, and level of obstetric care available are needed to have a clear understanding of what is required to achieve reported reductions in mortality. This information could support a more comprehensive exploration of heterogeneity which we were not able to do due to the small number of studies and insufficient reporting in individual studies.
Thirdly, a better understanding of modifiable risk factors associated with stillbirths and neonatal deaths is required to assess the extent to which MWHs could potentially facilitate improved perinatal outcomes. A study investigating modifiable health-system risk factors reported that having to wait more than 10Â min to receive care after reaching a facility was associated with higher odds of stillbirth [44]. Other modifiable risk factors for stillbirths include maternal infections and prolonged pregnancy [45] which may be addressed through quality antenatal and intrapartum care, irrespective of MWH use. Reporting the type of stillbirth (intrapartum or antepartum) in future studies may help to disentangle stillbirths that can be averted through access to timely obstetric care (intrapartum stillbirths) and those which result from more long-term issues such as foetal growth restriction [45]. Only one of the studies included in the review made this distinction [21] making it impossible to explore.
Stillbirths and neonatal deaths are also a relatively rare event, which would make it difficult for studies with small sample sizes to detect meaningful changes in outcomes. Any reported associations between MWH use and stillbirth rates or perinatal mortality should, thus, be interpreted with caution.
A defining feature of systematic reviews is the use of clearly articulated, well-documented, comprehensive search strategies targeting multiple sources that are designed to capture the highest proportion of eligible studies in a transparent and reproducible fashion. In this way, bias is minimized and more reliable estimates are generated [30]. Since our aim was to illustrate the issues associated with statistical modelling, we did not repeat the search but largely relied on studies identified by Bekele and colleagues [11].
Finally, no firm conclusions can be drawn about the effectiveness of MWHs in reducing perinatal mortality from meta-analyses that do not employ methods that appropriately incorporate contextual variation and adequately consider the quality of included studies. The need to update evidence on MWH effectiveness using well-designed studies from diverse settings that reflect current levels of service use and quality remains.
Availability of data and materials
Not applicable.
Abbreviations
- CI:
-
Confidence interval
- MWH:
-
Maternity waiting homes
- RR:
-
Relative risk
References
World Health Organization. Maternity waiting homes: a review of experiences. Geneva: World Health Organization; 1996.
Ministry of Health Ethiopia. Guideline for the establishment of standardized maternity waiting homes at health centres/facilities. Addis Ababa: Ministry of Health Ethiopia; 2015.
World Health Organization. WHO Recommendations on health promotion interventions for maternal and newborn health. Geneva: World Health Organization; 2015.
Wild K, Kelly P, Barclay L, Martins N. Agenda setting and evidence in maternal health: connecting research and policy in timor-leste. Front Public Heal. 2015;3(September):1–9.
Lori JR, Perosky JE, Rominski S, Munro-Kramer ML, Cooper F, Kofa A, et al. Maternity waiting homes in Liberia: Results of a countrywide multi-sector scale-up. PLoS ONE. 2020;15(6):e0234785.
Lori JR, Perosky J, Munro-kramer ML, Veliz P, Musonda G, Kaunda J, et al. Maternity waiting homes as part of a comprehensive approach to maternal and newborn care: a cross-sectional survey. BMC Pregnancy Childbirth. 2019;19:228.
Braat F, Vermeiden T, Getnet G, Schiffer R, van den Akker T, Stekelenburg J. Comparison of pregnancy outcomes between maternity waiting home users and non-users at hospitals with and without a maternity waiting home: retrospective cohort study. Int Health. 2018;10:47–53.
Kelly J, Kohls E, Poovan P, Schiffer R, Redito A, Winter H, et al. The role of a maternity waiting area (MWA) in reducing maternal mortality and stillbirths in high-risk women in rural Ethiopia. BJOG. 2010;117(11):1377–83.
van Lonkhuijzen L, Stekelenburg J, van Roosmalen J. Maternity waiting facilities for improving maternal and neonatal outcome in low-resource countries. Cochrane database Syst Rev. 2012. https://doi.org/10.1002/14651858.CD006759.pub3.
Buser JM, Lori JR. Newborn outcomes and maternity waiting homes in low and middle-income countries: a scoping review. Matern Child Health J. 2016. https://doi.org/10.1007/s10995-016-2162-2.
Bekele BB, Dadi TL, Tesfaye T. The significant association between maternity waiting homes utilization and perinatal mortality in Africa: systematic review and meta-analysis. BMC Res Notes. 2019;12:13.
Bonawitz R, Mcglasson KL, Kaiser JL, Ngoma T, Fong RM, Biemba G, et al. Quality and utilization patterns of maternity waiting homes at referral facilities in rural Zambia : a mixed-methods multiple case analysis of intervention and standard of care sites. PLoS ONE. 2019;14(11):e0225523.
Getachew B, Liabsuetrakul T, Gebrehiwot Y. Association of maternity waiting home utilization with women’s perceived geographic barriers and delivery complications in Ethiopia. Int J Heal Plan Manag. 2019. https://doi.org/10.1002/hpm.2940.
Kaiser JL, Fong RM, Ngoma T, Mcglasson KL, Biemba G, Hamer DH, et al. The effects of maternity waiting homes on the health workforce and maternal health service delivery in rural Zambia:a qualitative analysis. Hum Resour Health. 2019;17:93.
Perosky JE, Lockhart MLMN, Musonda GK, Naggayi A, Lori JR. Maternity waiting homes as an intervention to increase facility delivery in rural Zambia. Int J Gynaecol Obstet. 2019. https://doi.org/10.1002/ijgo.12864.
Pujihartati SH, Wijaya M, Demartoto A. The importance of socializing maternity waiting home in the attempt of reducing maternal mortality rate in wonogiri regency. Adv Soc Sci Educ Humanit Res. 2020;389:116–20.
Tiruneh GT, Getu YN, Abdukie MA, Eba GG, Keyes E, Bailey PE. Distribution of maternity waiting homes and their correlation with perinatal mortality and direct obstetric complication rates in Ethiopia. BMC Pregnancy Childbirth. 2019;19:214.
Coley KM, Perosky JE, Nyanplu A, Kofa A, Anankware JP, Moyer CA, et al. Acceptability and feasibility of insect consumption among pregnant women in Liberia. Matern Child Nutr. 2020. https://doi.org/10.1111/mcn.12990.
Idris IO, Araoye D, Chijioke OD, Gavkalova N. A Policy Discussion on maternity waiting home in Zambia to achieve its vision 2030 on maternal and perinatal mortality. J Fam Med Heal Care. 2020;6(1):1–7.
Chandramohan D, Cutts F, Millard P. The effect of stay in a maternity waiting homes on perinatal mortality. J Trop Med Hyg. 1995;98:261–7.
Fogliati P, Straneo M, Mangi S, Azzimonti G, Kisika F, Putoto G. A new use for an old tool: maternity waiting homes to improve equity in rural childbirth care. Results from a cross-sectional hospital and community survey in Tanzania. Health Policy Plan. 2017;32:1354–60.
Lori JR, Munro ML, Rominski S, Williams G, Dahn BT, Boyd CJ, et al. Maternity waiting homes and traditional midwives in rural Liberia. Int J Gynecol Obstet. 2013;123(2):114–8.
Meshesha B, Dejene G, Hailemariam T. The role of maternity waiting area in improving obstetric outcomes: a comparative cross-sectional study, Jinka Zonal Hospital, Southern regional state. J Womens Heal Care. 2017;6:6.
Poovan P, Kifle F, Kwast BE. A maternity waiting home reduces obstetric catastrophes. World Health Forum. 1990;11(4):440–5.
Singh K, Speizer I, Kim ET, Lemani C, Phoya A. Reaching vulnerable women through maternity waiting homes in Malawi. Int J Gynaecol Obstet. 2017;136:91–7.
Spaans W, van Roosmalen J, van Wiechen CMA. A maternity waiting home experience in Zimbabwe. Int J Gynecol Obstet. 1998;61(2):179–80.
van Lonkhuijzen L, Stegeman M, Nyirongo R, van Roosmalen J. Use of maternity waiting home in rural Zambia. Afr J Reprod Health. 2003;7(1):32–6.
Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to meta-analysis. 1st ed. Chichester: John Wiley & Sons Ltd; 2009. p. 412.
Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. A basic introduction to fixed-effect and random-effects models for meta-analysis. Res Synth Method. 2010. https://doi.org/10.1002/jrsm.12.
Higgins JPT, Chandler J, Cumpston M, Li T, Page M, Welch V. Cochrane handbook for systematic reviews of interventions (version 6.0). Hoboken: Wiley; 2019.
Pigott T. Advances in Meta-Analysis. New York: Springer, US; 2012.
Heterogeneity BM. Common mistakes in meta-analysis and how to avoid them. 1st ed. Englewood: Biostat Incorporated; 2019. p. 75–137.
Higgins JPT. Commentary: Heterogeneity in meta-analysis should be expected and appropriately quantified. Int J Epidemiol. 2008;37:1158–60.
Metelli S, Chaimani A. Challenges in meta-analyses with observational studies. Evid Based Ment Heal. 2020;23:83–7.
Tanner-smith EE, Grant S. Meta-analysis of complex interventions. Annu Rev Psychol. 2018;39:135–51.
Penn-Kekana L, Pereira S, Hussein J, Bontogon H, Chersich M, Munjanja S, et al. Understanding the implementation of maternity waiting homes in low- and middle-income countries: a qualitative thematic synthesis. BMC Pregnancy Childbirth. 2017;17:269.
Higgins JPT, Thompson SG, Spiegelhalter DJ. A re-evaluation of random-effects meta-analysis. J R Stat Soc. 2009;172(Part 1):137–59.
Higgins JPT, LĂłpez-lĂłpez JA, Becker BJ, Davies SR, Dawson S, Grimshaw JM, et al. Synthesising quantitative evidence in systematic reviews of complex health interventions. BMJ Glob Heal. 2019;4:e000858.
Mueller M, Addario MD, Egger M, Cevallos M, Dekkers O, Mugglin C, et al. Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations. BMC Med Res Methodol. 2018;18:44.
Gaym A, Pearson L, Soe KWW. Maternity waiting homes in Ethiopia-three decades experience. Ethiop Med J. 2012;50(3):209–19.
Millard P, Bailey J, Hanson J. Antenatal village stay and pregnancy outcome in rural Zimbabwe. Cent Afr J Med. 1991;37(1):1–4.
Tumwine JK, Dungare PS. Maternity waiting shelters and pregnancy outcome: experience from a rural area in Zimbabwe. Ann Trop Paediatr. 1996;16(1):55–9.
Viswanathan M, Mcpheeters ML, Murad MH, Butler ME, Beth EE, Dyson MP, et al. AHRQ series on complex intervention systematic reviews paper 4: selecting analytic approaches. J Clin Epidemiol. 2017;90:28–36.
Neogi SB, Sharma J, Negandhi P, Chauhan M, Reddy S, Sethy G. Risk factors for stillbirths: how much can a responsive health system prevent ? BMC Pregnancy Childbirth. 2018. https://doi.org/10.1186/s12884-018-1660-1.
Lawn JE, Blencowe H, Waiswa P, Amouzou A, Mathers C, Hogan D, et al. Stillbirths: rates, risk factors, and acceleration towards 2030. Lancet. 2016;387:587–603.
Downes MJ, Brennan ML, Williams HC, Dean RS. Development of a critical appraisal tool to assess the quality of cross-sectional studies (AXIS). BMJ Open. 2016;6:e011458.
Acknowledgements
We would like to acknowledge the feedback provided by Dr. Manisha Kulkarni, Dr. Gail Webber and an anonymous colleague on earlier versions of the manuscript. We would also like to recognize the constructive feedback from the anonymous reviewer which greatly improved the article.
Funding
The authors did not receive any funding for this work.
Author information
Authors and Affiliations
Contributions
JK conceived of the study, conducted the analysis of the study for illustrative purposes with ZL and drafted the first draft of the manuscript. JK, ZL, KH and KW were involved in interpretation of the results, reviewing and revising the manuscript, and all approved its final version. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Kurji, J., Hackett, K., Wild, K. et al. The effect of maternity waiting homes on perinatal mortality is inconclusive: a critical appraisal of existing evidence from Sub-Saharan Africa. BMC Res Notes 14, 86 (2021). https://doi.org/10.1186/s13104-021-05501-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13104-021-05501-2