- Research article
- Open Access
Telephone interviews and online questionnaires can be used to improve neurodevelopmental follow-up rates
BMC Research Notesvolume 7, Article number: 219 (2014)
Maximising response rates to neurodevelopmental follow-up is a key challenge for paediatric researchers. We have investigated the use of telephone interviews and online questionnaires to improve response rates, reduce non-response bias, maintain data completeness and produce unbiased outcomes compared with postal questionnaires when assessing neurodevelopmental outcomes at 2 years.
A prospective cohort study of babies born ≥32 weeks gestation. Neurodevelopmental outcomes were assessed at 2 years of age using a parent questionnaire completed via post, telephone or online. Relative Risks with 95% confidence intervals (RR; 95% CI) were calculated to identify participant characteristics associated with non-response and questionnaire response mode (postal vs. telephone/online). The proportion of missing data and prevalence of adverse outcomes was compared between response modes using generalized linear models.
Offering telephone/online questionnaires increased the study response rate from 55% to 60%. Telephone/online responders were more likely to be non-white (RR 1.6; [95% CI 1.1, 2.4]), non-English speaking (1.6; [1.0, 2.6]) or have a multiple birth (1.6; [1.1, 2.3]) than postal responders. There were no significant differences in the prevalence of adverse neurodevelopmental outcomes between those who responded via post vs. telephone/online (1.1; [0.9, 1.4]). Where parents attempted all questionnaire sections, there were no significant differences in the proportion of missing data between response modes.
Where there is sufficient technology and resources, offering telephone interviews and online questionnaires can enhance response rates and improve sample representation to neurodevelopmental follow-up, whilst maintaining data completeness and unbiased outcomes.
Minimising attrition and maximising response rates present major challenges for researchers conducting longitudinal studies. This is particularly evident in the field of paediatrics, in which birth cohorts are frequently followed up for epidemiological studies and randomised trials of perinatal interventions. Participant attrition not only reduces the power of a study, but can bias results : non-responders typically have greater medical and socio-economic risk and may differ systematically in terms of disorders of interest, resulting in biased estimates of adverse outcomes [2–7].
Postal questionnaires are frequently used to assess neurodevelopmental outcomes in large-scale studies and strategies for improving response rates have been well documented [8, 9]. In particular, offering telephone interviews in addition to postal questionnaires has been shown to improve response rates in health related research [10, 11]. Telephone interviews have also been shown to yield comparable or lower levels of missing data than postal questionnaires, thus improving the completeness of data obtained [11–14]. Offering telephone follow-up to non-responders to postal surveys has also been shown to improve response rates among participants from socio-economically deprived and ethnically diverse sectors of the population [15, 16]. Therefore employing multiple modes of data collection may also improve sample representation and reduce non-response bias [10, 11]. Electronic data collection methods are becoming increasingly popular and, like telephone interviews, these have also been shown to improve response rates over the use of postal questionnaires alone [13, 17, 18]. However, although mixed mode data collection may improve response rates, the mode of response may bias outcomes [10, 13]. Thus is it important to explore the effects of response mode on both response rates and measures of data quality.
These issues have not been studied in relation to methods for improving response rates to infant neurodevelopmental follow-up. In particular, the impact of employing mixed mode data collection on parent-reported neurodevelopmental outcomes requires investigation as both parent and child characteristics may affect response rates and reported outcomes. Using parent reports of neurodevelopment at two years of age, we have explored the use of telephone interviews and online questionnaires for (1) increasing response rates, (2) improving sample representation, (3) maintaining data completeness and (4) yielding unbiased outcomes over postal questionnaires alone.
The mothers of all babies born late and moderately preterm (32+0 to 36+6 weeks of gestation) from September 2009 to December 2010 and resident in a geographically-defined area of Leicestershire and Nottinghamshire were invited to participate in the Late And Moderately preterm Birth Study (LAMBS). A random sample of term-born babies (≥37 weeks gestation) was also recruited from the same region during the same time period. Mothers were approached as soon as possible after delivery to obtain informed consent and to carry out a maternal interview prior to their discharge from hospital. For mothers who were discharged in the first few hours after birth, a research midwife arranged a home visit as soon as possible after discharge in order to obtain consent. Contact was made as soon as possible in this way in order to minimise recall bias during the maternal interview. There were no exclusion criteria. Only data relating to live births are included in the present report. The study was approved by the Derbyshire NHS Research Ethics Committee (Ref 09/H0401/25).
At the time of consent, mothers were interviewed to obtain information about socio-economic factors and antenatal health. Information about the baby’s hospital admission was collected from medical notes and parents were asked to complete a postal questionnaire to assess their child’s use of healthcare services at 6 and 12 months of age. The present report focuses on the 2-year neurodevelopmental follow-up for which data were collected using a hierarchical mixed-mode approach. Seven to ten days before the child turned 2 years of age corrected for prematurity (2 y-CA), parents were sent a postal questionnaire to complete. If no response was made by the time the child reached 2 y-CA, parents were then initially contacted by telephone to remind them that a questionnaire had been posted, to ask whether help was needed in completing it and to offer the option of telephone or online completion. If this contact could not be made by telephone after repeated unsuccessful attempts, an attempt was made to contact the parent by email or text message (where details were available) to provide this reminder. Where parents opted for a telephone interview, these were carried out at a time convenient to them; interviews were carried out in the evening where parents specified this was the most appropriate time. For families in which no member was able to speak English sufficiently to answer the questionnaire, a member of the study team who spoke the parents’ language carried out the interview where possible. Parents who requested online completion were emailed a web-link to an electronic version of the questionnaire hosted on surveymonkey.com . If a completed questionnaire was not received 2 weeks later, parents were contacted again to prompt them to complete it and to again offer alternative modes of completion. At this point, parents who could not be contacted by telephone, text message or email were sent a second postal questionnaire. After a further 1-2 weeks, a final attempt was made to contact parents and offer alternatives for completing the postal questionnaire. Parents for whom a completed questionnaire was never received and those who could not be contacted despite repeated attempts were classified as non-responders (see Figure 1).
Information obtained from the maternal interview was used to classify mothers’ occupation using the UK National Statistics Socio-Economic Classification (NS-SEC)  from which occupations were classified as managerial, professional/intermediate, routine/semi-routine or unemployed. Mothers’ highest educational qualification was classified as none or secondary level vs. tertiary qualifications. Mothers were also asked about their general health (categorised excellent/good/fair vs. poor), mental health (no anxiety/depression vs. moderately to severely anxious/depressed) and their financial situation (doing well/OK vs. finding it difficult) during pregnancy. Information collected from the infant’s medical notes included gestational age, birthweight, multiple birth status, fetal growth restriction (classified using customised antenatal growth charts ), congenital anomalies, respiratory support, feeding and cranial MRI and ultrasound findings. Intra-cranial abnormalities were classified where Grade III or IV intra-ventricular haemorrhage (IVH), periventricular leukomalacia (PVL) and Grade II or III neonatal encephalopathy were present.
Two-year questionnaires were identical for all modes of completion. The prevalence of adverse neurodevelopmental outcomes and the proportion of missing data were calculated for two outcome measures.
Parent Report of Children’s Abilities-Revised (PARCA-R) , a 158-item parent report of cognitive and language development from which scores <49  were used to identify cognitive delay. As blank items could be a valid response on the language sub-scale, the proportion of missing data was calculated for the 34-item non-verbal subscale alone.
An additional four forced-choice items were used to assess the child’s communication, motor, vision and hearing function. Children with a moderate to severe impairment in any one of these domains were classified with neurosensory impairment (NSI) using standard criteria .
Data were double entered then verified and analysed using Stata v12. Data obtained from mothers of late and moderately preterm and term-born infants were combined to analyse follow-up rates for the study as a whole. Risk Ratios (RR) for non-response, with 95% confidence intervals (95% CI), were estimated for maternal and infant characteristics; the same methods were used to estimate RRs for completing the questionnaire via telephone or online for the same characteristics. To assess the completeness of data obtained by response mode (postal vs. telephone/online), the percentage of missing data on each outcome measure was computed using a published formula : number of missing items/[(number of responders) × (number of questions presented)]. The percentage of missing data with RR (95% CI) for absence of data were calculated using generalised linear models including robust variance estimation to account for clustering within questionnaires . The percentage of children with adverse outcomes was calculated by response mode (postal vs. telephone/online) both unadjusted and after adjustment for characteristics associated with mode of response to determine whether mode of completion biased outcomes on each of the two measures. For all analyses a p-value <0.05 was taken as evidence of a statistically significant effect.
Results are presented in relation to the 4 study aims:
Do telephone interviews and online questionnaires increase study response rates?
Of 2921 eligible live births, 2385 (82%) were recruited (Figure 2). Nine infants died following recruitment leaving 2376 children eligible for follow-up. At 2 years, the parents of 1422 children (59.8%) responded, of which 1296 (91%) completed a postal questionnaire, 58 (4%) a telephone interview and 68 (5%) an online questionnaire (Figure 2). Thus of the 1080 infants whose parents did not initially return a postal questionnaire, 11.7% ultimately provided data by telephone or online increasing the overall response rate by 5.3% from 54.5% to 59.8%.
Do telephone interviews and online questionnaires improve sample representation?
To answer this question, we compared the characteristics of non-responders with those who completed questionnaires by telephone or online. As shown in Table 1, non-responders had significantly greater socio-economic and demographic risk: these mothers were more likely to be younger, non-white, non-English speaking, less well educated, unemployed or in a manual or routine occupation, unsupported at home, finding it difficult to manage financially and to have poorer mental and general health (all p’s <0.05). They were also more likely to be mothers of multiples and less likely to be giving their child breast milk at discharge; however, no indices of poor neonatal outcome were significantly associated with non-response.
As shown in Table 2, mothers who completed questionnaires via telephone/online shared key demographic characteristics with non-responders: they were significantly more likely to be non-white, non-English speaking, have had a multiple birth and to have delivered at term (all p’s <0.05).
Do telephone interviews and online questionnaires yield comparable rates of missing data?
There were no missing data on the neurosensory items in either response mode (Table 3). In contrast, telephone/online completions resulted in significantly more missing data than postal questionnaires for the PARCA-R. However, this was due to online completions only; two parents abandoned the online questionnaire after the neurosensory items resulting in substantial portions of missing PARCA-R data for these children. When these two respondents were excluded, there was no significant difference in the proportion of missing PARCA-R data between response modes (Table 3).
Do telephone interviews and online questionnaires yield comparable rates of adverse outcomes?
The prevalence of adverse outcomes did not differ significantly between response modes (Table 3). Adjustment for factors that were significantly associated with response mode (i.e., ethnic group, language, multiple birth, preterm birth; see Table 2), and therefore potential confounders, did not change the conclusions (Table 3).
The results of this study show that offering telephone interviews and online questionnaires improves response rates and enhances sample representation over postal questionnaires alone, whilst retaining data completeness and unbiased outcomes. Although this has previously been shown in relation to adult self-report questionnaires, the utility of mixed mode data collection for assessing infant neurodevelopmental outcomes has not been explored.
Consistent with previous studies [2–4, 6], mothers who did not respond to follow-up had lower socio-economic status, poorer health and were more likely to be from non-white ethnic groups. As these factors are associated with adverse neurodevelopmental outcomes, improving response rates is therefore important for reducing non-response bias . Our results indicate that offering multiple modes of questionnaire completion is effective for enhancing both response rates and sample representation. Indeed, previous studies have shown that following up non-responders to postal surveys with telephone interviews or reminders improves response rates among participants from socio-economically deprived and ethnically diverse samples, consistent with the present results. Choosing to complete questionnaires via the telephone may have reflected mothers’ difficulty in completing postal questionnaires due to poor command of written English. Indeed, telephone interviews reduce the cognitive demands placed on participants compared with written questionnaires . Given the poorer response rate to postal surveys among ethnic minority samples , these are pertinent considerations for studies in which ethnic minority groups may be over-represented.
In addition to improving response rates and sample representation, our results show that the completeness of data obtained via telephone interviews and online questionnaires was comparable with that of postal questionnaires. Moreover, there were no significant differences in the prevalence of adverse outcomes between response modes. This is a key finding for studies in which parent reports of neurodevelopment at 2 years constitute primary outcomes. The PARCA-R in particular is increasingly used as an outcome measure in RCTs of perinatal interventions, both nationally and internationally, and as a clinical follow-up tool [28–31]. As response mode did not bias outcomes, researchers can have added confidence in using multiple methods to collect data using this scale.
Telephone interviews and online questionnaires were associated with a greater proportion of missing data, but this does not negate the efficacy of these for data collection, especially in the case of telephone interviews. The excess of missing data arose as a result of two respondents who abandoned the online questionnaire prior to completing the PARCA-R. After excluding data from these respondents, there was no difference in the proportion of missing data, as shown previously in relation to telephone interviews [11, 13, 14]. It should be noted that we did not allow multiple completions of the online questionnaire from the same IP address; as parents had to complete the questionnaire on one occasion the premature terminations may be an artefact of this. Missing data may be minimised by allowing online questionnaires to be completed in multiple sittings. In addition, other practical issues should be considered when offering mixed mode data collection. Contacting parents by telephone to provide reminders and offer an interview was labour intensive and required substantial additional resources to those initially envisaged; multiple attempts were often needed to telephone parents before contact was made. The added value of offering these methods of follow-up should therefore be weighed against the resources available for maintaining contact with a cohort in this manner .
Strengths and limitations
We believe our results will be applicable on a wider level. The challenges facing researchers are universal and the methods we explored are generic in nature. Our response rate was also comparable with that of postal surveys conducted in both healthcare and social science settings [27, 33]. However, there are a number of limitations that may be addressed in future research. The 5% increase in response rates for offering multiple response modes was modest compared with previous studies [10, 11]. However, these studies were conducted with adult self-report questionnaires and issues of non-response may be different in birth cohort studies. Our study was also limited by the use of a hierarchical design in which telephone and online response modes were offered to those who had not responded to a postal questionnaire in a timely manner, thus comprising a group who were already difficult to follow-up. It is impossible to determine how many parents would have opted for online or telephone completion had their preferred response mode been ascertained at study entry. Sending the initial request for follow-up using their preferred response mode may have substantially increased response rates, particularly for online questionnaires which are often preferred over postal questionnaires . In addition, the relatively small number of non-postal responders meant that we were unable to assess differences between the three response modes separately or to explore these issues in relation to different ethnic or socio-economic groups. Future studies should therefore aim to replicate these results using large prospective studies in which participants are randomised to response modes. More studies of the validity of electronic data collection for neurodevelopmental follow-up are needed as such methods are likely to become a mainstay with ongoing advances in technology.
Offering telephone interviews and online questionnaires as alternatives to postal questionnaires can enhance follow-up rates, improve sample representation and reduce non-response bias when assessing infant neurodevelopmental outcomes. Where the technology and resources allow, these measures may prove fruitful for maximising response rates and reducing non-response bias in birth cohort studies and RCTs of perinatal interventions. Larger studies are needed to replicate these findings using randomised allocation of response mode.
Fetal growth restriction
Parent Report of Children’s Abilities-Revised
Wolke D, Sohne B, Ohrt B, Riegel K: Follow-up of preterm children: important to document dropouts. Lancet. 1995, 345: 447-10.1016/S0140-6736(95)90425-5.
Guillen U, DeMauro S, Ma L, Zupancic J, Roberts R, Schmidt B, Kirpalani H: Relationship between attrition and neurodevelopmental impairment rates in extremely preterm infants at 18 to 24 months: a systematic review. Arch Pediatr Adolesc Med. 2012, 166 (2): 178-184. 10.1001/archpediatrics.2011.616.
Johnson S, Fawke J, Hennessy E, Rowell V, Thomas S, Wolke D, Marlow N: Neurodevelopmental disability through 11 years in children born before 26 weeks of gestation: the EPICure study. Pediatrics. 2009, 124: e249-e257. 10.1542/peds.2008-3743.
Moore T, Hennessy EM, Myles J, Johnson SJ, Draper ES, Costeloe KL, Marlow N: Neurological and developmental outcome in extremely preterm children born in England in 1995 and 2006: the EPICure studies. BMJ. 2012, 345: e7961-10.1136/bmj.e7961.
Pennefather PM, Tin W, Clarke MP, Dutton J, Fritz S, Hey EN: Bias due to incomplete follow up in a cohort study. Br J Ophthalmol. 1999, 83 (6): 643-645. 10.1136/bjo.83.6.643.
Tin W, Fritz S, Wariyar U, Hey E: Outcome of very preterm birth: children reviewed with ease at 2 years differ from those followed up with difficulty. Arch Dis Child Fetal Neonatal Ed. 1998, 79 (2): F83-F87. 10.1136/fn.79.2.F83.
Kramer MS, Wilkins R, Goulet L, Seguin L, Lydon J, Kahn SR, McNamara H, Dassa C, Dahhou M, Masse A, Miner L, Asselin G, Gauthier H, Ghanem A, Benjamin A, Platt RW, Montreal Prematurity Study Group: Investigating socio-economic disparities in preterm birth: evidence for selective study participation and selection bias. Paediatr Perinat Epidemiol. 2009, 23 (4): 301-309. 10.1111/j.1365-3016.2009.01042.x.
Edwards PJ, Roberts IG, Clarke MJ, DiGuiseppi C, Wentz R, Kwan I, Cooper R, Felix L, Pratap S: Methods to increase response rates to postal questionnaires. Cochrane Database Syst Rev. 2007, MR000008-2
Nakash RA, Hutton JL, Jorstad-Stein EC, Gates S, Lamb SE: Maximising response to postal questionnaires–a systematic review of randomised trials in health research. BMC Med Res Methodol. 2006, 6: 5-10.1186/1471-2288-6-5.
Brambilla DJ, McKinlay SM: A comparison of responses to mailed questionnaires and telephone interviews in a mixed mode health survey. Am J Epidemiol. 1987, 126 (5): 962-971.
Lall R, Mistry D, Bridle C, Lamb SE: Telephone interviews can be used to collect follow-up data subsequent to no response to postal questionnaires in clinical trials. J Clin Epidemiol. 2012, 65 (1): 90-99. 10.1016/j.jclinepi.2011.04.011.
Feveile H, Olsen O, Hogh A: A randomized trial of mailed questionnaires versus telephone interviews: response patterns in a survey. BMC Med Res Methodol. 2007, 7: 27-10.1186/1471-2288-7-27.
Bowling A: Mode of questionnaire administration can have serious effects on data quality. J Public Health (Oxf). 2005, 27 (3): 281-291. 10.1093/pubmed/fdi031.
Parker C, Dewey M: Assessing research outcomes by postal questionnaire with telephone follow-up: TOTAL study group: trial of occupational therapy and leisure. Int J Epidemiol. 2000, 29 (6): 1065-1069. 10.1093/ije/29.6.1065.
Kiezebrink K, Crombie IK, Irvine L, Swanson V, Power K, Wrieden WL, Slane PW: Strategies for achieving a high response rate in a home interview survey. BMC Med Res Methodol. 2009, 9: 46-10.1186/1471-2288-9-46.
Choudhury Y, Hussain I, Parsons S, Rahman A, Eldridge S, Underwood M: Methodological challenges and approaches to improving response rates in population surveys in areas of extreme deprivation. Prim Health Care Res Dev. 2012, 13 (3): 211-218. 10.1017/S1463423611000466.
Cook KF, Choi SW, Crane PK, Deyo RA, Johnson KL, Amtmann D: Letting the CAT out of the bag: comparing computer adaptive tests and an 11-item short form of the Roland-Morris disability questionnaire. Spine. 2008, 33 (12): 1378-1383. 10.1097/BRS.0b013e3181732acb.
Dillman DA, Phelps G, Tortora R, Swift K, Kohrell J, Berck J, Messer BL: Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the internet. Soc Sci Res. 2009, 38: 1-18. 10.1016/j.ssresearch.2008.03.007.
The Office for National Statistics: Standard Occupational Classification 2010: Volume 3 The National Statistics Socio-economic Classification User Manual. 2010, London: Palgrave Macmillan
Gardosi J, Francis A: Customised Weight Centile Calculator. GROW version 5.16. Year Gestation Network, [http://www.gestation.net]
Johnson S, Wolke D, Marlow N: Developmental assessment at 2 years: validity of parent reports. Dev Med Child Neurol. 2008, 50: 58-62. 10.1111/j.1469-8749.2007.02010.x.
Johnson S, Marlow N, Wolke D, Davidson L, Marston L, O’Hare A, Peacock J, Schulte J: Validation of a parent report measure of cognitive development in very preterm infants. Dev Med Child Neurol. 2004, 46: 389-397.
British Association of Perinatal Medicine: Report of a BAPM/RCPCH Working Group: Classification of health status at 2 years as a perinatal outcome. 2008, London: BAPM
Williams RL: A note on robust variance estimation for cluster-correlated data. Biometrics. 2000, 56 (2): 645-646. 10.1111/j.0006-341X.2000.00645.x.
Quigley MA, Poulsen G, Boyle E, Wolke D, Field D, Alfirevic Z, Kurinczuk JJ: Early term and late preterm birth are associated with poorer school performance at age 5 years: a cohort study. Arch Dis Child Fetal Neonatal Ed. 2012, 97 (3): F167-F173. 10.1136/archdischild-2011-300888.
Lynn P, Burton J, Kaminska O, Knies G, Nandi A: An initial look at non-response and attrition in Understanding Society. Understanding Society Working Paper Series. Edited by: Research IfSaE, vol. 2012-02. 2012, Essex: Institute for Social and Economic Research
Marlow N, Greenough A, Peacock JL, Marston L, Limb ES, Johnson AH, Calvert SA: Randomised trial of high frequency oscillatory ventilation or conventional ventilation in babies of gestational age 28 weeks or less: respiratory and neurological outcomes at 2 years. Arch Dis Child Fetal Neonatal Ed. 2006, 91 (5): F320-F326. 10.1136/adc.2005.079632.
Martin AJ, Darlow BA, Salt A, Hague W, Sebastian L, McNeill N, Tarnow-Mordi W: Performance of the parent report of children’s abilities-revised (PARCA-R) versus the Bayley scales of infant development III. Arch Dis Child. 2013, 98 (12): 955-958. 10.1136/archdischild-2012-303288.
Cuttini M, Ferrante P, Mirante N, Chiandotto V, Fertz M, Dall’Oglio AM, Coletti MF, Johnson S: Cognitive assessment of very preterm infants at 2-year corrected age: performance of the Italian version of the PARCA-R parent questionnaire. Early Hum Dev. 2012, 88 (3): 159-163. 10.1016/j.earlhumdev.2011.07.022.
Martin AJ, Darlow BA, Salt A, Hague W, Sebastian L, Mann K, Tarnow-Mordi W, Inis Trial Collaborative G: Identification of infants with major cognitive delay using parental report. Dev Med Child Neurol. 2012, 54 (3): 254-259. 10.1111/j.1469-8749.2011.04161.x.
Ziegenfuss JY, Burmeister KR, Harris A, Holubar SD, Beebe TJ: Telephone follow-up to a mail survey: when to offer an interview compared to a reminder call. BMC Med Res Methodol. 2012, 12: 32-10.1186/1471-2288-12-32.
Asch DA, Jedrziewski MK, Christakis NA: Response rates to mail surveys published in medical journals. J Clin Epidemiol. 1997, 50 (10): 1129-1136. 10.1016/S0895-4356(97)00126-1.
We would like to thank the families who participated in the Late and Moderately Preterm Birth Study (LAMBS). This article presents independent research funded by the National Institute for Health Research (NIHR) under its Programme Grants for Applied Research (PGfAR) Programme (Grant Reference Number RP-PG-0407-10029). The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health. Neil Marlow receives a proportion of funding from the Department of Health’s NIHR Biomedical Research Centres funding scheme at UCLH/UCL.
The authors declare that they have no competing interests.
SJ designed and managed the 2-year neurodevelopmental outcome assessment and was responsible for interpreting study results and drafting and revising the manuscript. SES carried out statistical analyses and interpretation of study results and reviewed and revised the manuscript critically for intellectual content. BM, LKS, DF, ESD, NM & EMB conceived and designed the study, contributed to study management and reviewed the manuscript critically for intellectual content. DF was the principal investigator. All authors read and approved the final manuscript for submission.