Inclusion of mobile telephone numbers into an ongoing population health survey in New South Wales, Australia, using an overlapping dual-frame design: impact on the time series

Background Since 1997, the NSW Population Health Survey (NSWPHS) had selected the sample using random digit dialing of landline telephone numbers. When the survey began coverage of the population by landline phone frames was high (96%). As landline coverage in Australia has declined and continues to do so, in 2012, a sample of mobile telephone numbers was added to the survey using an overlapping dual-frame design. Details of the methodology are published elsewhere. This paper discusses the impacts of the sampling frame change on the time series, and provides possible approaches to handling these impacts. Methods Prevalence estimates were calculated for type of phone-use, and a range of health indicators. Prevalence ratios (PR) for each of the health indicators were also calculated using Poisson regression analysis with robust variance estimation by type of phone-use. Health estimates for 2012 were compared to 2011. The full time series was examined for selected health indicators. Results It was estimated from the 2012 NSWPHS that 20.0% of the NSW population were mobile-only phone users. Looking at the full time series for overweight or obese and current smoking if the NSWPHS had continued to be undertaken only using a landline frame, overweight or obese would have been shown to continue to increase and current smoking would have been shown to continue to decrease. However, with the introduction of the overlapping dual-frame design in 2012, overweight or obese increased until 2011 and then decreased in 2012, and current smoking decreased until 2011, and then increased in 2012. Our examination of these time series showed that the changes were a consequence of the sampling frame change and were not real changes. Both the backcasting method and the minimal coverage method could adequately adjust for the design change and allow for the continuation of the time series. Conclusions The inclusion of the mobile telephone numbers, through an overlapping dual-frame design, did impact on the time series for some of the health indicators collected through the NSWPHS, but only in that it corrected the estimates that were being calculated from a sample frame that was progressively covering less of the population.


Background
Since 2002, information about the health of the NSW population-7.3 million-has been obtained using the NSW Population Health Survey (NSWPHS) [1]. This survey is a continuous sample survey of approximately 15,000 persons each year; with equal numbers selected from each of the strata -health administrative areasusing random digit dialing (RDD) of landline telephone numbers and computer assisted telephone interviewing (CATI). Prior to 2002, two surveys were conducted in 1997 and 1998. When the survey began in 1997 coverage of the Australian population by landline phone frames was estimated to be 96%, however by 2006 coverage had dropped to 91% and by 2009 coverage had declined even further to 85% [2,3]. This decline in the coverage of the population by landline phone frames was consistent with what was occurring in other countries [4][5][6].
Although not routinely available in Australia, differences in health risk factor and health status prevalence estimates between adults covered in a landline phone frame, and adults who live in a household without a landline telephone connection on which to make and answer calls (ie mobile-only phone users), have been measured in the USA using the National Health Interview Survey (NHIS) since 2003 [7]. This survey found substantial differences for the health indicators: five or more drinks in one day at least once in the past year (17.5% v 30.5% -74% higher), current smokers (14.5% v 24.3% -68% higher), and ever diagnosed with diabetes (10.8% v 6.2% -43% lower) [8].
Based on this overseas experience, it was estimated that when the overall landline phone coverage dropped below 85% in Australia, the differences in health indicators between people who were covered in the landline phone frame and mobile-only phone users differed by more than 50%, and/or there were associations between phone usage and the health indicators, it would impact appreciably on the overall NSW prevalence estimates [9]. In 2010, the landline phone coverage in Australia dropped below 85%, and an Australian study showed relative differences between those people who were covered by the landline phone frame, and mobile-only phone users, of 69% for current smoking (20.4% v 34.5%) and 56% for ever diagnosed with diabetes (11.7% v 5.2%) [10]. This study also showed that for current smoking, even after adjusting by age and sex, the mobile-only phone users were more likely to be current smokers. So, because of these changes in landline phone coverage, mobile telephone numbers were included in the 2012 NSWPHS, using an overlapping dual-frame design [11]. Within a stratum the landline sample was selected using equal probability of selection of landline telephone numbers and then random selection of one person from the selected household. In the mobile phone sample an equal probability sample of mobile telephone numbers in Australia was selected and screened for adult residents in NSW. If the respondent had one or more children one child was selected at random. Sample weights thus reflected the differing sampling probabilities. The sample weights of the dual phone-users were then adjusted so that the composite factor used to combine the estimates for this component obtained from the landline sample and the mobile phone sample, λ, was set at 0.5. Benchmarking to the reference population was then performed by adjusting the weights for differences between weighted estimates of the age and sex structure obtained from the combined landline and mobile phone sample and ABS mid-year population estimates for each stratum. More details about the weighting strategy are provided in Barr et al. [12].
This paper firstly provides prevalence estimates for type of phone-use, health risk factor and health status from the 2012 NSWPHS. It also examines if, as other authors have found, there are any associations between the health indicators and type of phone-use, adjusting for the weighting variables. Health estimates from the 2012 NSWPHS, and the landline frame sample, re-benchmarked to the NSW population, were then compared to the previous year's estimates. The impacts on the time series of the change in design to an overlapping dual-frame design is then discussed as are possible approaches to handling these impacts.

Data source
Data from the NSWPHS for 2012 was used. This consisted of data on 15,214 respondents with 10,518 (69.1%) from the landline phone frame, of which 1,792, (17.0%) were landline only, and 4,696 (30.1%) from the mobile phone frame, of which 1,121 (23.9%) were mobile only. The overall response rate was 31.0%, co-operation rate was 63.4%, refusal rate was 17.9% and contact rate was 66.9%.

Calculation of prevalence estimates
Estimates for type of phone-use were calculated overall and by selected demographic characteristics. Health risk factor and health status indicators were selected from the questions asked in the survey as shown in Table 1. Prevalence estimates and 95 per cent confidence intervals using the SURVEYFREQ procedure in SAS, which uses the Taylor expansion method to calculate sampling errors of complex sample designs, were calculated for each indicator [13].
Associations between the health indicators and type of phone-use Prevalence ratios (PR) for each of the health indicators were calculated using Poisson regression analysis with robust variance estimation by type of phone-use using the categories mobile-only, landline-only, dual phone users The indicator includes those who drink five or more standard drinks on a day when they drink alcohol.
How often do you usually drink alcohol?
On a day when you drink alcohol, how many standard drinks do you usually have?
A standard drink is equal to 1 middy of full-strength beer, 1 schooner of light beer, 1 small glass of wine, or 1 pub-sized nip of spirits.
More than two alcoholic drinks in a day The indicator includes those who drink more than two standard drinks on a day when they drink alcohol.
How often do you usually drink alcohol?
On a day when you drink alcohol, how many standard drinks do you usually have?
A standard drink is equal to 1 middy of full-strength beer, 1 schooner of light beer, 1 small glass of wine, or 1 pub-sized nip of spirits.
Recommended fruit intake The indicator includes those who consumed two or more serves of fruit a day. The recommended fruit intake is at least 2 serves a day for persons aged 19 years and over, depending on their overall diet. For simplification, this recommendation is applied to [16][17][18] year olds. One serve is equivalent to 1 medium piece or 2 small pieces of fruit.
How many serves of fruit do you usually eat each day?
Recommended vegetable intake The indicator includes those who consumed 5 or more serves of vegetables a day. The recommended vegetable intake is at least 5 serves a day for persons aged 16 years and over, depending on their overall diet. One serve is equivalent to 1/2 cup of cooked vegetables or 1 cup of salad vegetables.
How many serves of vegetables do you usually eat each day?

Current smoking
The indicator includes those who smoked daily or occasionally.
Which of the following best describes your smoking status: smoke daily, smoke occasionally, do not smoke now but I used to, I have tried it a few times but never smoked regularly, or I have never smoked?
Adequate physical activity The indicator includes those who did adequate physical activity. Adequate physical activity is a total of 150 minutes a week on 5 separate occasions. The total minutes were calculated by adding minutes in the last week spent walking continuously for at least 10 minutes, minutes doing moderate physical activity, plus 2 x minutes doing vigorous physical activity.
In the last week, how many times have you walked continuously for at least 10 minutes for recreation or exercise or to get to or from places?
What do you estimate was the total time you spent walking in this way in the last week?
In the last week, how many times did you do any vigorous physical activity that made you breathe harder or puff and pant?
What do you estimate was the total time you spent doing this vigorous physical activity in the last week?
In the last week, how many times did you do any other more moderate physical activity that you have not already mentioned?
Positive self-reported health status The indicator includes those responding excellent, very good, or good to a global self-rated health status question.
Overall, how would you rate your health during the last 4 weeks: Was it excellent, very good, good, fair, poor, or very poor?

Current asthma
The indicator includes those who had symptoms of asthma or treatment for asthma in the last 12 months.
Have you ever been told by a doctor or hospital you have asthma?
Have you had symptoms of asthma or treatment for asthma in the last 12 months?
Ever diagnosed with diabetes The indicator includes those who either had diabetes or high blood glucose but did not have gestational diabetes.
Have you ever been told by a doctor or hospital you have diabetes?
Have you ever been told by a doctor or hospital you have high blood glucose?
If female, Were you pregnant when you were first told you had diabetes or high blood glucose?
Have you ever had diabetes or high blood glucose apart from when you were pregnant?
in the mobile frame, and dual phone users in the landline frame as the reference category. This analysis was then repeated adjusting for all of the weighting variables including age group, sex, administration area, number of eligible persons in the household, and number of phone lines. This analysis used the GENMOD procedure in SAS. As the Poisson model uses the natural logarithm as the link function, exponentiation of the parameter estimates was used to obtain the PRs for the study factors [14][15][16].

Comparison of 2012 prevalence estimates with previous years
Estimates for health related variables for the 2012 NSWPHS, as well as using just the landline frame sample, re-benchmarked to the NSW population, were then compared to the 2011 NSWPHS. Significant differences were identified by comparing the differences between the two estimates, divided by the standard error of the differences, calculated as √[SE(E 2011PHS ) 2 + SE(E 2012PHS ) 2 ]), with the standard normal distribution [17]. The full time series was examined for health indicators where there was an association between type of phone-use and the indicator, and both significance and direction changed between sampling designs.
Two solutions were considered to adjust the time series for the expansion of the coverage of the survey. The first being the backcasting method [18] where the 2012 figures were used and a correction factor was applied to each of the proceeding annual time points using the for- where Y A t is the estimate from the landline phone frame,Ỹ t is the revised estimate, A is the landline frame, and b is the mobile-only phone users.
, and C A t is the coverage each year as reported by the Australian Communication and Media Authority (ACMA) [2,3]. The second method was the minimal coverage method, which only allowed inclusion of point estimates into the time series where there was adequate population coverage, with adequate population coverage being defined as 85% or above [9].

Prevalence estimates for 2012
It was estimated from the 2012 NSWPHS that 20.0% (95%CI 18.3%-22.0%) of the NSW population were mobile-only phone users, 9.6% (95%CI 8.8%-10.4%) were landline only, and 70.2% (95%CI 68.4%-72.1%) were dual phone users. Table 2 shows estimates for type of phone-use overall, and, for selected demographic characteristics. As shown in Table 2, mobile-only phone user rates were highest in young people, unmarried people, and Aboriginal and/or Torres Strait Islander peoples.
Associations between the health indicators and type of phone-use Table 3 also shows the health indicator prevalence estimates for the 2012 NSWPHS overall, and for those with landline phones, who may also have a mobile phone, and those who are mobile-only phone users. As shown in Table 3 there were relative differences of more than 50% for five or more drinks of alcohol in a day (9.0% v 19.3%, 114% higher), and current smoking (14.0% v 28.3%, 103% higher). Table 4 shows null PRs and adjusted PRs, for weighting variables, by type of phone-use for each of the selected health indicators from the 2012 NSWPHS. As shown in Table 4, after adjusting by the weighting variables of age group, sex, administration area, number of phone lines, and number of eligible persons in the household, mobile-only phone users were more likely to: drink five or more drinks of alcohol in a day (PR, 1.29; 95%CI, 1.04-1.59) and be current smokers (PR, 1.39; 95%CI, 1.20-1.63), and mobile-only phone users were less likely to meet the recommended vegetable intake (PR, 0.65; 95% CI 0.50-0.85) and be overweight or obese (PR, 0.90; 95%CI 0.83-0.97) than dual phone users from the landline frame. Also, after adjusting by the weighting variables of age group, sex, administration area, number of phone Comparison of 2012 prevalence estimates with previous years  Table 5 also shows the health indicators estimates, using just the landline frame sample for 2012, re-benchmarked to the NSW population, compared to the 2011 NSWPHS. Significantly higher estimates were again found for recommended fruit intake (from 50.4% to 55.9%, p < 0.001) and recommended vegetable intake (from 8.4% to 12.3%, p < 0.001), and significantly lower estimates for more than two alcoholic drinks in a day (29.6% to 27.1%,   p = 0.042). However current smoking, positive self-reported health status, and overweight or obese were no longer statistically significantly different, and the difference had changed in direction for current smoking and overweight or obese. Table 6 shows a summary of the factors used to predict if the sampling frame change is likely to impact on the time series. These factors are; 50% difference or more for non-landline frame persons, association between type of phone-use and the indicator, change in significance between sampling designs, and change in direction between sampling designs. Based on this analysis, the two indicators for which the time series was most likely to be affected, were current smoking and overweight or obese.
Looking at the full time series as shown in Figure 1, if the NSWPHS had continued to be undertaken only using a landline frame, overweight or obese would have been shown to continue to increase and current smoking would have been shown to continue to decrease. However, with the introduction of the overlapping dual-frame design in 2012, overweight or obese increased until 2011 and then decreased in 2012, and current smoking decreased until 2011, and then increased in 2012.

Adjusting the time series
The backcasting method, applying relative differences of 103% for current smoking and -21% for The minimal coverage method, removing years 2010 and 2011 when the population coverage was less than 85% resulted, as shown in Figure 2, in the trend for current smoking continuing to decrease in 2012, but at a lesser rate and the trend for overweight or obese increasing until 2008 and plateauing thereafter. Lines of best fit were: y = -0.0112x 2 -0.3169x + 25.34, R 2 = 0.9156 for current smoking, and y = -0.0641x 2 + 1.7555x + 39.093, R 2 = 0.9393 for overweight or obese where x = year-1996.
Preliminary estimates for the first quarter of 2013 were also included for current smoking and overweight or obese; to examine which of the adjustment methods would best predict the 2013 estimates. Both methods were very close for overweight and obese, and the backcasting method was slightly better for current smoking. However, both methods had similar trajectories into the future and both were getting further away from the landline frame trajectory.

Discussion
We had previously found that the dual-frame gave a more representative sample [11]. The type of phone-use estimates from the 2012 NSWPHS were similar to those published for Australia in 2012 by ACMA from the Roy Morgan Single Source Survey-19.9% for mobile-only, 8.0% for landline-only and 69.8% for dual phone users [19]. In this study, we found relative differences of over 50% between people who were covered by the landline frame and mobile-only phone users for health indicators: five or more drinks of alcohol in a day, and current smoking. We also found that type of phone-use was associated with many of the health indicators, in particular mobile-only phone users were significantly different for: drink five or more drinks of alcohol in a day, current smoking, recommended vegetable intake, and overweight or obese, even after adjusting for the weighting variables. Our results were consistent with other studies [10,20,21].
When we compared the health indicators estimates from the 2012 NSWPHS to the 2011 NSWPHS, we found significant differences for recommended fruit intake, recommended vegetable intake, current smoking, positive self-reported health status, and overweight or obese. However, when we compared the health indicators estimates using only the landline frame sample, re-benchmarked to the NSW population, to the 2011 NSWPHS, current smoking, positive self-reported health status, and overweight or obese were no longer significantly different, and the difference had changed in direction, for current smoking and overweight or obese.
So, how do we interpret these changes? Did current smoking really increase in 2012, and did overweight or obese really decrease in 2012, or is it a consequence of the design change? Our examination of the time series for current smoking and overweight or obese showed that it was a consequence of the design change and not a real change.
The backcasting method was best able to predict the 2013 figures for current smoking. This method appears superior to the minimal coverage method in that it not only corrects the years when the landline frame coverage was sub-optimal, but it also adjusts the estimates to what they should have been for all the other years, if mobile-only phone users were included. However the backcasting method also requires the making of several assumptions, that being that the relative difference between people covered by the landline phone frame and mobile-only phone users has remained constant over time, and, that the landline phone coverage estimates for Australia were appropriate for NSW. Also, for the backcasting method, a more complex formula would need to be used if it was being used for demographic groups-requiring C A t for each group which is not currently available-and could quite quickly become very complex with numerous assumptions needing to be made. The minimal coverage method does not require any additional assumptions to be made; it just requires a decision on what is considered adequate population coverage by the sample frame or frames. Figure 2 shows that the difference between the landline frame time series and the adjusted dual-frame time series, for current smoking and overweight or obese, are widening over time. A recent study on the European telephone surveys has concluded that coverage bias from surveys using only landline frames in Europe are increasing over time [22]. This study highlighted the need for mobile telephone number augmentation of the sample to occur prior to the landline phone coverage becoming sub-optimal.

Conclusions
The inclusion of the mobile telephone numbers through an overlapping dual-frame design did impact on the time series for some of the health risk factors and health status estimates, in that it corrected the estimates that were being calculated from a sample frame, which was getting progressively less representative of the population. Therefore, continuing to use only landline frames in Australia, although maintaining the same design, is not keeping the estimates the same because of the decreasing coverage of landline frames.

Key message
Health estimates from surveys using only landline frame sampling are progressively getting "further from the truth".