 Research article
 Open access
 Published:
Predicting preferencebased utility values using partial proportional odds models
BMC Research Notes volumeÂ 7, ArticleÂ number:Â 438 (2014)
Abstract
Background
The majority of analyses on utility data have used ordinary least square (OLS) regressions to explore potential relationships. The aim of this paper is to explore the benefits of response mapping onto health dimension profiles to generate preferencebased utility scores using partial proportional odds models (PPOM).
Methods
Models are estimated using EQ5D data collected in the Health Survey for England and the predicted utility scores are compared with those obtained using OLS regressions. Explanatory variables include age, acute illness, educational level, general health, deprivation and survey year. The expected EQ5D scores for the PPOMs are obtained by weighting the predicted probabilities of scoring one, two or three for the five health dimensions by the corresponding preferenceweights.
Results
The EQ5D scores obtained using the probabilities from the PPOMs characterise the actual distribution of EQ5D preferencebased utility scores more accurately than those obtained from the linear model. The mean absolute and mean squared errors in the individual predicted values are also reduced for the PPOM models.
Conclusions
The PPOM models characterise the underlying distributions of the EQ5D data better than models obtained using OLS regressions. Additional research exploring the effect of modelling conditional responses and two part models could potentially improve the results further.
Background
Many health care policy decisionmaking bodies require that economic evidence used to support submissions are reported in terms of the cost per quality adjusted life years (QALY) [1â€“5]. The QALY is a metric which combines both life expectancy and health related quality of life (HRQoL) where the HRQoL is informed by a preferencebased measure of health such as the EQ5D [6, 7]. As a consequence of submission requirements, there has been a substantial growth in the number of articles describing the results of statistical regression models involving preferencebased measures. This research may be performed either because the preferred preferencebased measure is not available in a particular clinical dataset, or the costeffectiveness model structure requires a method to predict changes in HRQoL values based on changes in clinical variables over time.
The majority of publications describing analyses of these types of data tend to describe regressions estimated using ordinary least squares (OLS) [8]. Although the statistical models obtained using OLS perform well on the aggregate level, they tend to underpredict at the top and overpredict at the bottom of the index and can predict values outside the actual index range [9, 10]. HRQoL data are generally multimodal with a mass at full health (i.e. EQ5Dâ€‰=â€‰1) and simple linear models are not appropriate [11]. Some researchers have explored alternatives such as generalised linear models with random effects, adjusted least square regression models, weighted least squares, Tobit models, and censored least absolute deviation models [8]. While these appear to provide little or no improvements in terms of predictive abilities or fit, an adjusted censored mixture model has been shown to improve predictive abilities compared to a model obtained using OLS in rheumatoid arthritis [11].
An alternative approach to mapping directly onto the preferencebased index is to predict the original responses to the health dimension questions (â€˜response mappingâ€™). Studies reporting these types of analyses have typically used multinomial logistic regressions and to date the benefits have appeared to be minimal [12, 13]. Multinomial logistic models are generally used when the dependent variable is nominal (i.e. the categories cannot be ordered in any meaningful way). As the responses for the health dimensions can be ranked according to the level of problems (i.e. none, some or extreme) an ordered logistic model (OLM) which ranks the responses for the health dimensions may be more appropriate. The aim of this paper is to develop the response mapping approach by exploring the use of partial proportional odds models (PPOM). These models are less restrictive than the parallel lines fit in the OLM but retain the ordered nature of the responses for the dependent variables [14]. The results obtained are compared to those obtained using OLS regressions as these are the most widely used in this area.
Methods
EQ5D
The EQ5D is a selfadministered questionnaire which covers five health dimensions: mobility, selfcare, usual activities, pain/discomfort, anxiety/depression. Each dimension in the descriptive system has three levels (no problems, some problems, extreme problems), producing a maximum of 243 (3^5) possible health states. Full health, with no impairment on all five dimensions, is defined by 11111. The worst possible health state, with maximum impairment on all five dimensions, is defined by 33333. EQ5D preferencebased utility values for each of the 243 health states are derived using the utility weights obtained from a sample of the British general population using timetradeoff methods [6]. The resulting EQ5D preferencebased index ranges from 1 (health state 11111) to0.594 (health state 33333).
Data
The Health Survey for England (HSE) is an annual crosssectional survey of randomly selected residents of private households in England. Ethical approval for the HSE was obtained from the London MultiCentre Research Ethics Committee and the data are freely available to download for academic purposes. Five cycles (years 2003, 2004, 2005, 2006 and 2008) of the HSE collected EQ5D data, a generic quality of life instrument, and these datasets were combined for the purpose of this study [15â€“19]. Analyses were conducted using subgroups of respondents who had completed the full EQ5D questionnaire and indicated they had one of the following prevalent selfreported limiting long term illnesses (LLTI): cardiovascular disease (CVD), diabetes, mental health conditions, musculoskeletal conditions, nervous system disorders, respiratory problems.
Based on their known effects on HRQoL, and at the request of the commissioners, the following variables were selected as explanatory variables in the regressions: age, age^{2}, education level, acute sickness within the previous two weeks, general health (selfreported by patients), and deprivation (based on the Overall Index of Multiple Deprivation) [20]. Education level was categorised as no formal qualification (base), GCSE O level or NVQ2/3 (GCSE), full time student or higher education below degree level including A level (A level), NVQ4/5 or degree (degree). Acute sickness was categorised as 0Â days (base), 1 to 6Â days (Sick 1), 6 to 13Â days (Sick 2), and the maximum recall period of 14Â days (Sick 3). General health was categorised as very good (GHVG), good (GHG), fair (base), bad (GHB), or very bad (GHVB), while deprivation was categorised as: least (base), a little deprived, very deprived, or most deprived (see Additional file 1).
Statistical models obtained for each of the LLTIs
A simple linear model was generated using an OLS regression. The dependent variable was the EQ5D preferencebased utility, with age, age^{2}, acute illness, education level, general health, deprivation, and survey year included as explanatory variables:
EQ5D represents the EQ5D preferencebased index, Î± represents the constant, the Î²â€™s represent the weights given to the various explanatory variables, and Ïµ represents the error term. As has been discussed in the literature, linear models such as these do not deal with the non normal characteristics typically observed in EQ5D data such as a mass at full health (EQ5Dâ€‰=â€‰1), a multimodal distribution, a long negative skew and the bounds of the index [11]. Predictions from these models are by definition concentrated around the mean (which is generally in the upper 50% of the index), consequently the models underpredict values at the top of the index and overpredict values at the bottom of the index [9]. This will be problematic when predicting mean values outside the inter quartile range or when predicting changes in mean values over time. In addition, when mean values are relatively high or low, the predicted values can exceed the limits of the index.
Williamâ€™s generalised ordered logit/partial proportional odds models (referred to as the PPOM) with three dependent variables (the probability of scoring none, some or extreme problems) were obtained [14] for each of the five EQ5D health dimensions:
Here the Î²â€™s reflect the weight given to the various independent variables and the kâ€™s define the separation between the probabilities (or cut points). In the case of the PPOM, the Î²s may differ across values of d (i.e. the regression lines may not be parallel). An advantage of using PPOMs is that it is more parsimonious than the generalised ordered logit model, where all the Î²s differ, but unlike the multinomial logit model, the order of the responses is retained.
EQ5D predictions for the OLS models were obtained in the normal manner using the OLS Î²s. As the logit models predict a range of probabilities on defined outcomes rather than a single value point, EQ5D predictions were calculated by estimating an expectation using the EQ5D scores for each of the 243 possible health states weighted by the probabilities of being in these health states. For example, if the models gave probabilities of 0.70, 0.2 and 0.1 for scoring level 1, 2 or 3 respectively for each of the five health dimensions, the preference based weights for each of dimensions levels were adjusted by the corresponding probability and the results summed to give the expected value. As this method produces the average expected score for each individual, as opposed to one of the 243 possible EQ5D scores, the predicted values were not expected to replicate the actual EQ5D scores exactly. For example, using the expected scores, it is not possible to generate a score of one.
Goodness of fit was assessed using standard summary statistics, and the ability of the statistical models to predict EQ5D scores was assessed using the mean absolute errors (MAE), and root mean squared errors (RMSE). Errors in predictions were compared for subgroups across the EQ5D range (EQ5Dâ€‰<â€‰0; 0â€‰â‰¤â€‰EQ5Dâ€‰<â€‰0.5; 0.5â€‰â‰¤â€‰EQ5Dâ€‰<â€‰0.75; EQ5Dâ€‰â‰¥â€‰0.75). The ability to represent the characteristics of the data was assessed graphically.
Results
Comparing the LLTIs, there are substantial differences in the proportions of respondents who have problems in each of the five health dimensions, reflecting the different aspects of health affected by the particular condition (TableÂ 1). For example 47% of respondents with respiratory conditions have problems with pain/discomfort compared to 80% of respondents with musculoskeletal conditions. Similarly, approximately 80% of respondents with mental health conditions have problems with anxiety/depression compared to 27% of respondents with CVD.
The individual EQ5D scores cover the full range (0.594 to 1) and there is little variation in the mean scores for the subgroups across the survey year (Additional file 1). The largest variation is observed in the subgroup with musculoskeletal conditions (range 0.610 in 2008 to 0.652 in 2003) whilst the smallest variation is observed in the subgroup with CVD (range 0.716 in 2006 to 0.740 in 2003). The EQ5D scores are not normally distributed irrespective of survey year or health condition and exhibit a long negative skew, a mass at full health, a second group centred around approximately 0.75 and a third group centred around approximately 0.2 (FigureÂ 1). The proportion of respondents scoring full health (TableÂ 1) is greatest in the subgroup with respiratory conditions (approximately 40%) and smallest in the subgroup with mental health conditions (approximately 13%).
Statistical models
The result for the musculoskeletal condition is used as an exemplar (TableÂ 2) as it has the largest sample and additional results are available online (Additional file 2). With the exception of the survey years, the majority of coefficients in the OLS models (TableÂ 2 and Additional file 2) are statistically significant (pâ€‰<â€‰0.05) and all have the expected sign. For example, the coefficients for acute illness (Sick 1, Sick 2 and Sick 3) are negative and significant and the coefficients for deprivation are positive and negative for the least deprived, and most deprived, reflecting the increase and decrease in EQ5D score respectively relative to the baseline (least deprived). The coefficients for the explanatory variables in the logit models are not as straightforward to interpret but if the relationship between the explanatory variable and HRQoL is negative, one would expect the corresponding coefficient in the logit model to be positive (increasing the probability of scoring 2 or 3 on the health dimension indicating a decrease in HRQoL).
The sign and statistical significance of the coefficients in the PPOMs (TableÂ 2, see Additional file 2) are both health dimension and condition specific, reflecting the differences in the relationships with the particular health dimensions for each of the health conditions. The coefficients in the PPOMs also demonstrate the relationships can vary substantially for the different levels within a health dimension, and in some cases can change direction. For example, looking at the mobility dimension coefficients for the survey years 2004 and 2008 for respiratory conditions. The negative coefficients for the first equation (contrasting no problem to some problems and extreme problems) indicate an increase in the likelihood of being in the current or lower category. Conversely, the positive coefficients for the second equation (contrasting no problem and some problem with extreme problems), indicate an increase in the likelihood of being in a higher category than the current one.
Errors in predicted values
While the OLS models are the most accurate in predicting the mean EQ5D score for each of the LLTIs and surveys (TableÂ 3), when comparing the dispersion of the actual and predicted EQ5D scores, the OLS predictions cover a much smaller range, do not predict negative scores and predict values greater than one. The errors in the values predicted using the PPOMs are substantially smaller than those predicted using the OLS irrespective of the LLTI. For the OLS predictions, MAEs (RMSEs) range from 0.149 to 0.212 (0.208 to 0.269) compared to 0.110 to 0.169 (0.142 to 0.206) for the PPOM predictions (TableÂ 3). When subgrouping by actual EQ5D score, the PPOM outperform the OLS in terms of both MAEs and RMSEs, with the RMSEs between 10% and 50% (for EQ5D scores smaller than 0) smaller for the PPOM predictions (TableÂ 4).
Discussion and conclusions
In this article we compare the results obtained when predicting EQ5D scores directly using the results of OLS regressions, to EQ5D scores obtained indirectly using response mapping and a methodology which retains the ordered nature of the original responses for the health dimensions. We found that while the OLS and PPOM models produce mean utility values that closely approximate the actual mean values, the OLS predict distributions that have little resemblance to the actual distributions. In direct comparison, the distributional characteristics (mass at full health, multimodel distribution, long negative skew) of the EQ5D data are captured and described better in the predictions from the PPOM models. The EQ5D scores estimated using the logit predictions are constrained by the EQ5D index while the OLS predictions are not and predict values greater than one. The RMSEs reflect these differences with substantially larger errors in the OLS predictions. This could be particularly important when using the models to estimate differences between subgroups in economic models, or when adjusting for casemix using individual predictions to compare differences between providers or changes over time.
When predicting scores for subgroups across the EQ5D range, the logit models outperform the OLS across all the LLTIs and compare favourably with results in the literature. For a subgroup with actual EQ5D scores below 0, when mapping between the SF12 and the EQ5D, Gray et al. report a mean squared error (MSE) and MAE of 0.166 and 0.304 respectively for response mapping using multinomial logit regressions (compared to 0.174 and 0.373 respectively for direct mapping using an OLS regression) [12]. Using the predictions for respondents with CVD as an exemplar, the MSE and MAE in our study were 0.077 and 0.245 respectively for response mapping using PPOMs (compared to 0.292 and 0.507 respectively for direct mapping using OLS regression). We chose to generate the expected values to enable comparison with the expected scores predicted from the OLS as opposed to simulating a sample of EQ5D scores which could explain the differences in results.
There are limitations with the dataset used in this study. For example, the HSE sampling mechanism excludes inhabitants of hospitals, residential, and nursing homes, hence the actual mean EQ5D scores for the LLTIs may be slightly lower than those reported in this study. In addition, the LLTIs are broadly defined and selfreported as opposed to clinically diagnosed by a doctor. This may introduce an element of bias as respondents may indicate they have a condition which has not been medically diagnosed. Conversely, respondents may indicate they do not have a particular condition which has actually been medically diagnosed. As the objective of the study was to compare the predicted values obtained using the different methodologies, as opposed to examining the mean EQ5D scores for particular conditions, the limitations with the data should not affect the findings of the analyses. However, the mean values for the particular subgroups may not be representative of the actual values.
The PPOM response mapping appears promising and there are several areas where additional research is warranted. A substantial proportion (range: 13% to 42%) of the respondents in each of the LLTI subgroups indicated they were at full health (EQ5Dâ€‰=â€‰1) which suggests that a twopart model may be appropriate. A second area that could be developed is the order in which the EQ5D questionnaire is completed (i.e. mobility followed by self care, usual activities, pain/discomfort, anxiety/depression). Intuitively one would expect that if someone scores no problem on the first four of the health dimensions, they are more likely to score no problems on the last dimension. Conversely, if they score extreme problems on the first four dimensions, it is unlikely that they will score no problems on the fifth dimension. We generated PPOMs for each health dimension independently, using the responses to the other four dimensions as explanatory variables. It is possible that results could be improved by capturing the conditional probabilities.
While the OLS results are more accurate on the aggregate level, there are additional benefits when using response mapping as opposed to mapping directly onto a preferencebased score. Firstly, the predictions from the PPOMs can be used in conjunction with alternative preference weights to generate country specific EQ5D scores. Secondly, the EQ5D index score is an aggregate measure of five different aspects of health. Predicting the overall mean EQ5D can mask changes in particular health dimensions and response mapping can provide additional information that would be lost at the summary level. The magnitude and direction of the coefficients in the PPOMs differed between equations for some health dimensions, reflecting the changes in the relationships when moving from one level of a health dimension to another and it would be difficult to capture these relationships in an OLS regression.
In summary, while the results presented here are promising, additional research exploring methods to improve the techniques used in responses mapping could improve results, increasing confidence in the predicted values when used to inform policy decision making of costeffectiveness interventions and when used to explore potential differences in health care providers.
Abbreviations
 CVD:

Cardiovascular disease
 HRQoL:

Health related quality of life
 HSE:

Health survey for England
 LLTI:

Limiting long term illnesses
 MAE:

Mean absolute errors
 MSE:

Mean squared error
 OLM:

Ordered logistic model
 OLS:

Ordinary least squares
 PPOM:

Partial proportional odds model
 QALY:

Quality adjusted life year
 RMSE:

Root mean squared error.
References
National Institute for Health and Clinical Excellence: Guide to the methods of technology appraisal. 2008, London, NICE: Ref N1618, Available from: http://www.nice.org.uk [Accessed 16th January 2012]
Canadian Agency for Drugs and Technologies in Health: Guidelines for the Economic Evaluation of Health Technologies: Canada. 2006, Ottawa: Canadian Agency for Drugs and Technologies in Health, http://www.cadth.ca/index. Accessed 17th May 2012, 3
Swedish Council on Health Technology Assessment: http://www.sbu.se/en/ accessed 17th May 2012
Institute for Quality and Efficiency in Health Care (IQWiG) in Germany: Methods and Tools. Available from: https://www.iqwig.de/en/home.2724.html. Accessed 17th May 2012
Johannesson M Australian Government Health Technology Assessment: The Australian Guidelines for subsidisation of pharmaceuticals: the road to costeffective prescribing?. Pharmacoeconomics. 1992, 2 (5): 355362. 10.2165/0001905319920205000003.
Dolan P, Gudex C, Kind P, Williams A: The time tradeoff method: results from a general population study. Health Econ. 1996, 5 (2): 14154. 10.1002/(SICI)10991050(199603)5:2<141::AIDHEC189>3.0.CO;2N.
Drummond MF, Oâ€™Brien B, Stoddart GL, Torrance GW: Methods for the Economic Evaluation of Health Care Programmes. 1997, Oxford: Oxford University Press, 2
Brazier JE, Yang Y, Tsuchiya A, Rowen DL: A review of studies mapping (or cross walking) nonpreference based measures of health to generic preferencebased measures. Eur J Health Econ. 2010, 11: 215225. 10.1007/s101980090168z.
Ara R, Brazier J: Deriving an algorithm to convert the eight mean SF36 dimensions scores into a mean EQ5D preferencebased score from published studies (where patient level data are not available. ViH. 2008, 12 (2): 346353.
Rowen D, Brazier J, Roberts J: Mapping SF36 onto the EQ5D Index: how reliable is the relationship?. Health Qual Life Outcomes. 2009, 7: 2710.1186/14777525727.
HernÃ¡ndez Alava M, Wailoo AJ, Ara R: Tails from the peak district: adjusted limited dependent variable mixture models of EQ5D questionnaire health state utility values. ViH. 2012, 15 (3): 550561.
Gray A, RiveroArias O, Clarke PM: Estimating the association between SF12 responses and EQ5D utility values by response mapping. Med Decision Making. 2006, 26: 1829. 10.1177/0272989X05284108.
Tsuchiya A, Brazier J, McColl E, Parkin D: Deriving preferencebased single indices from nonpreference based condition specific instruments: converting AQLQ into EQ5D indices. HEDS discussion paper (2002). Available at: http://eprints.whiterose.ac.uk/10952/. Accessed January 2012
Williams R: Generalized Ordered Logit/ Partial Proportional Odds Models for Ordinal Dependent Variables. Stata J. 2006, 6 (1): 5882. Available from http://www.nd.edu/~rwilliam/gologit2/
National Centre for Social Research and University College London: Department of Epidemiology and Public Health, Health Survey for England, 2004 [computer file]. 2010, Colchester, Essex: UK Data Archive [distributor], SN: 5439, 2
National Centre for Social Research and University College London: Department of Epidemiology and Public Health, Health Survey for England, 2003 [computer file]. 2010, Colchester, Essex: UK Data Archive [distributor], SN: 5098, 2
National Centre for Social Research and University College London: Department of Epidemiology and Public Health, Health Survey for England, 2005 [computer file]. 2010, Colchester, Essex: UK Data Archive [distributor], SN: 5675, 2
National Centre for Social Research and University College London: Department of Epidemiology and Public Health, Health Survey for England, 2006 [computer file]. 2010, Colchester, Essex: UK Data Archive [distributor], SN: 5809, 3
National Centre for Social Research and University College London: Department of Epidemiology and Public Health, Health Survey for England, 2008 [computer file]. 2010, Colchester, Essex: UK Data Archive [distributor], SN: 6397, 3
Communities and Neighbourhoods: Indices of Deprivation 2007. 2007, London: Department for Communities and Local Government,http://webarchive.nationalarchives.gov.uk/,
Funder for research
The research described in this article was funded by the Department of Health in England under the Policy Research Unit in Economic Evaluation of Health and Care Intervention (EEPRU) based at the University of Sheffield and University of York. The sponsors had no involvement in the analysis or interpretation of the data or findings presented here or the decision to submit the article for publication.
Health Survey for England data
The Health Survey for England is commissioned by the Department of Health and conducted by the Joint Health Survey Unit of National Centre for Social Research and Department of Epidemiology and Public Health at University College London. The original data creators, depositors or copyright holders, the funders of the Data Collections (if different) and the UK Data Archive bear no responsibility for the analysis presented here.
Author information
Authors and Affiliations
Corresponding author
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors' contributions
RA designed the study, acquired the data, conducted the analyses, interpreted the data, and drafted the manuscript. BK contributed to both the data analyses and manuscript. BvH contributed to the design of the study and the manuscript, and provided support with the analyses and data interpretation. JEB participated in the design of the study and contributed to the manuscript. All authors read and approved the final manuscript.
Authorsâ€™ original submitted files for images
Below are the links to the authorsâ€™ original submitted files for images.
Rights and permissions
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.
About this article
Cite this article
Ara, R., Kearns, B., vanHout, B.A. et al. Predicting preferencebased utility values using partial proportional odds models. BMC Res Notes 7, 438 (2014). https://doi.org/10.1186/175605007438
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/175605007438