Diabetes experts' reasoning about diabetes prevention studies: a questionnaire survey
© Mühlhauser; licensee BioMed Central Ltd. 2008
Received: 28 April 2008
Accepted: 14 October 2008
Published: 14 October 2008
Presentation of results of diabetes prevention studies as relative risk reductions and the use of diagnostic categories instead of metabolic parameters leads to overestimation of effects on diabetes risk. This survey examines to what extent overestimation of diabetes prevention is related to overestimation of prevention of late complications.
Participants of two postgraduate courses in clinical diabetology in Austria (n = 69) and Germany (n = 31) were presented a questionnaire with 8 items at the beginning of the meetings. All 100 questionnaires were returned with 92 filled in completely. Participants were asked 1) to rate the importance of differently framed results of prevention studies and, for comparison, of the United Kingdom Prospective Diabetes Study (UKPDS), 2) to estimate to what extent late complications could be prevented by the achieved reductions in diabetes risk or HbA1c values, respectively.
Prevention of diabetes by 60% was considered important by 84% of participants and 35% thought that complications could be prevented by ≥ 55%. However, if corresponding HbA1c values were presented (6.0% versus 6.1%) only 19% rated this effect important, and 12% thought that late complications could be prevented by ≥ 55%. The difference in HbA1c of 0.9% over 10 years in the UKPDS was considered important by 75% of participants and 16% thought that complications ('any diabetes related endpoint') were reduced by ≥ 55% (correct answer <15% by 20% participants).
The novel key message of this study is that the misleading reporting of diabetes prevention studies results in overestimation of effects on late complications.
Diabetes health care professionals judge the benefit of preventive interventions substantially higher when changes in diabetes risk are communicated rather than related glycaemia parameters. . In a recent study, we have surveyed participants of three European diabetes conferences. About 300 physicians and nurse educators were presented different formats of results of diabetes prevention studies. If results were communicated as a 57% reduction in diabetes risk about 90% of the diabetes experts interpreted the effect as important. In contrast, if the underlying changes of glucose or HbA1c values were presented, less than 40% rated the results as important . Transformation of continuous metabolic data into diagnostic categories interferes with understanding of study effects.
Framing of data is a well recognised cause of misconceptions about efficacy of health interventions by physicians [2–5], patients [2, 6], and health care decision makers . This may be particularly relevant to preventive medicine [8, 9]. Our survey has underscored that framing of data is also relevant for diabetes prevention .
However, it is not known why there are such impressive discrepancies in the ratings of the importance between changes in glycaemia parameters and risks of diabetes diagnosis. We hypothesised that diabetes health care providers may deduce from a 60% reduction in diabetes risk a comparably high reduction in diabetic late complications. Therefore, the aim of the present study was to further elucidate the framing of data in diabetes prevention and inferences by health care providers on the prevention of late complications.
The survey samples comprised participants of the 2007 EASD (European Association for the Study of Diabetes) Postgraduate Course in Evidence-based Clinical Diabetology in Austria (n = 69), and participants of the 2007 Annual Meeting of the Schleswig-Holstein Association of diabetologists in Northern Germany (n = 31). At both conferences diabetes prevention was a main topic. With few exceptions participants were certified diabetologists or other practising physicians with a special interest in diabetes care. The author was an invited speaker and not involved in the organisation of the meetings. She asked the audience for participation in the survey at the very beginning of the meetings. A questionnaire with 8 items was presented. In the Austrian meeting the questionnaires were analysed and descriptive results were presented and discussed at the end of the meeting. In the German meeting which lasted only half a day results of the Austrian meeting were presented and discussed. In order to achieve high participation both surveys were completely anonymous. Therefore, no further details on characteristics of participants were documented. Approval by an ethics committee was not considered necessary.
The questionnaire was in German language and based on the questionnaire which we had piloted and used in our first survey . Three of the 8 items were related to the primary outcome ('any diabetes related endpoint') of the UKPDS . The other five items referred to primary diabetes prevention trials [11–14]. Details on the publications which formed the basis of the questionnaires and the construction of the first questionnaire have been described previously . To minimise bias due to the sorting of items three versions of the questionnaire with different orders of items were used (A, B, C). The translated questionnaire (version A) with an explanation on how alternative presentations were derived is provided as additional file 1. Two items (1 and 6) were similar to the previous survey  to allow an estimate on the reproducibility of results. Identical wording was used in all items rating effects on diabetes risks or the primary outcome in the UKPDS: "How important do you consider the benefit of this intervention?" Participants were asked to mark their ratings as either "very important", "important", "not very important" or "not important at all". Wording on the effects related to prevention of late complications was also identical for all related items: "What is your estimate? On the long-term late complications could be reduced by ... " and participants were asked to score one answer among the following: "more than" 75%, "55 to 75%", "35 to 55%", "15 to 35%", "less than 15%".
Based upon a vast literature on framing of data [2–7] and the results of our previous survey  the following assumptions were made: 1) Items similar to those of the first survey (items 1 and 6) achieve comparable ratings as in the previous study . 2) Effect sizes are rated highest when presented as relative risk reduction (item 1), followed by decreasing importance when results are presented as HbA1c values (item 3) or natural frequencies (item 8) for the UKPDS primary endpoint, gain in life expectancy as estimated for primary prevention studies (item 5), and of lowest importance when glycosylated haemoglobin values (HbA1c) are presented for life style interventions in the primary diabetes prevention studies (item 6). 3) The extent to which late complications could be prevented would be rated highest for a 60% reduction of diabetes risk (item 2), lowest for a change in HbA1c values from 6.0% to 6.1% (item 7), and in between for the reduction of HbA1c by almost 1% over 10 years in the UKPDS (item 4).
Data collection and analysis
Data were analysed using SPSS, version 15.0 for windows. Comparable to our previous survey  only descriptive data are presented. No statistical comparisons between subgroups were made.
All 100 distributed questionnaires were returned, and except 8 all were filled in completely. Predefined assumptions were supported by the survey results (Table 1). Results were comparable across the two survey subpopulations (Table 1 ) and the three versions of the questionnaire (data not shown). 84% of participants considered a 60% reduction in diabetes risk as important (51% as "very important") but only 19% (2%) when a change in HbA1c of 0.1% was presented (Figure 1). When the result was presented as a 60% reduction in diabetes risk 35% of participants expected a reduction of long-term complications by ≥ 55% but only 12% did so when a decrease in HbA1c of 0.1% was used (Figure 1). The presentation of expected life years gained yielded results in between (Table 1).
Ratings of diabetes experts of study effects and associated risk reductions of late complications in relation to format of data presentation
Rating the benefit of the intervention as
Estimates of corresponding reductions of late complications
not very important
not important at all
60% diabetes risk reduction
(item 1 and 2)
HbA1c 6.0% versus 6.1%
(item 6 and 7)
0.288 life years gained
HbA1c 7.0% versus 7.9%
(item 3 and 4)
41 versus 46 out of 100 patients
The present study supports and extends the findings of our previous survey . Diabetes experts overestimate the benefit of preventive interventions when outcomes are presented as changes in diabetes risk rather than real metabolic changes. In addition, the findings further elucidate the reasoning of physicians confronted with different outcome descriptions. From a high reduction in diabetes risk they wrongly infer a high reduction of late complications. On the other hand, when the corresponding results of HbA1c values of the same large randomized trials are presented they expect a much lower reduction of late complications. Therefore, the original hypothesis of this survey is supported. Overall, reductions of late complications were substantially overestimated irrespective of outcome presentation although overestimation was highest when results were presented as reductions in diabetes risk and lowest when HbA1c vales were presented supporting the predefined assumptions of this study.
Cutoffs for plasma glucose to diagnose diabetes have been repeatedly challenged as there is no abrupt glycaemic threshold above which diabetes complications increase . Therefore, transition from impaired glucose tolerance to diabetes may be considered as a change in diagnostic labelling rather than change from a health to a disease state. Cost-effectiveness analyses have estimated that the Diabetes Prevention Program life style intervention would reduce a high-risk person's 30-year chances of a serious complication from about 38% to 30% which relates to an absolute risk reduction of 8% and a relative risk reduction of 21% . In some of the publications of the primary prevention studies crucial metabolic data are not communicated or difficult to extract . The Finish Diabetes Prevention Study included HbA1c as a secondary outcome measure  but did not report results in the main publications [11, 20]. Neither blood glucose nor HbA1c values have been reported in the core publication of the STOP-NIDDM Acarbose prevention study . Despite our previous survey  meta-analyses of diabetes prevention studies continue to be published without reporting or commenting on metabolic parameters . Incomplete reporting of outcomes within published articles of randomised trials is common and is associated with statistical non-significance . Biased reporting and framing of data by researchers and authors may enhance misconceptions about treatment effects among users of study results [1–7].
The present study has limitations. The study population is a convenience sample and includes only German speaking diabetes experts whereas the previous survey included both nurses and physicians of international diabetes meetings. In contrast to our first survey detailed characteristics of participants were not assessed in the present study. However, the previous survey also included a well characterised subgroup of 101 German diabetes experts who were participants of the 2005 EASD (European Association for the Study of Diabetes) Postgraduate Course in Evidence-based Clinical Diabetology in Jena, Germany, an equivalent of the Austrian meeting evaluated in the present survey. Among the Jena sample of diabetes experts about 40% worked at primary-care centres, 19% at non-university hospitals, 27% at university or research institutions, and the rest at other institutions. On average, they had worked for about 9 years in clinical diabetology and 57% reported to have participated in a course of evidence-based medicine . Results between both surveys were comparable on related items. In our previous study a reduction in diabetes risk of about 60% was rated important by 92% and a difference in HbA1c between 6.0% and 6.1% by 18% of participants  compared to 84% and 19%, respectively, in the present survey. In both surveys we have used data from the UKPDS. The UKPDS is a landmark study familiar to all diabetologists. Nevertheless, in our previous survey only 6 out of 299 diabetologists and diabetes educators could approximately state the frequencies for the primary outcome (41 versus 46 out of 100). Of those who responded most participants substantially overestimated the intervention effect. Also, in the present survey the effect of the intervention was overestimated. Only 20% gave the correct answer (less than 15%). Despite these similarities between the two surveys results of the present study may not be representative for other countries and other groups of physicians or health care providers.
This survey provides further insights into the reasoning of physicians. Diabetes experts overestimate the benefit of diabetes prevention studies irrespective of outcome presentation. The novel key message of this study is that from a high reduction of diabetes risk they wrongly infer a high reduction of late complications. The findings again urge presentation of study results in a format that can be understood by health care professionals. Understanding of prevention studies is fundamental for appropriate allocation of resources.
United Kingdom Prospective Diabetes Study
I thank the organizers of the two diabetes conferences and all participants for cooperation in this survey.
- Mühlhauser I, Kasper J, Meyer G, FEND: Understanding of diabetes prevention studies: questionnaire survey of professionals in diabetes care. Diabetologia. 2006, 49: 1742-1746. 10.1007/s00125-006-0290-8.View ArticlePubMedGoogle Scholar
- Edwards A, Elwyn G, Covey J, Matthews E, Pill R: Presenting risk information a review of the effects of "framing" and other manipulation on patient outcomes. J Health Commun. 2001, 6: 61-82. 10.1080/10810730150501413.View ArticlePubMedGoogle Scholar
- Forrow L, Taylor WC, Arnold RM: Absolutely relative: how research results are summarized can affect treatment decisions. Am J Med. 1992, 92: 121-124. 10.1016/0002-9343(92)90100-P.View ArticlePubMedGoogle Scholar
- Bucher HC, Weinbacher M, Gyr K: Influence of method of reporting study results on decision of physicians to prescribe drugs to lower cholesterol concentration. BMJ. 1994, 309: 761-764.PubMed CentralView ArticlePubMedGoogle Scholar
- Hoffrage U, Lindsey S, Hertwig R, Gigerenzer G: Communicating statistical information. Science. 2000, 290: 2261-2262. 10.1126/science.290.5500.2261.View ArticlePubMedGoogle Scholar
- Halvorsen PA, Selmer R, Kristiansen IS: Different ways to describe the benefits of risk-reducing treatments. A randomized trial. Ann Intern Med. 2007, 146: 848-856.View ArticlePubMedGoogle Scholar
- Fahey T, Griffiths S, Peters TJ: Evidence based purchasing: understanding results of clinical trials and systematic reviews. BMJ. 1995, 311: 1056-1060.PubMed CentralView ArticlePubMedGoogle Scholar
- Sackett DL: The arrogance of preventive medicine. CMAJ. 2002, 167: 363-364.PubMed CentralPubMedGoogle Scholar
- Sox HC: Straight talk about disease prevention. Ann Intern Med. 2007, 146: 891-892.View ArticlePubMedGoogle Scholar
- UK Prospective Diabetes Study (UKPDS) Group: Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33). Lancet. 1998, 352: 837-853. 10.1016/S0140-6736(05)61359-1.View ArticleGoogle Scholar
- Tuomilehto J, Lindström J, Eriksson J-G, Valle TT, Hämäläinen H, Ilanne-Parikka P, Keinänen-Kiukaanniemi S, Laakso M, Louheranta A, Rastas M, Salminen V, Uusitupa M, Finnish Diabetes Prevention Study Group: Prevention of type 2 diabetes mellitus by changes in lifestyle among subjects with impaired glucose tolerance. NEJM. 2001, 344: 1343-1350. 10.1056/NEJM200105033441801.View ArticlePubMedGoogle Scholar
- Lindström J, Louheranta A, Mannelin M, Rastas M, Salminen V, Eriksson J, Uusitupa M, Tuomilehto J, Finnish Diabetes Prevention Study Group: The Finnish diabetes prevention study (DPS). Diabetes Care. 2003, 26: 3230-3236. 10.2337/diacare.26.12.3230.View ArticlePubMedGoogle Scholar
- Diabetes Prevention Program Research Group: Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. NEJM. 2002, 346: 393-403. 10.1056/NEJMoa012512.PubMed CentralView ArticleGoogle Scholar
- Eddy DM, Schlessinger L, Kahn R: Clinical outcomes and cost-effectiveness of strategies for managing people at high risk for diabetes. Ann Intern Med. 2005, 143: 251-264.View ArticlePubMedGoogle Scholar
- Chiasson J-L, Josse RG, Gomis R, Hanefeld M, Karasik A, Laakso M, for The STOP-NIDDM Trial Research Group: Acarbose for prevention of type 2 diabetes mellitus: the STOP-NIDDM randomised trial. Lancet. 2002, 359: 2072-2077. 10.1016/S0140-6736(02)08905-5.View ArticlePubMedGoogle Scholar
- Mühlhauser I: Acarbose for type 2 diabetes prevention. Lancet. 2002, 360: 517-10.1016/S0140-6736(02)11458-9.View ArticleGoogle Scholar
- Tuomilehto J, Wareham N: Glucose lowering and diabetes prevention: are they the same?. Lancet. 2006, 368: 1218-1219. 10.1016/S0140-6736(06)69421-X.View ArticlePubMedGoogle Scholar
- Quresh M, Evans A: Retinopathy, plasma glucose, and the diagnosis of diabetes. Lancet. 2008, 371: 700-702. 10.1016/S0140-6736(08)60318-9.View ArticleGoogle Scholar
- Eriksson J, Lindström J, Valle T, Aunola S, Hämäläinen H, Ilanne-Parikka P, Keinänen-Kiukaanniemi S, Laakso M, Lauhkonen M, Lehto P, Lehtonen A, Louheranta A, Mannelin M, Martikkala V, Rastas M, Sundvall J, Turpeinen A, Viljanen T, Uusitupa M, Tuomilehto J: Prevention of Type II diabetes in subjects with impaired glucose tolerance: the Diabetes Prevention Study (DPS) in Finland. Study design and 1-year interim report on the feasibility of the lifestyle intervention programme. Diabetologia. 1999, 42: 793-801. 10.1007/s001250051229.View ArticlePubMedGoogle Scholar
- Lindström J, Ilanne-Parikka P, Peltonen M, Aunola S, Eriksson JG, Hemiö K, Hämäläinen H, Härkönen P, Keinänen-Kiukaanniemi S, Laakso M, Louheranta A, Mannelin M, Paturi M, Sundvall J, Valle TT, Uusitupa M, Tuomilehto J, Finnish Diabetes Prevention Study Group: Sustained reduction in the incidence of type 2 diabetes by lifestyle intervention: follow-up of the Finnish Diabetes Prevention Study. Lancet. 2006, 368: 1673-1779. 10.1016/S0140-6736(06)69701-8.View ArticlePubMedGoogle Scholar
- Gillies CL, Abrams KR, Lambert PC, Cooper NJ, Sutton AJ, Hsu RT, Khunti K: Pharmacological and lifestyle interventions to prevent or delay type 2 diabetes in people with impaired glucose tolerance: systematic review and meta-analysis. BMJ. 2007, 334: 299-10.1136/bmj.39063.689375.55.PubMed CentralView ArticlePubMedGoogle Scholar
- Chan AW, Altman DG: Identifying outcome reporting bias in randomised trials on PubMed: review of publications and surveys of authors. BMJ. 2005, 330: 753-756. 10.1136/bmj.38356.424606.8F.PubMed CentralView ArticlePubMedGoogle Scholar