Skip to main content

Development and validation of an equation to predict the incidence of coronary heart disease in patients with type 2 diabetes in Japan

Abstract

Objective

In the diabetes treatment policy after the Kumamoto Declaration 2013, it is difficult to accurately predict the incidence of complications in patients using the JJ risk engine. This study was conducted to develop a prediction equation suitable for the current diabetes treatment policy using patient data from Kitasato University Kitasato Institute Hospital (Hospital A) and to externally validate the developed equation using patient data from Kitasato University Hospital (Hospital B). Outlier tests were performed on the patient data from Hospital A to exclude the outliers. Prediction equation was developed using the patient data excluding the outliers and was subjected to external validation.

Results

By excluding outlier data, we could develop a new prediction equation for the incidence of coronary heart disease (CHD) as a complication of type 2 diabetes, incorporating the use of antidiabetic drugs with a high risk of hypoglycemia. This is the first prediction equation in Japan that incorporates the use of antidiabetic drugs. We believe that it will be useful in preventive medicine for treatment for people at high risk of CHD as a complication of diabetes or other diseases. In the future, we would like to confirm the accuracy of this equation at other facilities.

Introduction

It is important to prevent the development of diabetic complications during diabetes treatment [1]. One of the methods to predict the incidence of diabetic complications is through the use of risk engine, which is used to personalize medicine for patients. Currently, several risk engines have been developed to predict the incidence of diabetic complications [1,2,3,4,5]. In 2012, the Japan Diabetes Complications Study (JDCS)/The Japanese Elderly Diabetes Intervention Trial (J-EDIT) risk engine (JJ risk engine) was developed to accurately predict macro- and microvascular complications in Japanese patients with type 2 diabetes [1]. However, after the Kumamoto Declaration 2013, this risk engine was evaluated only through internal validation, without considering hypoglycemia prevention as the priority, and the implementation of external validation has been a challenge [1, 6]. In our previous study, we externally validated the prediction accuracy of the JJ risk engine using data from patients with type 2 diabetes at Hospital A. The results showed that the prediction of the JJ risk engine and the actual frequency of diabetic complications in Hospital A diverged [7]. Although the cause of this discrepancy is unknown, one reason may be the change in diabetes treatment to one that emphasizes hypoglycemia prevention [6, 8,9,10]. Therefore, we conclude that it is difficult to accurately predict the complication rates in patients using the JJ risk engine based on the diabetes treatment policies after the Kumamoto Declaration of 2013 [7].

Besides hypoglycemia, other risk factors for coronary heart disease (CHD) include aging, hypertension, hyperlipidemia, obesity, and chronic kidney disease (CKD) [11, 12]. Outlier tests were conducted for each risk factor, because prediction without outliers is more accurate than prediction with outliers [13, 14]. Therefore, in this study, we developed a new prediction equation that is more accurate and suitable for the patient population of Hospital A and externally validated the new prediction equation using patient data from Hospital B.

Main text

Methods

Hospital (target facility)

Kitasato University Kitasato Institute Hospital (Hospital A).

Kitasato University Hospital (Hospital B).

Patients

Selection criteria

The subjects were patients with type 2 diabetes who visited Hospital A or Hospital B from January 2013 to December 2013 and continued treatment for the following 5 years until 2018.

Exclusion criteria

Patients who refused to participate in the study or had a history of any of the following diseases were excluded: angina, myocardial infarction, stroke, peripheral arterial disease, familial hypercholesterolemia, familial type III hyperlipidemia, nephrotic syndrome, renal diseases other than diabetic nephropathy, microhematuria, pre-proliferative and proliferative retinopathy, or major ocular diseases (e.g., glaucoma, dense cataract, or a history of cataract surgery).

This study was conducted in accordance with the Ethical Guidelines for Medical and Health Research Involving Human Subjects. The Kitasato University Kitasato Institute Hospital, Research Ethics Committee, approved the study (Control Number: 20051 and 20051-2) and provided permission to review patient records and use the corresponding data. The option to opt-out of the study was provided to the patients at the start of the study (2021).

Statistical analysis

We developed a prediction equation based on the Cox proportional hazard model using patient data from Hospital A [15]. The backward stepwise method was used for the selection of variables [16].

Discrimination

It is an index that evaluates how accurately the presence or absence of an event can be predicted by a prediction model. The C-statistic, which is calculated based on the receiver operating characteristic (ROC) curve, is used as a criterion for measuring the predictive accuracy [17, 18].

Calibration

It is an index to measure the degree of agreement between the prediction by the model and the actual outcome. The significance probability calculated using the Hosmer–Lemeshow test is used as the criterion for predictability. The significance level was set at 0.05 (p < 0.05) [18, 19].

External validation

We developed prediction equations using Hospital A data and then performed external validation using Hospital B data.

Outlier testing using box plots

Outlier tests with box plots were performed to reduce the impact of outliers of each risk factor on the prediction accuracy.

A box-and-whisker diagram of data for age, total cholesterol, HDL cholesterol, body mass index (BMI), urinary albumin, upper quartile (Q3), and lower quartile (Q1) was generated, and values that deviated from the range of Q1–(Q3–Q1) to Q3 + (Q3–Q1) were considered outliers [20].

Although blood pressure was measured at the time of medical examination, it was excluded from the risk factors in this study because the time of measurement varied among subjects [21].

R version 2.5.1 (http://www.r-project.org, library Design, Hmisc, ROCR) was used to determine discrimination and calibration, whereas the ROC curve, Hosmer–Lemeshow test, and box plot were used for calculation [18,19,20].

Results

There were 572 and 285 patients in Hospitals A and B, respectively. The baseline characteristics of the patients are presented in Table 1.

Table 1 Baseline characteristics

Patients who used either sulfonylurea (SU) drugs or insulin were considered medicine users. Among the variables, only medicine was found to have a value of p < 0.05 (p = 0.03) (Additional file 1: Table S1). Therefore, only medicine was included as a variable in the prediction equation, and the prediction equation developed is as follows:

$$ \left[ {{\text{K}}{-}{\text{medicine equation}}} \right] $$
$$ \lambda_{{\text{t}}} \, = \,\lambda_{{0{\text{t}}}} \, \times \,{\text{exp }}\left\{ {\beta \, \times \,{\text{medicine }}\left( {0,{ 1}} \right)} \right\} $$

λt: Incidence rate by time t; λ0t: Baseline hazard for time t; β: partial regression coefficient; medicine (0, 1): 1 for patients who used either SU or insulin, 0 for patients who used neither.

The prediction equation using data from Hospital A resulted in a C-statistic of 0.734 and a calibration of p > 0.05, indicating no significant difference between the measured and predicted values. In contrast, external validation using data from Hospital B resulted in a C-statistic of 0.809 and a calibration of p < 0.05, indicating a significant difference between the measured and predicted values (Table 2).

Table 2 Development and external validation of prediction equations

Therefore, an outlier test using a box-and-whisker diagram was performed to improve the prediction accuracy. The outliers were age: 39 years or less; total cholesterol: ≥ 266 and ≤ 96; HDL cholesterol: ≥ 105.5; BMI: ≥ 35; and urine albumin: ≥ 78.7. A total of 120 patients from Hospital A and 84 patients from Hospital B were excluded.

In the analysis of the variables after exclusion, the p-value for medicine was < 0.05, indicating a significant difference.

The developed prediction equation is as follows:

$$ \lambda_{{{1825}}} \, = \,0.0{1}\, \times \,{\text{exp }}\left\{ {{1}.{73}\, \times \,{\text{medicine }}\left( {0,{ 1}} \right)} \right\} $$

λ1825: Incidence rate within 5 years; medicine (0, 1): 1 for patients who used either SU or insulin, 0 for patients who used neither.

The C statistic was 0.644 and the calibration was p > 0.05. There were no significant differences between the measured and predicted values. A total of 201 excluded patients from Hospital B were used for external validation; the C statistic was 0.750, and the calibration was p > 0.05. There were no significant differences between the measured and predicted values (Table 3).

Table 3 Development and external validation of prediction equation after outlier testing

Discussion

After the outliers were excluded, a prediction equation (K-medicine equation) was developed for the incidence of CHD in patients with type 2 diabetes using patient data from Hospital A. Furthermore, after excluding outliers, the external validation using data from Hospital B showed that the C-statistic was moderate and the calibration was not significantly different, indicating a correct prediction. Exclusion of outlier data for age, total cholesterol, HDL cholesterol, BMI, and urinary albumin levels was a condition used in this prediction equation.

Compared to the JJ risk engine, we incorporated the use of antidiabetic drugs into the prediction equation. In addition, while risk engines developed in other countries have incorporated therapeutic drugs (dyslipidemia drugs) into the prediction equation [22], our study is the first to incorporate therapeutic drugs (diabetes drugs) into the prediction equation in Japan. This prediction equation is intended for the primary prevention of CHD in Japanese patients with type 2 diabetes. Compared with other prediction formulae, this equation has the advantage that the variables can be selected according to the characteristics of each institution; the disadvantage is that the risk of developing CHD is calculated simultaneously prior to the administration of SU drugs and insulin, because SU drugs and insulin are introduced much later in diabetes treatment.

In contrast, validation using Hospital B patient data showed significant differences in the calibration, indicating an incorrect prediction. As per previous studies [11,12,13,14], the incorrect prediction might be owing to outliers of risk factors that influence the development of CHD, which affect the prediction accuracy.

In a previous study, the risk of developing CHD was higher in patients who used SU drugs and insulin than in those who received dipeptidyl peptidase-4 inhibitors [23], indicating that SU drugs and insulin are risk factors for CHD.

In the selection of variables, there was a significant difference in the presence or absence of SU drugs or insulin use, and it was reasonable to include it in the variables of the prediction equation. The risk of CHD associated with the use of SU drugs and insulin is consistent with the results of previous studies [23]. This prediction equation indicates that the use of diabetes medications with a high risk of hypoglycemia influences the development of CHD as a complication of type 2 diabetes.

Because of outliers, about 21% and 30% of patients were excluded in Hospital A and Hospital B, respectively. As more than 70% of the patients in each institution remained after exclusion, we believe that our prediction formula is applicable to many patients with type 2 diabetes.

In the variable analysis, there was no significant difference in the risk factors for CHD. Therefore, in this study, the risk factors for CHD were not included as a variable in the prediction equation; however, for patients with elevated laboratory values for risk factors for CHD, it will need to be considered as risk in the future. The column for LDL cholesterol under Hospital B (Table 1) is blank, because the LDL cholesterol values could not be obtained at this hospital. Since the blood pressure data presented in this study is only what was collected at the clinic, it is necessary to collect blood pressure data multiple times at the time of consultation and conduct additional analysis using more reliable blood pressure data [21].

The unique feature of this study is that the prediction equation was developed using patient data from a medium-sized hospital in Japan, rather than large-scale clinical data. Furthermore, we were able to create a prediction equation and validate externally. Although this prediction equation focuses on drugs that tend to cause hypoglycemia, which is a risk factor for the development of CHD, some hypoglycemic drugs, such as SGLT2 inhibitors and GLP-1 receptor agonists, reduce the risk of CHD [24]. In Hospital A, we have been using these drugs at full scale since 2016. Less than 9% of patients use these drugs; and therefore, it is difficult to consider the impact of these drugs in this study. The frequency of use of these drugs is expected to increase in the future; and it is necessary to consider the development of a prediction formula that includes them.

Conclusion

We developed an equation to predict the incidence of CHD in patients with type 2 diabetes. Based on the prediction equation developed in this study, we believe that the use of diabetic drugs with a high risk of hypoglycemia influences the incidence of CHD as a complication of type 2 diabetes. Although this prediction equation is based on the patient population of Hospital A, we would like to confirm the accuracy of our prediction theory in other institutions in the future.

Limitations

The analysis of this study included patients who were using medications to prevent cardiovascular disease, but the effects of these medications were not considered in the analysis. Consequently, the possibility that they may affect the outcome cannot be excluded. A history of CHD in the family could influence a patient's risk of developing CHD. However, in this study, we were unable to investigate family history, which could affect the results of the prediction equation. Therefore, family history should be considered in future studies.

The relatively small number of people who developed CHD may have affected the reliability of the analysis [25].

Availability of data and materials

The datasets generated and analyzed during the current study are not publicly available owing to the privacy of the research participants. Data are available from the corresponding author upon reasonable request; however, permission from the Kitasato University Kitasato Institute Hospital Research Ethics Committee is required.

Abbreviations

CHD:

Coronary heart disease

ROC:

Receiver operating characteristic

HbA1c:

Glycated hemoglobin

NGSP:

National Glycohemoglobin Standardization Program

HR:

Hazard ratio

SU:

Sulfonylurea

LDL:

Low-density lipoprotein

References

  1. 1.

    Tanaka S, Tanaka S, Iimuro S, Yamashita H, Katayama S, Akanuma Y, et al. Predicting macro- and microvascular complications in type 2 diabetes: the Japan Diabetes Complications Study/the Japanese Elderly Diabetes Intervention Trial risk engine. Diabetes Care. 2013;36:1193–9.

    Article  Google Scholar 

  2. 2.

    Stevens RJ, Kothari V, Adler AI, Stratton IM; United Kingdom Prospective Diabetes Study (UKPDS) Group. The UKPDS risk engine: a model for the risk of coronary heart disease in Type II diabetes (UKPDS 56). Clin Sci (Lond). 2001;101(6):671–9. Erratum in: Clin Sci (Lond). 2002;102(6):679.

  3. 3.

    Shao H, Fonseca V, Stoecker C, Liu S, Shi L. Novel risk engine for diabetes progression and mortality in USA: building, relating, assessing, and validating outcomes (BRAVO). Pharmacoeconomics. 2018;36:1125–34.

    Article  Google Scholar 

  4. 4.

    Montagud-Marrahi E, Molina-Andújar A, Pané A, Ruiz S, Amor AJ, Esmatjes E, et al. Impact of simultaneous pancreas-kidney transplantation on cardiovascular risk in patients with diabetes. Transplantation. 2021. https://doi.org/10.1097/TP.0000000000003710.

    Article  PubMed  Google Scholar 

  5. 5.

    Postmus D, van Veldhuisen DJ, Jaarsma T, Luttik ML, Lassus J, Mebazaa A, et al. The COACH risk engine: a multistate model for predicting survival and hospitalization in patients with heart failure. Eur J Heart Fail. 2012;14:168–75. https://doi.org/10.1093/eurjhf/hfr163.

    Article  PubMed  Google Scholar 

  6. 6.

    Nakanishi S, Hirukawa H, Shimoda M, Tatsumi F, Kohara K, Obata A, et al. Verification of kumamoto declaration 2013 and glycemic targets for elderly patients with diabetes in Japan for prevention of diabetic complications: a retrospective longitudinal study using outpatient clinical data. J Diabetes Investig. 2019;10:290–301.

    Article  Google Scholar 

  7. 7.

    Yamashita Y, Kitajima R, Matsubara K, Inoue G, Matsubara H. A retrospective study on the usefulness of the JJ risk engine for predicting the incidence rate of coronary heart disease in type 2 diabetes patients. BMC Res Notes. 2021;14:92.

    Article  Google Scholar 

  8. 8.

    Goto A, Arah OA, Goto M, Terauchi Y, Noda M. Severe hypoglycaemia and cardiovascular disease: systematic review and meta-analysis with bias analysis. BMJ. 2013;347:f4533.

    Article  Google Scholar 

  9. 9.

    Japan Diabetes Society. Treatment Guide for Diabetes, Bunkodo; 2020–2021.

  10. 10.

    Araki E, Haneda M, Kasuga M, Nishikawa T, Kondo T, Ueki K, et al. New glycemic targets for patients with diabetes from the Japan Diabetes Society. Diabetol Int. 2016;7:327–30.

    Article  Google Scholar 

  11. 11.

    The Japan circulation society, Shimamoto K, Arai H, Iso H, Ouchi Y. Guidelines for the primary prevention of ischemic heart disease revised version (JCS 2012).

  12. 12.

    Hubert HB, Feinleib M, McNamara PM, Castelli WP. Obesity as an independent risk factor for cardiovascular disease: a 26-year follow-up of participants in the Framingham Heart Study. Circulation. 1983;67:968–77.

    CAS  Article  Google Scholar 

  13. 13.

    Nnamoko N, Korkontzelos I. Efficient treatment of outliers and class imbalance for diabetes prediction. Artif Intell Med. 2020;104: 101815. https://doi.org/10.1016/j.artmed.2020.101815.

    Article  PubMed  Google Scholar 

  14. 14.

    Ferdowsi H, Jagannathan S, Zawodniok M. An online outlier identification and removal scheme for improving fault detection performance. IEEE Trans Neural Netw Learn Syst. 2014;25:908–19. https://doi.org/10.1109/TNNLS.2013.2283456.

    Article  PubMed  Google Scholar 

  15. 15.

    Fisher LD, Lin DY. Time-dependent covariates in the Cox proportional-hazards regression model. Annu Rev Public Health. 1999;20:145–57. https://doi.org/10.1146/annurev.publhealth.20.1.145.

    CAS  Article  PubMed  Google Scholar 

  16. 16.

    Kleinbaum DG, Kupper LL, Morgenstern H. Epidemiologic research: principles and quantitative methods. Belmont: Lifetime Learning Publications; 1982.

    Google Scholar 

  17. 17.

    Shintani A. Everyone's medical statistics Complete mastery of basic theory and EZR in 12 days. Kodansha; 2016.

  18. 18.

    Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15:361–87.

    Article  Google Scholar 

  19. 19.

    Hosmer DW, Lemeshow S. Applied logistic regression, 2nd ed. A Wiley-Interscience Publication: John Wiley & Sons, Inc; 2000.

  20. 20.

    Walfish S. A review of statistical outlier methods. Pharm Technol. 2006;30(11):82.

    Google Scholar 

  21. 21.

    Umemura S, Arima H, Arima S, Asayama K, Dohi Y, Hirooka Y, et al. The Japanese Society of Hypertension guidelines for the management of hypertension (JSH 2019). Hypertens Res. 2019;42:1235–481. https://doi.org/10.1038/s41440-019-0284-9.

    Article  PubMed  Google Scholar 

  22. 22.

    Rawshani A, Rawshani A, Franzén S, Sattar N, Eliasson B, Svensson AM, et al. Risk factors, mortality, and cardiovascular outcomes in patients with type 2 diabetes. N Engl J Med. 2018;379:633–44. https://doi.org/10.1056/NEJMoa1800256.

    Article  PubMed  PubMed Central  Google Scholar 

  23. 23.

    O’Brien MJ, Karam SL, Wallia A, Kang RH, Cooper AJ, Lancki N, et al. Association of second-line antidiabetic medications with cardiovascular events among insured adults with type 2 diabetes. JAMA Netw Open. 2018;1: e186125. https://doi.org/10.1001/jamanetworkopen.2018.6125.

    Article  PubMed  PubMed Central  Google Scholar 

  24. 24.

    American Diabetes Association. 9. Pharmacologic approaches to glycemic treatment: standards of medical care in diabetes. Diabetes Care. 2019;42(Suppl 1):S90–S102. https://doi.org/10.2337/dc19-S009.

  25. 25.

    Collins GS, Ogundimu EO, Altman DG. Sample size considerations for the external validation of a multivariable prognostic model: a resampling study. Stat Med. 2016;35:214–26.

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This research received no specific grant from any funding agency in public, commercial, or not-for-profit sectors.

Author information

Affiliations

Authors

Contributions

YY designed the study and wrote the initial draft of the manuscript. YN and RK contributed to data collection and interpretation. GI, KM, TH, JM, KA, and HM critically reviewed the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Yasunari Yamashita.

Ethics declarations

Ethics approval and consent to participate

This study was conducted in accordance with the Ethical Guidelines for Medical and Health Research Involving Human Subjects. The Kitasato University Kitasato Institute Hospital, Research Ethics Committee, approved the study (control number: 20051, 20051-2). For this retrospective study, we disclosed the purpose of the study to the participants and provided the option to opt-out before their participation. All participants provided written informed consent to participate in the study. In addition, the choice to opt-out provided participants an opportunity to refuse participation in the research.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Table S1.

Analysis results for variables.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yamashita, Y., Inoue, G., Nozaki, Y. et al. Development and validation of an equation to predict the incidence of coronary heart disease in patients with type 2 diabetes in Japan. BMC Res Notes 14, 426 (2021). https://doi.org/10.1186/s13104-021-05844-w

Download citation

Keywords

  • JJ risk engine
  • Type 2 diabetes
  • Coronary heart disease
  • Discrimination
  • Calibration
  • Hypoglycemia
  • C-statistic
  • Box plot
  • Outlier