- Research note
- Open Access
Validation of the REduction of Atherothrombosis for Continued Health (REACH) prediction model for recurrent cardiovascular disease among United Arab Emirates Nationals
BMC Research Notes volume 13, Article number: 484 (2020)
Prediction models assist health-care providers in making patient care decisions. This study aimed to externally validate the REduction of Atherothrombosis for Continued Health (REACH) prediction model for recurrent cardiovascular disease (CVD) among the Emirati nationals.
There are 204 patients with established CVD, attending Tawam Hospital from April 1, 2008. The data retrieved from electronic medical records for these patients were used to externally validate the REACH prediction model. Baseline results showed the following: 77.0% were men, 69.6% were diagnosed with coronary artery disease, and 87.3% have a single vascular bed involvement. The risk prediction model for cardiovascular mortality performed moderately well [C-statistic 0.74 (standard error 0.11)] in identifying those at high risk for cardiovascular death, whereas for recurrent CVD events, it performed poorly in predicting the next CVD event [C-statistic 0.63 (standard error 0.06)], over a 20-month follow-up. The calibration curve showed poor agreement indicating that the REACH model underestimated both recurrent CVD risk and cardiovascular death. With recalibration, the REACH cardiovascular death prediction model could potentially be used to identify patients who would benefit from aggressive risk reduction.
Prediction models assist health-care providers in making important daily patient care decisions by population screening, assessing responses to treatment, intensifying management, and motivating patients toward behavior change . In addition, prediction models help risk-stratify patients into low, moderate, high, or very high-risk groups for appropriate prevention and management strategies [1, 2]. Those deemed high- or very-high-risk individuals need far more aggressive management and monitoring . Having the diagnosis of cardiovascular disease (CVD) puts patients at an increased risk for a recurrent CVD event . In a previous study, ischemic heart disease was projected to increase by 138% and 158% for women and men, respectively, from 1990 to 2020 in the Middle East . Although alarming, these figures are likely to increase or decrease in the future as the pattern of CVD risk continues to change in most of the developing countries [4, 5].
Wilson et al. developed a risk prediction model for recurrent CVD events or cardiovascular death, with participants from the REduction of Atherothrombosis for Continued Health (REACH) Registry . The researchers conducted and internally validated their risk prediction tool on 68,000 patients from 44 countries. These include patients from the Middle East , thus making it the most globally inclusive, ethnically diverse, and geographically extensive registry of CVD patients. The patient population from the Middle East made up 1.25% (850) of the total sample. It included countries such as Saudi Arabia [198 participants (0.29%)] and the United Arab Emirates (UAE) [149 participants (0.22%)] . This study revealed that the risk for recurrent CVD was high among the Middle Eastern population . Therefore, based on the geographical region and availability of similar data as defined in the REACH model, it played a significant role in our decision to externally validate this risk tool in our clinical setting. Empirical evidence shows that models tend to perform better when internally validated and not as well in other cohorts in which the model may be applied [8,9,10,11]. This illustrates the need for external validation (discrimination and calibration) among patients exclusively from the UAE in order to assess its fitness for recurrent CVD risk prediction. Thus, this study aimed to externally validate the REACH prediction tool among UAE nationals with established CVD.
Study design, data source, and study cohort
This is an external validation study. The study site was Tawam Hospital’s outpatient clinics. Tawam Hospital is the largest publicly funded teaching hospital in Al Ain providing health care exclusively to UAE nationals. It is affiliated with the College of Medicine and Health Sciences, UAE University. UAE nationals are defined as the citizens of the UAE.
All UAE nationals enrolled in this study were diagnosed with a CVD event, who were visiting the outpatient clinics over a 9-month period in 2008. The recruitment was done retrospectively using electronic medical records (EMRs). The inclusion criteria for this study were matched to that of the developmental REACH cohort . It included patients aged 45 years and older with established CVD. Patients with CVD are defined as patients with cerebrovascular disease, coronary artery disease (CAD), or peripheral vascular disease (PVD), based on the documented diagnosis by a physician. There were 206 patients who met the inclusion criteria, but 2 patients with missing data at baseline were excluded.
Baseline data were obtained from the EMRs. The sociodemographic and clinical variables were as follows: age, sex, body mass index (BMI), smoking status, number of vascular beds involved, geographic region (in this case, the UAE), and history of diabetes mellitus, congestive heart failure (CHF), atrial fibrillation (AF), a baseline cardiovascular event (defined as a documented diagnosis of myocardial infarction or cerebrovascular disease) occurring in the previous 24 months, antihypertensive and lipid-lowering medications, and acetylsalicylic acid therapy. CHF and AF were not defined as a CVD event by Wilson et al.  In this study, smokers had either a history of smoking or currently smoking, and BMI was stratified as < 20, 20–30, and > 30 kg/m2. Diabetes mellitus was established by a documented clinician’s diagnosis. Patients were classified as having hypertension, when they have a diastolic blood pressure ≥ 90 mmHg or systolic blood pressure ≥ 140 mmHg or are on antihypertensive medications.
Although participants were followed up for 24 months, the cutoff used was 20 months as defined by Wilson et al.  Cardiovascular death was defined as the occurrence of a fatal MI or fatal stroke, whereas a recurrent CVD event was the occurrence of cardiovascular death, a cerebrovascular disease, or a myocardial infarction (MI) event.
Baseline characteristics of the validation cohort were compared to the developmental cohort. Student’s t-test was used for continuous variables, while Fisher’s exact test or chi-square was used for categorical variables. The time between study enrolment and the first occurrence of a recurrent CVD event during the follow-up period was defined as the months at risk for developing the endpoints.
The probability of each participant developing each endpoint was calculated by applying the regression coefficients of the original Cox regression models for recurrent CVD and cardiovascular death in the developmental study. External validation is performed by evaluating the REACH model’s predictive performance by calibration and discrimination . Calibration and discrimination were evaluated at 20 months of follow-up. Discrimination is defined as the capability to correctly differentiate between two participants based on a risk score, i.e., one who will not develop the endpoint and one who will over the follow-up duration . On the other hand, calibration is defined as the degree of agreement between the risks predicted by the model and the observed frequencies of the outcomes under study. Discrimination was calculated using Harrell’s C-statistic with standard error (SE) . Calibration was assessed graphically by comparing the predicted 20-month events with the observed events, grouped by quintiles of predicted probabilities. The R software version 3.5.2, using the packages “rms” and “riskRegression,” was used to analyze our data.
Table 1 shows the baseline characteristics of both the developmental  and validation cohorts. The validation sample size included 204 participants, of which 77% were men, with a mean age of 67 years at baseline. Participants with CAD, cerebrovascular disease, and PVD together formed the CVD burden at 69.6, 38.2, and 6.4%, respectively. On the other hand, those with one vascular bed involvement were the most prevalent at 87.3%. Among the study cohort, 32% had experienced a CVD event in the previous year, whereas 10.3 and 6.4% had CHF and AF, respectively. Almost 26% were smokers, and approximately 63% were patients with diabetes mellitus. A very high percentage were on antihypertensive medications (93.6%), lipid-lowering medications (85.3%), and only 52.5% on acetylsalicylic acid (commonly known as aspirin).
Additional file 1: Table S1 outlines the number of recurrent CVD events and cardiovascular deaths observed with almost 4% having a non-fatal MI and 3% a fatal MI.
C-statistic was used to assess the association between the individuals from the validation sample and the developmental sample over 20 months. In the model performance analysis for recurrent CVD events, the C-statistic was 0.63 (SE 0.06) for a 20-month risk. Thus, the model performed poorly in predicting the next CVD event. On the other hand, for cardiovascular death, discrimination for a 20-month risk prediction was moderate, with a C-statistic of 0.74 (SE 0.11). Thus, the model performed moderately well to identify patients at the highest risk for cardiovascular mortality.
The calibration curves for predicting the 20-month survival probability for recurrent CVD events and cardiovascular death are shown in Figs. 1 and 2, respectively. The calibration plot showed poor agreement between predicted recurrent CVD events and observed 20-month recurrent CVD event probabilities. In addition, the REACH model underestimated both the risk of recurrent CVD events and cardiovascular death.
Predicting death using the risk prediction model: an example
Developed by Wilson et al. , the recurrent risk prediction tool included the sum of numerical values assigned to the risk factors present in a patient to predict the next cardiovascular event. An arbitrary case in practice of a 45-year-old UAE man, with a history of diabetes mellitus, smoking, CHF, BMI > 20 kg/m2, and involvement of a single vascular bed, would have a 20-month risk of cardiovascular death of approximately 4.9%.
This study represents the first external validation of the REACH prediction model for recurrent CVD exclusively for UAE nationals. The external validation results performed moderately well in discriminating those participants likely to die from a recurrent CVD event from those who were more likely to survive, whereas the calibration performed poorly by underestimating the cardiovascular mortality risk, over a 20-month period of follow-up. This outcome is not unusual as research has shown that prognostic tools developed to predict mortality are less than perfect . Moreover, CVD risk prediction tools show a reduced accuracy when validated in new cohorts, mainly due to the differences in characteristics between the developmental and validation cohorts .
The notable differences between the developmental cohort and our study cohort could explain the underestimation of predicted risk of recurrent CVD and cardiovascular death seen in this study. In this study, patients were more likely to smoke and have diabetes mellitus, and only half were on aspirin therapy. Furthermore, the distribution of participants in the developmental model was across 44 countries. The types of included clinical practices were far more extensive, covering both urban and rural areas . Despite this, with recalibration, this model could have the potential to help clinicians identify individuals at the greatest short-term risk and, therefore, benefit from intense follow-up and management.
Using the REACH model, recurrent CVD risk prediction was found to demonstrate a modest discriminative capacity for predicting death, with an underestimation of CVD risk in the short term. Although the current application may fail to identify high-risk individuals, with recalibration, it may still be of value for close monitoring, intensification of risk factor management, and tertiary prevention. Following international guidelines for CVD, management should be the ultimate goal of primary care physicians and specialists alike.
The accuracy of documentation has to be accepted when using retrospective data. Differences in results may be attributed to a homogeneous case mix in our validation set. Also, the relatively low number of events may have affected the reliability of the analysis . Despite the small sample size and recruitment of UAE nationals from a single center, the performance with the new data may be sufficient for the model to be potentially useful in our population until further validation studies are conducted. We encourage other researchers to also externally validate the REACH risk prediction tool, particularly with respect to recurrent CVD events. Nevertheless, this is the first study that externally validated a model for clinicians treating an increasing number of high-risk patients with CVD in this region.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Body mass index
Coronary artery disease
Congestive heart failure
Electronic medical records
Peripheral vascular disease
United Arab Emirates
Smith GC, Seaman SR, Wood AM, Royston P, White IR. Correcting for optimistic prediction in small data sets. Am J Epidemiol. 2014;180:318–24.
Lin JS, Evans CV, Grossman DC, Tseng CW, Krist AH. Framework for using risk stratification to improve clinical preventive service guidelines. Am J Prev Med. 2018;54:S26-37.
Lin FJ, Tseng WK, Yin WH, Yeh H-I, Chen J-W, Wu C-C. Residual risk factors to predict major adverse cardiovascular events in atherosclerotic cardiovascular disease patients with and without diabetes mellitus. Sci Rep. 2017;7:9179.
Yusuf S, Reddy S, Ounpuu S, Anand S. Global burden of cardiovascular diseases: part 1: general considerations, the epidemiologic transition, risk factors, and impact of urbanization. Circulation. 2001;104:2746–53.
Gaziano TA, Bitton A, Anand S, Abrahams-Gessel S, Murphy A. Growing epidemic of coronary heart disease in low- and middle-income countries. Curr Probl Cardiol. 2010;35:72–115.
Wilson PW, D’Agostino R, Bhatt DL, Eagle K, Pencina MJ, Smith SC, Alberts MJ, Dallongeville J, Goto S, Hirsch AT, Liau CS, Ohman EM, Röther J, Reid C, Mas JL, Steg PG, REACH Registry. An international model to predict recurrent cardiovascular disease. Am J Med. 2012;125:695–703.
Bhatt DL, Steg PG, Ohman EM, Hirsch AT, Ikeda Y, Mas JL, Goto S, Liau CS, Richard AJ, Röther J, Wilson PW, REACH Registry Investigators. International prevalence, recognition, and treatment of cardiovascular risk factors in outpatients with atherothrombosis. JAMA. 2006;295:180–9.
Kaasenbrood L, Boekholdt SM, van der Graaf Y, Ray KK, Peters RJ, Kastelein JJ, Amarenco P, LaRosa JC, Cramer MJ, Westerink J, Kappelle LJ, de Borst GJ, Visseren FL. Distribution of estimated 10-year risk of recurrent vascular events and residual risk in a secondary prevention population. Circulation. 2016;134:1419–29.
Debray TP, Vergouwe Y, Koffijberg H, Nieboer D, Steyerberg EW, Moons KGM. A new framework to enhance the interpretation of external validation studies of clinical prediction models. J ClinEpidemiol. 2015;68:279–89.
Nieboer D, van der Ploeg T, Steyerberg EW. Assessing discriminative performance at external validation of clinical prediction models. PLoS ONE. 2016;11:e0148820.
van Eekelen R, McLernon DJ, van Wely M, Eijkemans MJ, Bhattacharya S, van der Veen F, van Geloven N. External validation of a dynamic prediction model for repeated predictions of natural conception over time. Hum Reprod. 2018;133:2268–75.
Royston P, Altman DG. External validation of a cox prognostic model: principles and methods. BMC Med Res Methodol. 2013;13:33.
Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: a framework for traditional andnovel measures. Epidemiology. 2010;21:128–38.
Harrell FE, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15:361–87.
Siontis GCM, Tzoulaki I, Ioannidis JPA. Predicting death: An empirical evaluation of predictive tools for mortality. Arch Intern Med. 2011;171:1721–6.
Mansell H, Stewart SA, Shoker A. Validity of cardiovascular risk prediction models in kidney transplant recipients. Sci World J. 2014;2014:750579.
Collins GS, Ogundimu EO, Altman DG. Sample size considerations for the external validation of a multivariable prognostic model: a resampling study. Stat Med. 2016;35:214–26.
We are indebted to Shehab Majud for his helpful comments on the language in the manuscript.
This study is funded by the UAE University (grant number: G00002956).
Ethics approval and consent to participate
Ethical approval to review and use the data was obtained from the United Arab Emirates University and Tawam Hospital Review Boards (CRD 239/13). Patient information was anonymized prior to the analyses to protect the patient’s identity, and thus, the informed consent was waived.
Consent to publish
The authors have no potential competing interests to disclose.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Al-Shamsi, S., Govender, R.D. Validation of the REduction of Atherothrombosis for Continued Health (REACH) prediction model for recurrent cardiovascular disease among United Arab Emirates Nationals. BMC Res Notes 13, 484 (2020). https://doi.org/10.1186/s13104-020-05331-8
- External validation
- Prediction model
- Recurrent cardiovascular disease
- United arab emirates