Can a score derived from the Critical Care Minimum Data Set be used as a marker of organ dysfunction? – a pilot study
© Felton et al; licensee BioMed Central Ltd. 2009
Received: 27 November 2008
Accepted: 06 May 2009
Published: 06 May 2009
The aim of this study was to develop a simple organ score derived from the Critical Care Minimum Data Set (CCMDS) to compare with the Sequential Organ Failure Assessment (SOFA) score, a previously validated score of organ dysfunction.
The CCMDS collects data regarding the support of seven organ systems. To create a CCMDS derived score each level of organ support was allocated a numerical value. SOFA scores were collected retrospectively from each patient in the study. Data was collected in 50 sequential admissions over the first 5 days of their admission. This generated a total of 147 pairs of data for comparison.
Scatter plots and Spearman's rank correlation coefficient suggest a weak positive association between our CCMDS-derived score and the SOFA score. Daily Bland-Altman plots reveal minimal bias between the score but wide limits of agreement.
Our CCMDS-derived score cannot be regarded as an indicator of severity of organ dysfunction and cannot replace SOFA scores when a daily marker of organ dysfunction is required.
The Critical Care Minimum Data Set (CCMDS) was introduced in April 2006 and collates data daily regarding all patients, excluding neonates, in level 2 or 3 critical care beds in England and Wales. The CCMDS was developed by the Critical Care Information Advisory Group and is endorsed by the Intensive Care Society. The data consists of 34 fields which allow calculation of admission duration, basic demographic information, level of care and category of intensive care unit (general, neurosurgical, cardiothoracic etc). Information is also collected regarding the number of organs supported and, in the case of the cardiovascular and respiratory systems, the level of support. Data is currently submitted to the Intensive Care National Audit and Research Centre (ICNARC) as part of the Case Mix Programme. During 2008 data collection became mandatory and will be used to calculate a payment tariff under the Payment by Results (PbR) scheme within all NHS hospitals.
Our aim was to develop a simple organ score derived from the CCMDS to compare with a previously validated score of organ dysfunction. The Sequential Organ Failure Assessment (SOFA) score is a very commonly used scoring system designed to follow morbidity of critically ill patients. It records severity of dysfunction of six organ systems. Improvement in organ dysfunction scores, such as SOFA, monitor improvements in a patient's condition following an intervention or as an end point for clinical trials. More recently, organ dysfunction scores have been validated, not only as an initial indicator of outcome on admission to ICU, but also as a predictor of mortality, dependant on changes in daily scores during a patient's ICU admission [2–4]. Organ dysfunction scores are time consuming and expensive to collect, requiring additional clerical support so a CCMDS-derived score could represent a saving of both time and money to UK intensive care units especially those involved in clinical trials.
Summary of definitions of levels of organ support in the critical care minimum data set .
Definition of level of support
• > 50% oxygen delivered by face mask
• Close observation due to the potential for acute deterioration
• Physiotherapy or suction to clear secretions at least two hourly
• Patients recently extubated after a prolonged period of intubation and mechanical ventilation
• Mask CPAP or non-invasive ventilation.
• Patients who are intubated to protect the airway but needing no ventilatory support and who are otherwise stable
• Invasive mechanical ventilatory support
• Extracorporeal respiratory support
• Treatment of circulatory instability due to hypovolaemia
• Use of a CVP line for basic monitoring or central access
• Use of an arterial line for basic monitoring or sampling
• Single intravenous vasoactive
• Intravenous drugs to control cardiac arrhythmias
• Non-invasive measurement of cardiac
• Multiple intravenous vasoactive and/or rhythm controlling drugs
• Patients resuscitated after cardiac arrest where intensive therapy is considered clinically appropriate.
• Observation of cardiac output and derived
• Intra aortic balloon pumping.
• Temporary cardiac
• Placement of a gastrointestinal tonometer
• Acute renal replacement therapy
• Central nervous system depression sufficient to prejudice the airway and protective reflexes
• Invasive neurological monitoring
• Severely agitated or epileptic patients requiring constant nursing attention and/or heavy sedation
• Feeding with parenteral or enteral nutrition
• Patients with major skin rashes, exfoliation or burns
• Use of multiple, large trauma dressings
• Use of complex dressings
• Extracorporeal liver replacement device
System used for conversion from sedation score to neurological component of the SOFA score.
of SOFA score
1 (GCS 13–14)
1 (GCS 13–14)
0 (GCS 15)
Roused by voice
2 (GCS 10–12)
Roused by movement
3 (GCS 6–9)
Rouse by pain
3 (GCS 6–9)
4 (GCS < 6)
4 (GCS < 6)
The CCMDS-derived score
Our aim was to retrospectively collect SOFA scores and the CCMDS-derived score on fifty admissions to a teaching hospital general intensive care unit from January 1st 2007. The local ethics committee was informed and indicated that no formal approval of this retrospective study was required. Data was collected from day 0 to day 5 of each patient's admission. Seven of the original fifty patients had insufficient data to allow calculation of either a SOFA score or CCMDS derived score. In total 147 pairs of data, both SOFA and CCMDS-derived scores were collected with each patient contributing between two and six data points. Data was initially analysed for correlation and agreement by plotting a scatter plot of the scores of one method against those of the other for each day of admission. Subsequently a Spearman's rank test and a Bland and Altman plot were performed on data collected for each day to assess whether these two methods can be used interchangeably.
Spearman's rank correlation coefficient for the CCMDS-derived score compared to SOFA scores for day 0 to day 5.
Correlation coefficient (r value)
Levels of agreement derived from the Bland and Altman plots for the 5 days of admission.
Limits of Agreement
Day 0 to 1
-84% to 86%
Day 0 to 2
-58% to 65%
Day 0 to 3
-49% to 63%
Day 0 to 4
-59% to 78%
Day 0 to 5
-50% to 72%
The aim of our CCMDS-derived score was to produce a simple, daily scoring system that would monitor severity of organ dysfunction but require no additional administrative input and therefore no additional cost implications. The results show a statistical correlation between the CCMDS-derived score and SOFA score as indicators of organ dysfunction, confirming the ranking of patients is similar for the two scoring systems. The r2 values suggest that factors not included in the SOFA score affect our CCMDS-derived score. However, the Bland and Altman plots display a poor degree of agreement between the two scores but with minimal overall bias.
Based on this pilot data from 43 patients our CCMDS-derived score cannot be regarded as an indicator of the severity of organ dysfunction when compared with the SOFA score. It is our view, at present in critically ill patients, an organ dysfunction score derived form the CCMDS cannot replace the SOFA score in studies where a daily indicator of the severity of organ dysfunction is required. This may be explained by the different purpose of each score. The SOFA score was developed and validated as a tool to quantify organ dysfunction. In contrast, the CCMDS was developed to calculate the level of dependency of a patient as a way of calculating remuneration for critical care units. Whilst the level of organ dysfunction is related to the dependency of the patient the two scores quantify two distinct entities.
The sample size was set at 50 as this was a pilot study to determine the feasibility of using a CCMDS-derived score as an organ dysfunction score. Although a larger sample size may reduce the chance of a negative result being a type II error we feel that this is unlikely in view of the consistency of the results. The small sample size and original premise of this pilot study makes more detailed statistical analysis beyond the remit of this study. This does, however, limit the conclusions that can be drawn from the data. Based on the results of this study it is unlikely that a large prospective study with detailed statistical analysis would produce a satisfactory new organ dysfunction score based on the CCMDS. Specific weaknesses of the study are the derived neurological component of the SOFA score, its retrospective nature and small sample size. The Glasgow Coma Score is not routinely collected in the study intensive care unit thereby necessitating a conversion of sedation scores to GCS. A prospective study would be able to record GCS. However this would defeat the main intention of the study to record a score that requires no additional administrative costs. This study is to our knowledge, the first attempt to use the critical care minimum data set to predict organ dysfunction.
Critical Care Minimum Data Set
Sequential Organ Failure Assessment
Intensive Care National Audit and Research Centre
Payment by Results
Glasgow Coma Scale.
- Vincent J, Moreno R, Takala J, Willatts S, Mendonca AD, Bruining H, Reinhart C, Suter P, Thijs L: The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Medicine. 1996, 22: 707-710. 10.1007/BF01709751.View ArticlePubMedGoogle Scholar
- Pettila V, Pettila M, Sarna S, Voutilainen P, Takkunen O: Comparison of multiple organ dysfunction scores in the prediction of hospital mortality in the critically ill. Critical Care Medicine. 2002, 30: 1705-1711. 10.1097/00003246-200208000-00005.View ArticlePubMedGoogle Scholar
- Ferreira F, Bota D, Bross A, Mélot C, Vincent J: Serial Evaluation of the SOFA Score to Predict Outcome in Critically Ill Patients. JAMA. 2001, 286: 1754-1758. 10.1001/jama.286.14.1754.View ArticlePubMedGoogle Scholar
- Timsit J, Fosse J, Troche G, Lassence AD, Alberti C, Garrouste-Orgeas M, Bornstain C, Adrie C, Cheval C, Chevret S: Calibration and discrimination by daily Logistic Organ Dysfunction scoring comparatively with daily Sequential Organ Failure Assessment scoring for predicting hospital mortality in critically ill patients. Critical Care Medicine. 2002, 30: 2003-2013. 10.1097/00003246-200209000-00009.View ArticlePubMedGoogle Scholar
- Data Standards:Critical Care Minimum Dataset – summary of changes. [http://www.dh.gov.uk/en/Healthcare/Emergencycare/ModernisingEmergencyCare/DH_4126508]
- Armstrong R, Bullen C, Cohen S, Singer M, Webb A: Critical Care Algorithms. Clinical Intensive care. 1992, 3:Google Scholar
- Myles P, Cui J: Using the Bland-Altman method to measure agreement with repeated measures. British Journal of Anaesthesia. 2007, 309-311. 10.1093/bja/aem214.Google Scholar