- Research note
Short form version of the Quality of Trauma Care Patient-Reported Experience Measure (SF QTAC-PREM)
BMC Research Notesvolume 10, Article number: 693 (2017)
To enable the valid and reliable measurement of patient experiences we previously published a multicenter multi-center validation of the Quality of Trauma Care Patient-Reported Experience Measure (QTAC-PREM). The purpose of this study was to derive a simplified, short form version of the QTAC-PREM to further enhance the feasibility of measuring patient experiences in injury care. To identify candidate items for the short form we reviewed the results of the original multi-center long form validation cohort study, which included 400 injury care patients and their family members recruited from three trauma centers. We only included the best performing items on the revised short form.
The acute care component of the measure was shortened by 30% and the post-acute care component was shortened by 42%. We identified two subscales on the acute measure (information and communication; clinical and ancillary care) and one subscale on the post-acute measure (post-discharge information and communication). The measurement properties of the short form measure were similar to that of the validated long form. This short form assessment of patient injury care experiences offers a useful, practical, and easy tool for trauma centers to implement for service evaluation, quality improvement, and injury care research.
Governments, health care regulators, funders, and researchers have made clear that the processes and outcomes of care should align with patients’ needs, preferences, and values . Therefore, patient experience is a central component of the quality of care and assessing it is necessary to evaluate and improve care.
The value of measuring patient experiences has been demonstrated. For example, patient experiences have been used to identify gaps in care and guide quality improvement in primary care [2,3,4] and medical, surgical, and obstetrical inpatient care [5, 6]. To date, there have been few studies of patient experiences in injury care; however, there is evidence to suggest that injury care experiences could be utilized to inform health service delivery. For example, researchers in the Victorian State Trauma System conducted interviews with patients and identified specific, actionable targets for improvement based on patients’ perceived gaps in care . This study is valuable for informing local service design however, validated measurement tools are needed to enable trauma programs to efficiently and routinely incorporate patient experiences into quality measurement and improvement.
To enable the valid and reliable measurement of patient experiences we previously developed  and validated  the Quality of Trauma Care Patient-Reported Experience Measure (QTAC-PREM). This was the first measure of injury care experiences to capture both the acute and post-acute phases of care. We found that the measure was feasible to use and had evidence of validity and reliability. However, the tool examines 63 items across five domains and can take considerable time to complete. This may act as a barrier to utilization in routine practice [11, 12].
The purpose of this analysis was therefore to derive a simplified, short form version of the QTAC-PREM to further enhance the feasibility of measuring patient experiences in injury care. In this article we report the derivation and key measurement properties of the new SF QTAC-PREM.
Description of the long form QTAC-PREM and validation study
The long form QTAC-PREM is a patient experience measure consisting of two parts administered separately: a 35-item acute care component to be self-completed in-hospital by patients (or patient proxy) and a 28-item post-acute care component administered via telephone interview 2–3 months after hospital discharge. The measure includes both close-ended and open-ended items. The multi-step development of the original scale included a literature review, focus groups with patients and providers, prospective single-center pilot-testing with 134 participants , revision using cognitive testing , and a prospective multi-center validation with 400 participants .
Data sources and content flagged for deletion
We used the quantitative and qualitative data from our multi-center validation study to inform our selection of items for the short form .
We flagged for deletion items: (1) that did not form part of existing subscales (2) to which 20% or more of patients selected “not-applicable” or “not able to answer” response options, (3) that did not correlate highly with the overall rating of the quality of care, (4) were shown to be redundant by measures of collinearity, (5) that were conceptually redundant, (6) with the lowest test–retest reliability coefficients, and (7) with limited response variation.
We also used the qualitative responses that participants provided to open-ended questions and documented participants’ comments obtained during administration of the post-acute measure via telephone interview.
Dataset and statistical analysis
To assess the operating characteristics of the new combination of items appearing on the short form we repeated the analyses using data from the original multi-center prospective cohort study . The dataset included acute care measure responses from 400 participants recruited between June 2012 and November 2013 from three trauma centers in Canada: two Level 1 trauma centers (University Hospital, Edmonton, and Foothills Medical Center, Calgary) and one Level 3 trauma center (Chinook Regional Hospital, Lethbridge). Follow-up post-acute care interviews were completed on 207 of the 400 participants.
We conducted factor analysis to identify subscales; assessed the internal consistency of the subscales using Cronbach’s alphas; and assessed construct validity by calculating univariate (Spearman correlations) and multivariate (ordinal regression) associations between the subscales and global rating item to determine if the subscales were predictors of overall ratings of the quality of care.
We used only complete cases for all analyses (i.e. cases with missing data or ‘not applicable’ or ‘not able to answer’ response selections were excluded).
We have re-reported the test–retest reliability coefficients for the retained items and spearman correlations between individual items and the global rating item obtained during the long form validation. These properties are expected to remain stable between the long form and short form versions of the measure.
To derive the short form the following changes were made: 19 items were eliminated; 33 items were retained with no revision or minor revision to wording or response options; 2 items assessing general health status were replaced by 4 validated items assessing physical and mental health status separately; and 2 items assessing the provision and adequacy of discharge information were merged to create a single item. Detailed explanations of the item selections and revisions for both short form components can be found in Additional files 5, 6.
Identification of subscales
The acute care factor analysis included 154 complete cases (Table 1). We identified a two-factor solution accounting for 74% of observed variance. Factor 1, information and communication, included items on the scope, clarity, consistency, and availability of acute care information and communication. Factor 2, clinical and ancillary care, included items assessing the clinical processes of care (e.g., pain well-controlled) and ancillary components including being treated unfairly. Spearman’s correlations between the subscales were moderate (0.48) suggesting that the subscale scores should not be combined to form an overall acute care score.
The post-acute care factor analysis included 117 complete cases (Table 2). A one-factor solution accounting for 78% of observed variance was identified. The factor, post-discharge information and communication, included items on the provision, adequacy, scope, and availability of follow-up information and communication.
Subscale Cronbach’s Alpha values were 0.55 and 0.87 for the acute measure subscales (Table 1) and 0.64 for the post-acute care subscale (Table 2). Corrected item—subscale correlations ranged from 0.43 to 0.85 for the acute care measure (Table 1) and from 0.18 to 0.64 for the post-acute measure (Table 2).
All subscales and items on the acute care measure had significant univariate correlations (p < 0.01) with the global rating item (Table 1). In multivariate analysis (n = 154), information and communication (p < 0.001) and clinical and ancillary care (p < 0.001) were independently associated with the global rating item (Additional file 7).
Univariate analysis of the post-acute measure showed that the subscale and eight of nine items were significantly associated with the global rating item (Table 2). In multivariate analysis (n = 117), the subscale (p < 0.001) and one of the stand-alone items (item 10, family physician informed, p = 0.01) were independently associated with the global rating item, while the item on pain management (Item 4, p = 0.09) and perceived safety of care (Item 11, p = 0.57) were not (Additional file 7).
Revisions to simplify analysis and reporting
The response options “not applicable” and “not able to answer” introduce challenges in the analysis and interpretation of results . We identified these challenges in our validation study and they have been noted in other survey development efforts including the Consumer Assessment of Health Plans . Therefore, we eliminated these options from the short form patient versions of the tools. A version of these response options was retained for the proxy-respondent tools, but we replaced “not applicable” and “not able to answer” with “I don’t know” as this option aligned more closely with proxy participants’ language during telephone interviews in the validation study.
We changed the phrase “healthcare providers” to “healthcare practitioners”. In some health systems, such as the United States, “healthcare providers” may be interpreted to include payers that finance or reimburse the cost of health services. However, our intention is to ask about practitioners that deliver care.
These analyses were conducted to derive short form versions of the Quality of Trauma Care Patient-Reported Experience Measure (QTAC-PREM). We shortened the measure by 23 items. The operating characteristics of the short form QTAC-PREM identified in this study are comparable with other validated and widely used measures of patient and family experiences of care [14,15,16].
We were able to identify subscales on the short form that will allow for simplified reporting and interpretation. The acute care short form consists of two subscales (clinical and ancillary care, information and communication), while the post-acute care measure consists of one (post-discharge information and communication).
The subscales on both short form measures showed adequate internal consistency, indicating that summary subscale scores would accurately represent item level data [17, 18]. We also found evidence of construct validity, as all the subscales were significant, independent predictors of patients’ overall ratings of care.
The measurement properties of the short form measure are similar to those of the validated long form. The only notable differences include slightly lower internal consistency of the short form acute care subscale information and communication compared to the long form information-related subscales (0.55 vs. 0.67 and 0.76). This may be the result of merging items from two scales on the long form that assessed narrower constructs to form a single scale addressing a more generic construct; internal consistency is lower when the construct measured is more generic . The scale also has few items, which is known to affect internal consistency. This value suggests that a summary score for the scale is not a perfect representation of the underlying item scores and is a trade-off to gain the benefits of a shorter, more efficient instrument.
There are three benefits of using the short form QTAC-PREM as opposed to the long form. First, response burden will be reduced. Second, the short form may be a more efficient way to obtain informative and comparative data on quality of care. We slightly revised item wording and response options such that the content of the short form should be applicable to all injury patients that are hospitalized and discharged alive. As a result, this should increase the amount of useable and comparable data from each administered measure. Third, the results can be summarized and reported more efficiently. A greater proportion of items fit within subscales compared to the long form and therefore, the results can be summarized with fewer metrics.
The QTAC-PREM and SF QTAC-PREM provide two complementary instruments for assessing patient experiences. The QTAC-PREM may be more relevant for research or detailed audits given its larger number of items, while the short-form may be suited for more frequent applications, such as routine audits. We believe both instruments are potentially important tools for measuring care and guiding quality improvement initiatives.
Assessing patient experience is vital for designing and delivering high-quality injury care. To increase the feasibility of measuring patient experiences we derived a short form version of the Quality of Trauma Care Patient-Reported Experience Measure (SF QTAC-PREM). The short form tool has evidence of validity and reliability. The SF QTAC-PREM offers a useful, practical, and easy tool for trauma centers to implement for service evaluation, quality improvement, and injury care research.
This study has some limitations. First, the data used in this study were obtained using the long form measure and, therefore, the characteristics of the items may vary slightly when implemented as a short form. Although this method has been utilized to derive and validate widely-used and highly cited scales, such as the short form oral health impact profile  and the stroke impact scale , future research should verify the measurement properties by administering the short form version to another population. Second, we made minor revisions to the wording and response options of some of the items retained for the short form and this may affect their measurement properties. Therefore, verifying the operating characteristics may be of value. However, the revisions have simplified the tool for end-users and have been informed by evidence. Therefore, it is likely that the properties of the items that may require re-validation will be acceptable.
- SF QTAC-PREM:
short form Quality of Trauma Care Patient Reported Experience Measure
Quality of Trauma Care Patient Reported Experience Measure
Committee on Quality of Health Care in America Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academies Press; 2001.
Crofton C, Lubalin J, Darby C. Consumer assessment of health plans study (CAHPS). Foreword Med Care. 1999;37:MS1–9.
Cleary P, Edgman-Levitan S. Health care quality: incorporating consumer perspectives. J Am Med Assoc. 1997;278:1608–12.
Davies E, Shaller D, Edgman-Levitan S, Safran DG, Oftedahl G, Sakowski J, et al. Evaluating the use of a modified CAHPS survey to support improvements in patient-centred care: lessons from a quality improvement collaborative. Health Expect. 2008;11:160–76.
Giordano LA, Elliott MN, Goldstein E, Lehrman WG, Spencer PA. Development, implementation, and public reporting of the HCAHPS survey. Med Care Res Rev. 2009;67:27–37.
Jha A, Orav E, Zheng J, Epstein A. Patients’ perception of hospital care in the United States. N Engl J Med. 2008;18:1921–31. https://doi.org/10.1056/NEJMsa0804116.
Gabbe BJ, Sleney JS, Gosling CM, Wilson K, Hart MJ, Sutherland AM, et al. Patient perspectives of care in a regionalised trauma system: lessons from the Victorian State Trauma System. Med J Aust. 2013;198:149–52.
Bobrovitz N, Santana MJ, Ball CG, Kortbeek J, Stelfox HT. The development and testing of a survey to measure patient and family experiences with injury care. J Trauma Acute Care Surg. 2012;73:1332–9.
Bobrovitz N, Santana MJ, Kline T, Kortbeek J, Stelfox HT. The use of cognitive interviews to revise the Quality of Trauma Care Patient-Reported Experience Measure (QTAC-PREM). Qual Life Res. 2015;24:1911–9. https://doi.org/10.1007/s11136-015-0919-5.
Bobrovitz N, Santana MJ, Kline T, Kortbeek J, Widder S, Martin K, et al. Multicenter validation of the quality of trauma care patient-reported experience measure (QTAC-PREM). J Trauma Acute Care Surg. 2016;80:111–8.
Jenkinson C, Gibbons E, Fitzpatrick R. A structured review of patient-reported outcome measures (PROMs) for stroke. Oxford: Department of Health; 2009.
Murray J, Forster A, Young J. Response and completion rates for postal outcomes booklets in stroke rehabilitation. Int J Ther Rehabil. 2007;14:440–5.
McGee J, Kanouse DE, Sofaer S, Hargraves JL, Hoy E, Kleimann S. Making survey results easy to report to consumers: how reporting needs guided survey design in CAHPS. Consumer assessment health plans study. Med Care. 1999;37:MS32–42.
Keller S, O’Malley AJ, Hays RD, Matthew RA, Zaslavsky AM, Hepner KA, et al. Methods used to streamline the CAHPS hospital survey. Health Serv Res. 2005;40:2057–77.
Heyland DK, Tranmer JE. Measuring family satisfaction with care in the intensive care unit: the development of a questionnaire and preliminary results. J Crit Care. 2001;16:142–9.
Wall RJ, Engelberg RA, Downey L, Heyland DK, Curtis JR. Refinement, scoring, and validation of the Family Satisfaction in the Intensive Care Unit (FS-ICU) survey. Crit Care Med. 2007;35:271–9.
Streiner D, Norman G. Health measurement scales: a practical guide to their development and use. Oxford: Oxford University Press; 2008.
Helmstadter G. Principles of psychological measurement. New York: Appleton-Century-Crofts; 1964.
Gary D. Derivation and validation of a short-form oral health impact profile. Commun Dent Oral Epidemiol. 1997;25:284–91.
Jenkinson C, Fitzpatrick R, Crocker H, Peters M. The stroke impact scale. Stroke. 2013;44:2532–6.
NB, MJS, SW, KM and HTS conceived and designed this study. NB, TK, JK, SW, KM, and HTS contributed to data acquisition, analysis, and interpretation. NB, TK, MJS, and HTS performed statistical analysis. NB and HTS drafted the manuscript. JB, MJS, TK, JK, SW, KM, and HTS critically revised the manuscript for important intellectual content. NB and HTS obtained funding. JK and HTS contributed administrative, technical, or material support. HTS provided supervision. All authors read and approved the final manuscript.
We thank Heather Allen, Theresa Pasquotti, and Yvonne Tul for patient recruitment at the subsites. We also thank Christine Vis, Sue-Ann Nodder, Rachel Burton, Natasha Schindler, and the physicians, nursing, and physiotherapy staff on Unit 44 at Foothills Hospital for facilitating recruitment and supporting study activities.
The authors declare that they have no competing interests.
Availability of data and materials
All data analysed during this study are included in this published article (and its Additional files). The raw data generated during this study includes identifying/confidential patient data and cannot be shared.
Consent to publish
Ethics approval and consent to participate
Ethics approval was not required for this study as it is a re-analysis of previously collected data. Ethics approval for the original study was granted by the relevant health research ethics boards at each participating site (ID: 23535; Pro00041277). All participants provided written informed consent before their inclusion in the study.
The project was supported by a Partnerships in Health System Improvement Grant (PHE-238551) from the Canadian Institutes of Health Research and Alberta Innovates. Nik Bobrovitz was supported by a Health Quality Council of Alberta Studentship and an Alberta Innovates Health Solutions Studentship. Nik Bobrovitz is supported by a Clarendon Scholarship, a Goodger and Schorstein Medical Sciences Research Scholarship, and by the National Institute for Health Research School for Primary Care Research (NIHR SPCR). The views expressed are those of the authors and not necessarily those of the NIHR, the NHS or the Department of Health. Dr. Stelfox was supported by a New Investigator Award from the Canadian Institutes of Health Research and a Population Health Investigator Award from Alberta Innovates. Funding sources had no role in the design of this study and had no role in the conduct or reporting of this study. None of the authors had financial or professional conflicts of interest that influenced the conduct or reporting of this study. Mr. Bobrovitz and Dr. Stelfox had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.