Skip to content

Advertisement

BMC Research Notes

What do you think about BMC? Take part in

Open Access

Short form version of the Quality of Trauma Care Patient-Reported Experience Measure (SF QTAC-PREM)

BMC Research Notes201710:693

https://doi.org/10.1186/s13104-017-3031-9

Received: 15 August 2017

Accepted: 29 November 2017

Published: 6 December 2017

Abstract

Objective

To enable the valid and reliable measurement of patient experiences we previously published a multicenter multi-center validation of the Quality of Trauma Care Patient-Reported Experience Measure (QTAC-PREM). The purpose of this study was to derive a simplified, short form version of the QTAC-PREM to further enhance the feasibility of measuring patient experiences in injury care. To identify candidate items for the short form we reviewed the results of the original multi-center long form validation cohort study, which included 400 injury care patients and their family members recruited from three trauma centers. We only included the best performing items on the revised short form.

Results

The acute care component of the measure was shortened by 30% and the post-acute care component was shortened by 42%. We identified two subscales on the acute measure (information and communication; clinical and ancillary care) and one subscale on the post-acute measure (post-discharge information and communication). The measurement properties of the short form measure were similar to that of the validated long form. This short form assessment of patient injury care experiences offers a useful, practical, and easy tool for trauma centers to implement for service evaluation, quality improvement, and injury care research.

Keywords

Patient experiencePatient reported measureInjuryTraumaQuality of care

Introduction

Governments, health care regulators, funders, and researchers have made clear that the processes and outcomes of care should align with patients’ needs, preferences, and values [1]. Therefore, patient experience is a central component of the quality of care and assessing it is necessary to evaluate and improve care.

The value of measuring patient experiences has been demonstrated. For example, patient experiences have been used to identify gaps in care and guide quality improvement in primary care [24] and medical, surgical, and obstetrical inpatient care [5, 6]. To date, there have been few studies of patient experiences in injury care; however, there is evidence to suggest that injury care experiences could be utilized to inform health service delivery. For example, researchers in the Victorian State Trauma System conducted interviews with patients and identified specific, actionable targets for improvement based on patients’ perceived gaps in care [7]. This study is valuable for informing local service design however, validated measurement tools are needed to enable trauma programs to efficiently and routinely incorporate patient experiences into quality measurement and improvement.

To enable the valid and reliable measurement of patient experiences we previously developed [8] and validated [10] the Quality of Trauma Care Patient-Reported Experience Measure (QTAC-PREM). This was the first measure of injury care experiences to capture both the acute and post-acute phases of care. We found that the measure was feasible to use and had evidence of validity and reliability. However, the tool examines 63 items across five domains and can take considerable time to complete. This may act as a barrier to utilization in routine practice [11, 12].

The purpose of this analysis was therefore to derive a simplified, short form version of the QTAC-PREM to further enhance the feasibility of measuring patient experiences in injury care. In this article we report the derivation and key measurement properties of the new SF QTAC-PREM.

Main text

Methods

Description of the long form QTAC-PREM and validation study

The long form QTAC-PREM is a patient experience measure consisting of two parts administered separately: a 35-item acute care component to be self-completed in-hospital by patients (or patient proxy) and a 28-item post-acute care component administered via telephone interview 2–3 months after hospital discharge. The measure includes both close-ended and open-ended items. The multi-step development of the original scale included a literature review, focus groups with patients and providers, prospective single-center pilot-testing with 134 participants [8], revision using cognitive testing [9], and a prospective multi-center validation with 400 participants [10].

Data sources and content flagged for deletion

We used the quantitative and qualitative data from our multi-center validation study to inform our selection of items for the short form [10].

We flagged for deletion items: (1) that did not form part of existing subscales (2) to which 20% or more of patients selected “not-applicable” or “not able to answer” response options, (3) that did not correlate highly with the overall rating of the quality of care, (4) were shown to be redundant by measures of collinearity, (5) that were conceptually redundant, (6) with the lowest test–retest reliability coefficients, and (7) with limited response variation.

We also used the qualitative responses that participants provided to open-ended questions and documented participants’ comments obtained during administration of the post-acute measure via telephone interview.

Dataset and statistical analysis

To assess the operating characteristics of the new combination of items appearing on the short form we repeated the analyses using data from the original multi-center prospective cohort study [10]. The dataset included acute care measure responses from 400 participants recruited between June 2012 and November 2013 from three trauma centers in Canada: two Level 1 trauma centers (University Hospital, Edmonton, and Foothills Medical Center, Calgary) and one Level 3 trauma center (Chinook Regional Hospital, Lethbridge). Follow-up post-acute care interviews were completed on 207 of the 400 participants.

We conducted factor analysis to identify subscales; assessed the internal consistency of the subscales using Cronbach’s alphas; and assessed construct validity by calculating univariate (Spearman correlations) and multivariate (ordinal regression) associations between the subscales and global rating item to determine if the subscales were predictors of overall ratings of the quality of care.

We used only complete cases for all analyses (i.e. cases with missing data or ‘not applicable’ or ‘not able to answer’ response selections were excluded).

We have re-reported the test–retest reliability coefficients for the retained items and spearman correlations between individual items and the global rating item obtained during the long form validation. These properties are expected to remain stable between the long form and short form versions of the measure.

Results

The acute care measure was shortened to 24 items and the post-acute measure was shortened to 16 items. The final short form versions of the measure are available in the Additional files 1, 2, 3, 4.

To derive the short form the following changes were made: 19 items were eliminated; 33 items were retained with no revision or minor revision to wording or response options; 2 items assessing general health status were replaced by 4 validated items assessing physical and mental health status separately; and 2 items assessing the provision and adequacy of discharge information were merged to create a single item. Detailed explanations of the item selections and revisions for both short form components can be found in Additional files 5, 6.

Identification of subscales

The acute care factor analysis included 154 complete cases (Table 1). We identified a two-factor solution accounting for 74% of observed variance. Factor 1, information and communication, included items on the scope, clarity, consistency, and availability of acute care information and communication. Factor 2, clinical and ancillary care, included items assessing the clinical processes of care (e.g., pain well-controlled) and ancillary components including being treated unfairly. Spearman’s correlations between the subscales were moderate (0.48) suggesting that the subscale scores should not be combined to form an overall acute care score.
Table 1

Measurement properties of the short form QTAC-PREM acute care measure

Subscales

Item numbers and descriptionsa

Sub-scale factor loadings (n = 154)b

Subscale Cronbach’s alpha and corrected item-subscale correlations, (n = 400)c

Item/subscale to global rating item correlationsd, (n = 400)c

Reliability coefficient (95% CI), (n = 78)e

Information and communication

0.55

0.50g

 

1. All injuries explained (10)

0.83

0.50

0.34g

0.51 (0.13–0.88)h

2. Explained self-care (12)

0.75

0.43

0.29g

0.64 (0.41–0.86)h

3. Explained recovery timeline (13)

0.96

0.43

0.19g

0.69 (0.45–0.94)h

4. Consistent information (14)

0.35

0.50

0.47g

0.78 (0.68–0.85)

Clinical and ancillary care

0.85

0.67g

 

5. Pain well-controlled (18)

0.68

0.84

0.50g

0.72 (0.59–0.81)

6. Helped with agitation (21)

0.75

0.81

0.57g

0.71 (0.55–0.82)

7. Handled carefully (22)

0.83

0.83

0.44g

0.64 (0.48–0.76)

8. Helped with hygiene (23)

0.60

0.84

0.40g

0.68 (0.52–0.79)

9. Providers explained their roles (24)

0.61

0.83

0.44g

0.78 (0.67–0.85)

10. Addressed concerns (25)

0.70

0.82

0.55g

0.68 (0.52–0.80)

11. Dignity considered (27)

0.77

0.82

0.44g

0.57 (0.39–0.70)

12. Offered to discuss emotional needs (26)f

13. Perceived safety of care (28)

0.76

0.85

0.34g

0.88 (0.82–0.92)

14. Treated unfairly (30)

0.80

0.85

0.28g

0.44 (0.24–0.61)

Stand-alone items

   

15. Global rating (34)

 

0.85 (0.77–0.90)

aCorresponding long form item number in (brackets)

bn = 154 complete cases available out of total sample of 400 completed surveys

cSample size varied depending on the number of complete cases in each sub-scale: clinical and ancillary care (n = 225) and information and communication (n = 273)

dSpearman’s correlations, including Bonferroni correction to account for multiple testing

eIntraclass correlation coefficient unless otherwise indicated

fThis item underwent major revision for inclusion on the short form and, therefore, could not be assessed using data from the long form validation study. However, we have included it within the clinical and ancillary care subscale as it conceptually fits within this construct

gSignificant at the p < 0.01 level

hCohen’s Kappa coefficient

The post-acute care factor analysis included 117 complete cases (Table 2). A one-factor solution accounting for 78% of observed variance was identified. The factor, post-discharge information and communication, included items on the provision, adequacy, scope, and availability of follow-up information and communication.
Table 2

Measurement properties of the short form QTAC-PREM post-acute care measure

Subscales

Item numbers and descriptionsa

Sub-scale factor loadings (n = 117)b

Subscale Cronbach’s alpha and corrected item-subscale correlations, (n = 117)b

Subscale to global rating item correlationsd, (n = 207)c

Reliability coefficient (95% CI), (n = 76)e

Post-discharge information and communication

 

0.64

0.61g

 

1. Received adequate written instructions (2/3)

0.40

0.27

0.25g

0.78 (0.67–0.86)

7. Explained next steps in recovery (9)

0.82

0.58

0.52g

0.71 (0.52–0.91)h

8. Described recovery timeline (10)

0.55

0.34

0.29g

0.55 (0.35–0.76)h

9. Understandable explanations (12)

0.74

0.57

0.43g

0.69 (0.55–0.80)

12. Post-discharge recovery guidance (17)

0.68

0.55

0.79g

0.86 (0.79–0.91)

Stand-alone items

2. Pain management (4)f

0.25g

0.90 (0.77–1.00)h

5. Difficulty scheduling appointments (8a–c)i

10. Family physician informed (16)f

0.40g

0.87 (0.78–0.92)

11. Perceived safety of care (15)j

0.10

0k

13. Global rating item (18)

0.90 (0.83–0.94)

aCorresponding long form item number in (brackets)

bn = 117 complete cases available out of total sample of 207 completed surveys

cThe available sample size varied from n = 118 to n = 207 depending on the number of complete cases for the subscale/individual items

dSpearman’s correlations

eIntraclass correlation coefficient unless otherwise indicated

fItem did not load onto a sub-scale

gSignificant at the p < 0.01 level

hCohen’s Kappa coefficient

iThis item underwent major revision for inclusion on the short form and therefore could not be assessed using data from the long form validation study. However, we recommend it be reported as a stand-alone item as it does not conceptually match the construct of the subscale

jItem excluded from factor analysis due to limited variation among complete cases

kNo reliability estimate because of limited response variance among the sample of test–retest respondents

Internal consistency

Subscale Cronbach’s Alpha values were 0.55 and 0.87 for the acute measure subscales (Table 1) and 0.64 for the post-acute care subscale (Table 2). Corrected item—subscale correlations ranged from 0.43 to 0.85 for the acute care measure (Table 1) and from 0.18 to 0.64 for the post-acute measure (Table 2).

Construct validity

All subscales and items on the acute care measure had significant univariate correlations (p < 0.01) with the global rating item (Table 1). In multivariate analysis (n = 154), information and communication (p < 0.001) and clinical and ancillary care (p < 0.001) were independently associated with the global rating item (Additional file 7).

Univariate analysis of the post-acute measure showed that the subscale and eight of nine items were significantly associated with the global rating item (Table 2). In multivariate analysis (n = 117), the subscale (p < 0.001) and one of the stand-alone items (item 10, family physician informed, p = 0.01) were independently associated with the global rating item, while the item on pain management (Item 4, p = 0.09) and perceived safety of care (Item 11, p = 0.57) were not (Additional file 7).

Revisions to simplify analysis and reporting

The response options “not applicable” and “not able to answer” introduce challenges in the analysis and interpretation of results [1]. We identified these challenges in our validation study and they have been noted in other survey development efforts including the Consumer Assessment of Health Plans [13]. Therefore, we eliminated these options from the short form patient versions of the tools. A version of these response options was retained for the proxy-respondent tools, but we replaced “not applicable” and “not able to answer” with “I don’t know” as this option aligned more closely with proxy participants’ language during telephone interviews in the validation study.

Other revisions

We changed the phrase “healthcare providers” to “healthcare practitioners”. In some health systems, such as the United States, “healthcare providers” may be interpreted to include payers that finance or reimburse the cost of health services. However, our intention is to ask about practitioners that deliver care.

Discussion

These analyses were conducted to derive short form versions of the Quality of Trauma Care Patient-Reported Experience Measure (QTAC-PREM). We shortened the measure by 23 items. The operating characteristics of the short form QTAC-PREM identified in this study are comparable with other validated and widely used measures of patient and family experiences of care [1416].

We were able to identify subscales on the short form that will allow for simplified reporting and interpretation. The acute care short form consists of two subscales (clinical and ancillary care, information and communication), while the post-acute care measure consists of one (post-discharge information and communication).

The subscales on both short form measures showed adequate internal consistency, indicating that summary subscale scores would accurately represent item level data [17, 18]. We also found evidence of construct validity, as all the subscales were significant, independent predictors of patients’ overall ratings of care.

The measurement properties of the short form measure are similar to those of the validated long form. The only notable differences include slightly lower internal consistency of the short form acute care subscale information and communication compared to the long form information-related subscales (0.55 vs. 0.67 and 0.76). This may be the result of merging items from two scales on the long form that assessed narrower constructs to form a single scale addressing a more generic construct; internal consistency is lower when the construct measured is more generic [17]. The scale also has few items, which is known to affect internal consistency. This value suggests that a summary score for the scale is not a perfect representation of the underlying item scores and is a trade-off to gain the benefits of a shorter, more efficient instrument.

There are three benefits of using the short form QTAC-PREM as opposed to the long form. First, response burden will be reduced. Second, the short form may be a more efficient way to obtain informative and comparative data on quality of care. We slightly revised item wording and response options such that the content of the short form should be applicable to all injury patients that are hospitalized and discharged alive. As a result, this should increase the amount of useable and comparable data from each administered measure. Third, the results can be summarized and reported more efficiently. A greater proportion of items fit within subscales compared to the long form and therefore, the results can be summarized with fewer metrics.

The QTAC-PREM and SF QTAC-PREM provide two complementary instruments for assessing patient experiences. The QTAC-PREM may be more relevant for research or detailed audits given its larger number of items, while the short-form may be suited for more frequent applications, such as routine audits. We believe both instruments are potentially important tools for measuring care and guiding quality improvement initiatives.

Conclusion

Assessing patient experience is vital for designing and delivering high-quality injury care. To increase the feasibility of measuring patient experiences we derived a short form version of the Quality of Trauma Care Patient-Reported Experience Measure (SF QTAC-PREM). The short form tool has evidence of validity and reliability. The SF QTAC-PREM offers a useful, practical, and easy tool for trauma centers to implement for service evaluation, quality improvement, and injury care research.

Limitations

This study has some limitations. First, the data used in this study were obtained using the long form measure and, therefore, the characteristics of the items may vary slightly when implemented as a short form. Although this method has been utilized to derive and validate widely-used and highly cited scales, such as the short form oral health impact profile [19] and the stroke impact scale [20], future research should verify the measurement properties by administering the short form version to another population. Second, we made minor revisions to the wording and response options of some of the items retained for the short form and this may affect their measurement properties. Therefore, verifying the operating characteristics may be of value. However, the revisions have simplified the tool for end-users and have been informed by evidence. Therefore, it is likely that the properties of the items that may require re-validation will be acceptable.

Abbreviations

SF QTAC-PREM: 

short form Quality of Trauma Care Patient Reported Experience Measure

QTAC-PREM: 

Quality of Trauma Care Patient Reported Experience Measure

Declarations

Authors’ contributions

NB, MJS, SW, KM and HTS conceived and designed this study. NB, TK, JK, SW, KM, and HTS contributed to data acquisition, analysis, and interpretation. NB, TK, MJS, and HTS performed statistical analysis. NB and HTS drafted the manuscript. JB, MJS, TK, JK, SW, KM, and HTS critically revised the manuscript for important intellectual content. NB and HTS obtained funding. JK and HTS contributed administrative, technical, or material support. HTS provided supervision. All authors read and approved the final manuscript.

Acknowledgements

We thank Heather Allen, Theresa Pasquotti, and Yvonne Tul for patient recruitment at the subsites. We also thank Christine Vis, Sue-Ann Nodder, Rachel Burton, Natasha Schindler, and the physicians, nursing, and physiotherapy staff on Unit 44 at Foothills Hospital for facilitating recruitment and supporting study activities.

Competing interests

The authors declare that they have no competing interests.

Availability of data and materials

All data analysed during this study are included in this published article (and its Additional files). The raw data generated during this study includes identifying/confidential patient data and cannot be shared.

Consent to publish

Not applicable.

Ethics approval and consent to participate

Ethics approval was not required for this study as it is a re-analysis of previously collected data. Ethics approval for the original study was granted by the relevant health research ethics boards at each participating site (ID: 23535; Pro00041277). All participants provided written informed consent before their inclusion in the study.

Funding

The project was supported by a Partnerships in Health System Improvement Grant (PHE-238551) from the Canadian Institutes of Health Research and Alberta Innovates. Nik Bobrovitz was supported by a Health Quality Council of Alberta Studentship and an Alberta Innovates Health Solutions Studentship. Nik Bobrovitz is supported by a Clarendon Scholarship, a Goodger and Schorstein Medical Sciences Research Scholarship, and by the National Institute for Health Research School for Primary Care Research (NIHR SPCR). The views expressed are those of the authors and not necessarily those of the NIHR, the NHS or the Department of Health. Dr. Stelfox was supported by a New Investigator Award from the Canadian Institutes of Health Research and a Population Health Investigator Award from Alberta Innovates. Funding sources had no role in the design of this study and had no role in the conduct or reporting of this study. None of the authors had financial or professional conflicts of interest that influenced the conduct or reporting of this study. Mr. Bobrovitz and Dr. Stelfox had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Nuffield Department of Primary Care Health Sciences, University of Oxford, Radcliffe Observatory Quarter
(2)
Department of Community Health Sciences, University of Calgary
(3)
Department of Psychology, University of Calgary
(4)
Department of Surgery, University of Calgary
(5)
Department of Surgery, University of Alberta
(6)
Department of Critical Care Medicine, University of Alberta
(7)
Chinook Regional Hospital
(8)
Department of Critical Care Medicine, University of Calgary
(9)
Department of Medicine, University of Calgary

References

  1. Committee on Quality of Health Care in America Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academies Press; 2001.Google Scholar
  2. Crofton C, Lubalin J, Darby C. Consumer assessment of health plans study (CAHPS). Foreword Med Care. 1999;37:MS1–9.View ArticlePubMedGoogle Scholar
  3. Cleary P, Edgman-Levitan S. Health care quality: incorporating consumer perspectives. J Am Med Assoc. 1997;278:1608–12.View ArticleGoogle Scholar
  4. Davies E, Shaller D, Edgman-Levitan S, Safran DG, Oftedahl G, Sakowski J, et al. Evaluating the use of a modified CAHPS survey to support improvements in patient-centred care: lessons from a quality improvement collaborative. Health Expect. 2008;11:160–76.View ArticlePubMedPubMed CentralGoogle Scholar
  5. Giordano LA, Elliott MN, Goldstein E, Lehrman WG, Spencer PA. Development, implementation, and public reporting of the HCAHPS survey. Med Care Res Rev. 2009;67:27–37.View ArticlePubMedGoogle Scholar
  6. Jha A, Orav E, Zheng J, Epstein A. Patients’ perception of hospital care in the United States. N Engl J Med. 2008;18:1921–31. https://doi.org/10.1056/NEJMsa0804116.View ArticleGoogle Scholar
  7. Gabbe BJ, Sleney JS, Gosling CM, Wilson K, Hart MJ, Sutherland AM, et al. Patient perspectives of care in a regionalised trauma system: lessons from the Victorian State Trauma System. Med J Aust. 2013;198:149–52.View ArticlePubMedGoogle Scholar
  8. Bobrovitz N, Santana MJ, Ball CG, Kortbeek J, Stelfox HT. The development and testing of a survey to measure patient and family experiences with injury care. J Trauma Acute Care Surg. 2012;73:1332–9.View ArticlePubMedGoogle Scholar
  9. Bobrovitz N, Santana MJ, Kline T, Kortbeek J, Stelfox HT. The use of cognitive interviews to revise the Quality of Trauma Care Patient-Reported Experience Measure (QTAC-PREM). Qual Life Res. 2015;24:1911–9. https://doi.org/10.1007/s11136-015-0919-5.View ArticlePubMedGoogle Scholar
  10. Bobrovitz N, Santana MJ, Kline T, Kortbeek J, Widder S, Martin K, et al. Multicenter validation of the quality of trauma care patient-reported experience measure (QTAC-PREM). J Trauma Acute Care Surg. 2016;80:111–8.View ArticlePubMedGoogle Scholar
  11. Jenkinson C, Gibbons E, Fitzpatrick R. A structured review of patient-reported outcome measures (PROMs) for stroke. Oxford: Department of Health; 2009.Google Scholar
  12. Murray J, Forster A, Young J. Response and completion rates for postal outcomes booklets in stroke rehabilitation. Int J Ther Rehabil. 2007;14:440–5.View ArticleGoogle Scholar
  13. McGee J, Kanouse DE, Sofaer S, Hargraves JL, Hoy E, Kleimann S. Making survey results easy to report to consumers: how reporting needs guided survey design in CAHPS. Consumer assessment health plans study. Med Care. 1999;37:MS32–42.View ArticlePubMedGoogle Scholar
  14. Keller S, O’Malley AJ, Hays RD, Matthew RA, Zaslavsky AM, Hepner KA, et al. Methods used to streamline the CAHPS hospital survey. Health Serv Res. 2005;40:2057–77.View ArticlePubMedPubMed CentralGoogle Scholar
  15. Heyland DK, Tranmer JE. Measuring family satisfaction with care in the intensive care unit: the development of a questionnaire and preliminary results. J Crit Care. 2001;16:142–9.View ArticlePubMedGoogle Scholar
  16. Wall RJ, Engelberg RA, Downey L, Heyland DK, Curtis JR. Refinement, scoring, and validation of the Family Satisfaction in the Intensive Care Unit (FS-ICU) survey. Crit Care Med. 2007;35:271–9.View ArticlePubMedGoogle Scholar
  17. Streiner D, Norman G. Health measurement scales: a practical guide to their development and use. Oxford: Oxford University Press; 2008.View ArticleGoogle Scholar
  18. Helmstadter G. Principles of psychological measurement. New York: Appleton-Century-Crofts; 1964.Google Scholar
  19. Gary D. Derivation and validation of a short-form oral health impact profile. Commun Dent Oral Epidemiol. 1997;25:284–91.View ArticleGoogle Scholar
  20. Jenkinson C, Fitzpatrick R, Crocker H, Peters M. The stroke impact scale. Stroke. 2013;44:2532–6.View ArticlePubMedGoogle Scholar

Copyright

© The Author(s) 2017

Advertisement