Skip to main content

Preliminary development of a questionnaire measuring patient views of participation in clinical trials

Abstract

Objective

This study aimed to develop a questionnaire for measuring patient perceptions of participating in clinical trials. Development was based on earlier research on patient views of involvement in medical care and a literature review. Patients were recruited from an ongoing clinical trial focused on cardiovascular illness and from an outpatient psychiatry department. Factor analysis was conducted on a pilot version of the questionnaire in 2016 and on a revised version in 2017.

Results

A total of 53 patients were recruited for the pilot study and 55 were recruited for the main study, substantially below the goal of 100 participants. Factor analysis revealed six factors measuring aspects of patients’ perceptions of participating in clinical trials, including motivation, risks and benefits, the nature of the trial itself, and practical considerations, such as cost and convenience. Inter-scale correlations ranged between 0.06 and 0.64, indicating acceptable scale independence. Reliability scores (Cronbach’s alphas) ranged from 0.62 to 0.85. Factor analysis results were somewhat unstable, with shared variance for several items across scales. This is likely due to the small sample sizes. In larger, more diverse patient samples, this questionnaire can be useful for measuring and incorporating patients’ views into the design and execution of clinical trials.

Introduction

Randomized controlled clinical trials continue to be considered the gold standard for establishing efficacy of medical treatment. Yet recruitment of patients for participation in trials is a continuous challenge, causing delays in both research studies [1] and scientific discovery [2]. Factors that have been associated with patient participation include communication with providers and family members [2, 3], preference for alternative treatments, convenience [4], medical history, and insurance [4, 5]. A meta-analysis and systematic review found that barriers to participation in clinical trials were related to the study protocol, patient factors, or physician factors [6]. These studies concerned oncology patients, and it is not known whether the same factors apply for patients with other medical conditions. In general, a better understanding of patients’ views of participating in clinical trials would provide a critical missing piece in the design, conduct and successful execution of clinical trials.

The current study was part of a larger project examining the use of Bayesian statistical methods in non-inferiority trials [7]. The aim was to develop a questionnaire for measuring patient perceptions and expectations of participating in clinical trials, with the intention to understand how these perceptions could be converted into probability distributions and integrated into the design of clinical trials. This brief report presents preliminary factor analysis results from the development of the questionnaire.

Main text

Methods

Questionnaire development

Development of the questionnaire was based on earlier research on patient perceptions of involvement in medical care [8] and a literature review. Much of the existing research on patient views of participating in clinical trials has focused on cancer patients [3, 6, 9,10,11,12,13], while a few studies examined cardiology [14] and heart attack patients [15] or the general public [9, 14]. While most studies utilized questionnaires [3, 9, 11,12,13, 15,16,17], only three reported psychometric properties of their instruments [9, 11, 17]. All of the referenced studies were at least 10 years old.

We developed a 44-item pilot questionnaire. Thirteen items covered demographic/personal information. Six items adapted from a previously-validated questionnaire on cardiology patients’ views of involvement in hospital care [8] measured patients’ views of what involvement in clinical trials meant to them. The remaining 24 items were derived from the literature review. Two studies on patient motivation to participate in clinical trials [3, 13] included questions based on questionnaire items originally published by Daugherty et al. [3] that the research team deemed potentially relevant to our study. The 24 items, identified by two members of the research team, were reviewed for content validity by a team of researchers and healthcare professionals as to how well they represented the constructs they were intended to measure. The questionnaire items were derived from studies using focus groups [8] and interviews [3] with patients, which supported their relevance to the patient experience.

All 24 items were scored on a four-point scale from either disagree completely [1] to agree completely [4], or not at all important [1] to very important [4]. One open-ended item enabled patients to share any additional thoughts about participation in clinical trials and 4 additional questions asked for perceptions about the survey itself (e.g., length, clarity of questions). Based on psychometric analysis of the 2016 pilot questionnaire, described below, a final, 41-item version of the questionnaire was developed and administered in 2017 (Additional file 1).

Setting and study participants

The questionnaire was developed for use with any patient group, regardless of diagnosis. The goal for this study was therefore to recruit two different groups of patients. Based on feasibility and availability, we focused on those seeking care for either cardiovascular or mental health conditions. Participants with heart failure were recruited in Detroit, Michigan from an ongoing clinical trial, the Acute Heart Failure Registry of Emergency Department Patients study, part of the Emergency Medicine Research and Outcomes Consortium (EMROC). Patients with mental health conditions were recruited from an outpatient university psychiatry department in Detroit.

Data collection

Patients for the pilot study were recruited in an urban hospital emergency department (ED), as well as in the Department of Psychiatry of a participating university. ED patients were recruited in conjunction with their agreement to participate in the EMROC study, while mental health patients were recruited when coming to their scheduled appointments at an outpatient university psychiatry department. Patients for the main questionnaire study were recruited only in the hospital emergency department. Data collection for the pilot study took place between January and February 2016, with a total of 53 patients agreeing to respond. Of these, 24 (45%) were from psychiatry and 29 (55%) were cardiovascular patients. Data collection for the main questionnaire study took place between April and July 2017, with 55 patients recruited. All respondents willing to provide their mailing address received a $15 gift card for the pilot study, and a $25 gift card for the main study. Respondents were assured that their personal information would be kept separate from their questionnaire responses.

Data analysis

Analysis of both the pilot and the main questionnaires utilized exploratory factor analysis to determine whether any scales could be created. Procedures were identical for both questionnaires. In a first step, a correlation matrix was run to study the inter-item correlations of all non-demographic variables. The matrix was comprised of 30 items for the pilot questionnaire and 27 items for the main questionnaire. Bartlett’s test of sphericity and the Kaiser–Meyer–Olkin measure of sampling adequacy (KMO) were used to assess the factorability of the correlation matrix. To justify factor analysis, Bartlett’s test should be significant and KMO values should exceed 0.60 [18]. Next, exploratory factor analysis was used to examine the structure of the relationships between questionnaire items and to determine whether subscales (factors) could be created. Principal component analysis using Varimax rotation and scree plots was used to extract the factors. Correlation analysis was used to examine the independence of the factors as a measure of construct validity, and Cronbach’s alpha was used to measure each factor’s internal reliability. Second-order factor analysis was then used to see whether forcing the factors to a limited number of dimensions improved the overall explained variance. Three criteria were used in creating subscales: [1] scales should have patient relevance based on content validity; [2] item loadings should be ≥ 0.40 [19]; and [3] reliability coefficients should be ≥ 0.70. Due to the small sample sizes, no tests of either questionnaire’s convergent or discriminant validity were conducted.

Results

Pilot questionnaire

Patient respondents (n = 53) were primarily African-American (68%), White (30%), and female (55%), with a mean age of 51.09. The initial exploratory factor analysis resulted in the extraction of 8 factors with Eigen values ≥ 1 that together explained 74.75 of the overall variance. The KMO measure of sampling adequacy was 0.475, which is considered inadequate [18]. However, Bartlett’s test of sphericity was significant (χ2 = 1222, p = 0.000), which justified proceeding with the analysis. All coefficients that loaded below 0.40 were suppressed. Since several of the 8 factors were only comprised of 2 items and there was shared variance between factors, attempts were made to force the items to a 7- and a 6-factor solution. The 6-factor solution was best, explaining 65% of the total variance, with an overall alpha of 0.82 (Table 1). Three items did not factor into any of the subscales and were therefore excluded, reducing the total number of items to 27. The 3 excluded items were the following statements that patients were to agree/disagree with or rate the importance of: “The patient has the main responsibility for his/her future health” (item 19); “Who are in the research team” (item 29); and “That you could withdraw from the clinical trial at any point in time at your will without affecting your future care” (item 41).

Table 1 Loadings of variables on factors (italic print) emerging from rotated component matrix

Correlations between scales were examined using Spearman’s rho (Table 2). Inter-scale correlations ranged between 0.06 and 0.64, indicating generally acceptable scale independence. The highest correlation between risks and cost and convenience (0.64, p < 0.001) indicates some overlap between those scales.

Table 2 Bivariate correlations (Spearman’s rho) between the patient questionnaire subscales (n = 53)

Main questionnaire

Patient respondents (n = 55) were primarily African-American (95%), 51% were male, and the mean age was 58.85. Exploratory factor analysis included all 27 items and resulted in a 5-factor solution explaining 77.8% of the variance. KMO was 0.749 and Bartlett’s test was significant (χ2 = 1780, p = 0.000). Multiple iterations were conducted, including forcing the items to 4, 6, and 7 factors. Each solution had substantial shared variance between items. We ran reliability estimates (Cronbach’s alpha) on the 6 scales that had been identified in the pilot study, even though they were not supported by the factor analysis in this second phase. Reliability estimates were 0.78 or higher for the main questionnaire and ranged from 0.62 to 0.85 for the pilot data (Table 3).

Table 3 Patient questionnaire subscales with mean scores and reliability estimates (Cronbach’s alpha)

Discussion

The aim of this study was to develop a questionnaire for measuring patient perceptions and expectations of participating in clinical trials. Patients with acute heart failure were recruited once they had agreed to enroll in an ongoing clinical trial, a method that has been used in previous cancer therapy trials [3, 13]. Preliminary analyses identified six scales measuring various aspects of patients’ perceptions of participating in clinical trials, encompassing factors concerning motivation, risks and benefits of participation, the nature of the trial itself, and practical considerations, such as cost and convenience. Our findings that patients are motivated to participate by expectations of potential health benefits are in line with previous studies of cancer patients [3, 13]. However, contributing to other patients’ health, also a factor in the current study, was less important to the cancer patients [3, 13], although altruism was identified as a factor in earlier research of myocardial infarction (MI) patients, as was convenience [15].

Limitations

Building on the pilot analysis, our goal was to replicate the factor analysis in a larger sample. However, due to time and resource constraints, the research team was only able to recruit 55 patients from the EMROC study for the second phase of questionnaire analysis. Based on the pilot study data, a sample size of 147 would be required to detect significant differences in the Motivation subscale between male and female patients with a power of 0.80 and a type I error rate of 0.05. With only 55 participants, our main study was therefore clearly underpowered.

Preliminary analyses indicated good internal consistency of the scales, measured with Cronbach’s alpha, but factor analysis results were somewhat unstable, with shared variance for several items across scales. This is likely due to the small sample sizes. Despite much discussion in the scientific literature regarding minimum sample sizes for factor analysis, it is suggested that a greater number of variables per factor and higher levels of communality usually allow for smaller sample sizes [20]. Nevertheless, results of these analyses suggest a need to re-evaluate with additional, larger samples of patients. Larger samples would also enable more robust validity testing, the aim of which would be to establish that the scales generated by the factor analysis measure what they intend to measure. Discriminant validity would be examined by studying the associations between subscales, patient demographic characteristics, and patient diagnosis (psychiatric vs. cardiovascular) in order to test each scale’s ability to capture differences between groups. Finally, data were collected from a large, urban hospital and the demographics of patient respondents may not be representative of the general population. This questionnaire therefore needs to be tested in both larger and more diverse populations. Despite these study limitations, this questionnaire can be useful for measuring and incorporating patients’ views into the design and execution of clinical trials.

Designing clinical trials to be more patient-centered could potentially increase the likelihood of patient recruitment and even of trial effectiveness. However, quantifying the patient perspective and utilizing it to improve trials is in its infancy. These preliminary results reveal several domains of importance to patients when considering participation in clinical trials. While further psychometric testing is needed, this questionnaire represents an important first step in incorporating patient views into the design and recruitment process of clinical trials.

Availability of data and materials

The data sets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

EMROC:

Emergency Medicine Research and Outcomes Consortium

ED:

Emergency Department

KMO:

Kaiser–Meyer–Olkin measure of sampling adequacy

PCORI:

Patient Centered Outcomes Research Institute

References

  1. 1.

    Allison M. Can web 2.0 reboot clinical trials? Nat Biotechnol. 2009;27(10):895–902.

    CAS  Article  Google Scholar 

  2. 2.

    Albrecht TL, Eggly SS, Gleason ME, Harper FW, Foster TS, Peterson AM, et al. Influence of clinical communication on patients’ decision making on participation in clinical trials. J Clin Oncol. 2008;26(16):2666–73.

    Article  Google Scholar 

  3. 3.

    Daugherty C, Ratain MJ, Grochowski E, Stocking C, Kodish E, Mick R, et al. Perceptions of cancer patients and their physicians involved in phase I trials. J Clin Oncol. 1995;13(5):1062–72.

    CAS  Article  Google Scholar 

  4. 4.

    Lara PN Jr, Higdon R, Lim N, Kwan K, Tanaka M, Lau DH, et al. Prospective evaluation of cancer clinical trial accrual patterns: identifying potential barriers to enrollment. J Clin Oncol. 2001;19(6):1728–33.

    Article  Google Scholar 

  5. 5.

    Lewis JH, Kilgore ML, Goldman DP, Trimble EL, Kaplan R, Montello MJ, et al. Participation of patients 65 years of age or older in cancer clinical trials. J Clin Oncol. 2003;21(7):1383–9.

    Article  Google Scholar 

  6. 6.

    Mills EJ, Seely D, Rachlis B, Griffith L, Wu P, Wilson K, et al. Barriers to participation in clinical trials of cancer: a meta-analysis and systematic review of patient-reported factors. Lancet Oncol. 2006;7(2):141–8.

    Article  Google Scholar 

  7. 7.

    Chowdhury S, Tiwari RC, Ghosh S. Approaches for testing noninferiority in two-arm trials for risk ratio and odds ratio. J Biopharm Stat. 2019;12:1–21.

    Google Scholar 

  8. 8.

    Arnetz JE, Hoglund AT, Arnetz BB, Winblad U. Development and evaluation of a questionnaire for measuring patient views of involvement in myocardial infarction care. Eur J Cardiovasc Nurs. 2008;7(3):229–38.

    Article  Google Scholar 

  9. 9.

    Comis RL, Miller JD, Aldige CR, Krebs L, Stoval E. Public attitudes toward participation in cancer clinical trials. J Clin Oncol. 2003;21(5):830–5.

    Article  Google Scholar 

  10. 10.

    Ellis PM. Attitudes towards and participation in randomised clinical trials in oncology: a review of the literature. Ann Oncol. 2000;11(8):939–45.

    CAS  Article  Google Scholar 

  11. 11.

    Ellis PM, Butow PN, Tattersall MH, Dunn SM, Houssami N. Randomized clinical trials in oncology: understanding and attitudes predict willingness to participate. J Clin Oncol. 2001;19(15):3554–61.

    CAS  Article  Google Scholar 

  12. 12.

    Llewellyn-Thomas HA, McGreal MJ, Thiel EC, Fine S, Erlichman C. Patients’ willingness to enter clinical trials: measuring the association with perceived benefit and preference for decision participation. Soc Sci Med. 1991;32(1):35–42.

    CAS  Article  Google Scholar 

  13. 13.

    Nurgat ZA, Craig W, Campbell NC, Bissett JD, Cassidy J, Nicolson MC. Patient motivations surrounding participation in phase I and phase II clinical trials of cancer chemotherapy. Br J Cancer. 2005;92(6):1001–5.

    CAS  Article  Google Scholar 

  14. 14.

    Cassileth BR, Lusk EJ, Miller DS, Hurwitz S. Attitudes toward clinical-trials among patients and the public. J Am Med Assoc. 1982;248(8):968–70.

    CAS  Article  Google Scholar 

  15. 15.

    Mattson ME, Curb JD, McArdle R. Participation in a clinical trial: the patients’ point of view. Control Clin Trials. 1985;6(2):156–67.

    CAS  Article  Google Scholar 

  16. 16.

    Agoritsas T, Deom M, Perneger TV. Study design attributes influenced patients’ willingness to participate in clinical research: a randomized vignette-based study. J Clin Epidemiol. 2011;64(1):107–15.

    Article  Google Scholar 

  17. 17.

    Verheggen FW, Nieman F, Jonkers R. Determinants of patient participation in clinical studies requiring informed consent: why patients enter a clinical trial. Patient Educ Couns. 1998;35(2):111–25.

    CAS  Article  Google Scholar 

  18. 18.

    Tabachnick BG, Fidell LS. Using multivariate statistics. 7th ed. Boston: Pearson; 2019.

    Google Scholar 

  19. 19.

    Rattray J, Jones MC. Essential elements of questionnaire design and development. J Clin Nurs. 2007;16(2):234–43.

    Article  Google Scholar 

  20. 20.

    Mundfrom DJ, Shaw DG, Ke TL. Minimum sample size recommendations for conducting factor analyses. Int J Test. 2005;5(2):159–68.

    Article  Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge the patients who generously offered their time to participate in this study.

Funding

This study was funded by The Patient Centered Outcomes Research Institute (PCORI), Grant ME-1409-21410. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funders. The funding agency had no role in the design of the study; in data collection, analysis, or interpretation; or in writing the manuscript.

Author information

Affiliations

Authors

Contributions

JA drafted the paper. JA, BA, and SG are responsible for the intellectual content of the paper. LG and SM were responsible for data collection. CG assisted with the literature review and SS assisted with data analysis. All authors reviewed earlier drafts. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Judith Arnetz.

Ethics declarations

Ethics approval and consent to participate

The study received ethical approval from Wayne State University, Protocol nr. 1509014335, and was the Institutional Review Board of record, with participation by Michigan State University. The study was also approved by the Clinical Research Office of the Detroit Medical Center. All participants or their family members were given detailed oral and written information about the study. Consent forms describing the study procedures and risks were given to each participant or their family members and written documentation of informed consent was required prior to enrollment.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

Patient_questionnaire.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Arnetz, J., Sudan, S., Goetz, C. et al. Preliminary development of a questionnaire measuring patient views of participation in clinical trials. BMC Res Notes 12, 667 (2019). https://doi.org/10.1186/s13104-019-4724-z

Download citation

Keywords

  • Questionnaire
  • Patients
  • Clinical trials
  • Psychometric evaluation
  • Factor analysis