Skip to main content

Competition for recruitment in SARS-CoV-2 Trials in the United States: a longitudinal cohort analysis

Abstract

Objective

Competition among trials for patient enrollment can impede recruitment. We hypothesized that this occurred early in the COVID-19 pandemic, when an unprecedented number of clinical trials were launched. We performed a simple and multivariable regression analysis evaluating the relationship between the proportion of SARS-CoV-2 investigational trial sites within each USA state with unsuccessful patient-participant recruitment and: (i) the proportion of cases required to reach state recruitment goals; (ii) state population based on data from the US Census; and, (iii) number of trial sites per state.

Results

Our study included 151 clinical trials. The proportion of trials with successful recruitment was 72.19% (109 of 151 trials). We did not find a significant relationship between unsuccessful patient-participant recruitment, state recruitment goals, state population or the number of trial sites per state in both our simple and multivariable regression analyses. Our results do not suggest that early in the COVID-19 pandemic, competition for patient-participants impeded successful recruitment in SARS-CoV-2 trials. This may reflect the unique circumstances of the first few months of the pandemic in the United States, in which the number and location of SARS-CoV-2 cases was sufficient to meet trial recruitment requirements, despite the large number of trials launched.

Introduction

In the first year of the pandemic the international research community launched an unprecedented number of clinical trials directed at the treatment and prevention of SARS-CoV-2 [1, 2]. Despite the large volume of documented SARS-CoV-2 cases, we hypothesized that competition for trial participants may have negatively impacted patient-participant recruitment.

Concerns regarding inadequate patient-participant recruitment leading to early trial termination predate the COVID-19 pandemic [3,4,5,6,7]. Recent studies have evaluated the proportion of SARS-CoV-2 trials reaching 75% of goal recruitment [8] and have compared trial recruitment targets and feasible recruitment numbers in England [9]. One study evaluated geographic alignment of SARS-CoV-2 cases and interventional clinical trial sites in the United States, noting decreasing alignment between trials sites and new cases as the pandemic progressed [10].

In what follows we evaluate patient-participant recruitment of SARS-CoV-2 treatment trials registered on ClinicalTrials.gov, with a site in the United States, and assess dependency of unsuccessful trial recruitment and state recruitment goals. We consider whether local competition between trials may have impacted recruitment success.

Main text

Methods

Trial sample

We included interventional SARS-CoV-2 treatment efficacy trials registered on ClinicalTrials.gov with a trial site in the United States (USA) and a start date between 2020–01-01 and 2020–06-30. We excluded trials evaluating interventions aimed at the prevention of SARS-CoV-2. Prevention trials typically enroll healthy volunteers, which was not the population of interest in our study. We included all eligible trials in our cohort; therefore, no sample size calculation was performed. Data were downloaded from the web front-end of ClinicalTrials.gov on 2020–12-01 and again on 2021–01-04, allowing us to evaluate patient-participant recruitment results at the 6-month mark from date of trial start (see [11] for further elaboration of our methods).

Data curation

We evaluated patient-participant recruitment for each trial in our cohort. Pre-specified criteria for unsuccessful recruitment were: (i) trial was “terminated” or “suspended” for a reason unrelated to efficacy, safety or progression of science (an automated extraction of the “why_stopped” element from ClinicalTrials.gov trial registration records was performed, enabling this categorization); (ii) trial was “completed” or “active, not recruiting” with a final enrollment less than 85% of the anticipated enrollment cited in the registration record at trial start, thus reflecting a substantial loss of statistical power for the primary outcome [4]; or, (iii) trial was “recruiting” or “enrolling by invitation” and the recruitment period had been extended to at least twice as long as anticipated, based on the planned recruitment length at trial start.

We aggregated trial recruitment goals on a state-by-state basis, and then compared this to the number of contemporaneous active SARS-CoV-2 cases in each state between 2020–01-01 and 2020–06-30. This involved (i) estimating the goal patient-participant enrollment per state for each trial (dividing planned enrollment per trial by the total number of sites per trial and multiplying this by the number of trial sites per USA state); (ii) aggregating total patient-participant enrollment goals per state; (iii) estimating the number of active SARS-CoV-2 cases per state between 2020–01-01 and 2020–06-30 using data downloaded from usafacts.org; and, (iv) calculating the proportion of SARS-CoV-2 cases that would need to be recruited to fulfill aggregate trial enrollment goals by state. Multisite trials with at least 1 site in the USA were included in our analysis. Patient-participant enrollment numbers were prorated up to 2020–06-30 for trials not achieving primary outcome completion by this date.

Outcomes and statistical analysis

We report the proportion of trials with successful patient-participant recruitment, as well as the proportion of SARS-CoV-2 cases required to fulfill trial recruitment goals by state. For all states with two or more active trials during our 6-month time period, we performed a simple and multivariable regression analysis evaluating the relationship between the proportion of trial sites within each USA state with unsuccessful patient-participant recruitment and: (i) the proportion of cases required to reach state recruitment goals; (ii) state population based on data from the US Census [12]; and, (iii) number of trial sites per state. We hypothesized that increased state recruitment goals would result in an increase in unsuccessful patient-participant recruitment. To investigate this further, a robust regression was performed using the M-estimation with Huber weights [13]. This specific type of regression analysis was chosen due to the continuous nature of the dependent variable and our detection of influential observations based on the Cook distance. The results of regression analysis were presented as beta coefficient (b) with 95% confidence interval (95% CI). Robust regressions were performed using the robustbase package [14], R version 4.1.0 (2021) [15]. We defined p < 0.05 as statistically significant.

Results

We included 151 interventional SARS-CoV-2 treatment trials in our cohort. The majority (68.21%) were Phase 2; 129 (85.43%) were randomized. Study status at 6 months since trial start was “Recruiting” in 99 trials (65.56%) (Table 1). Median anticipated enrollment per trial was 152 patients (IQR 60—400). The proportion of trials with successful patient-participant recruitment was 72.19% (109 of 151 trials).

Table 1 Characteristics of trial cohort

Forty-seven of 50 states launched at least one SARS-CoV-2 treatment trial. Three states (Kansas, Vermont and North Dakota) only had 1 trial location per state and were excluded from our linear analysis. California had the greatest number of trial sites per state (268), followed by New York (204) and Texas (197). Fourteen states and the District of Columbia required an enrollment of at least 1% of all persons with SARS-CoV-2 in their region to fulfill recruitment goals (Table 2). The elevated New Mexico recruitment requirement was mostly driven by a single-site clinical trial (NCT04458948) with an anticipated enrollment of 10,000 participants.

Table 2 Percent recruitment required per state

We did not find any significant relationship between unsuccessful patient-participant recruitment, increased state recruitment goals, state population or the number of trial sites per state in our simple and multivariable regression analyses (Table 3).

Table 3 Simple and multivariable robust regression analyses

Discussion

Most trials in our cohort demonstrated successful patient-participant recruitment (72.19%; 109 of 151 trials). Fourteen states and the District of Columbia required the enrollment of at least 1% of the SARS-CoV-2 cases in their jurisdiction to fulfill recruitment goals. Prior research in oncology has demonstrated that trials with a lower enrollment fraction (percentage of eligible patient-participants needed to fulfill goal trial enrollment) are more likely to successfully accrue [3]. However, the level of competition for participants in USA oncology trials appears to be greater than was seen early in the pandemic. Across all cancer types, approximately 8% of newly diagnosed patients in the USA would need to enroll in an interventional oncology trial to fulfill trial recruitment goals [16]. In comparison, the percentage recruitment requirements per state in our study were modest (Table 2), which is in keeping with our regression analyses that did not demonstrate a significant correlation between state-level recruitment targets and unsuccessful trial recruitment.

Our results differ from those of Franks et al. who noted declining alignment between SARS-CoV-2 cases and trial sites during the course of the pandemic, as evidenced by an increase in the number of counties with significant case numbers without local trial sites, as well as an upward trend in counties with a high volume of trial sites for the number of local cases [10]. These findings were most pronounced after June 2020. Our study focused on the first 6 months of the pandemic, which may explain why we did not find a significant relationship between recruitment goals and unsuccessful patient-participant recruitment.

Conclusion

Others have criticized a lack of coordination in SARS-CoV-2 trial activation early in the pandemic [2, 17, 18]. Despite these early failures in trial coordination, our findings did not demonstrate a significant relationship between competition for a limited pool of participants and under-accrued clinical trials. This may reflect the unique circumstances of the first few months of the pandemic in the United States, in which the number and location of SARS-CoV-2 cases was sufficient to meet trial recruitment requirements.

Limitations

Our study has four main limitations. First, we assumed absence of interstate and international travel in our calculation of the proportion of patient-participants required to fulfill trial enrollment goals by state. Given reduced travel during the early months of the pandemic due to stay-at-home orders adopted in the majority of USA states and territories, [19] this seemed a reasonable assumption. Second, we estimated competition for patient-participant trial enrollment at a state level, rather than at a county level, under the assumption that a degree of inter-county travel was maintained in the setting of trial SARS-CoV-2 clinical care and trial recruitment. Third, for multi-site trials we assumed equal goal patient-participant enrollment per trial site, due to lack of public access to site-specific enrollment goals. Fourth, we relied on data from ClinicalTrials.gov and usafacts.org to estimate trial feasibility and to model state level trial recruitment dynamics. Our results are thus dependent on the accuracy of both ClincalTrials.gov registration records and usafacts.org 2020 SARS-CoV-2 daily new case counts.

Availability of data and materials

The data sets used in this analysis can be freely and openly accessed on Open Science Framework under 10.17605/OSF.IO/FP726. Please see reference [20] for link to the data.

References

  1. Bugin K, Woodcock J. Trends in COVID-19 therapeutic clinical trials. Nat Rev Drug Discov. 2021;20(4):254–5.

    Article  CAS  Google Scholar 

  2. Jones CW, Woodford AL, Platts-Mills TF. Characteristics of COVID-19 clinical trials registered with ClinicalTrials.gov: cross-sectional analysis. BMJ Open. 2020;10(9):e041276.

    Article  Google Scholar 

  3. Bennette CS, Ramsey SD, McDermott CL, Carlson JJ, Basu A, Veenstra DL. Predicting low accrual in the national cancer institute’s cooperative group clinical trials. J Natl Cancer Inst. 2016. https://doi.org/10.1093/jnci/djv324.

    Article  Google Scholar 

  4. Carlisle B, Kimmelman J, Ramsay T, MacKinnon N. Unsuccessful trial accrual and human subjects protections: an empirical analysis of recently closed trials. Clin Trials. 2015;12(1):77–83.

    Article  Google Scholar 

  5. Damen L, van Agt F, de Boo T, Huysmans F. Terminating clinical trials without sufficient subjects. J Med Ethics. 2012;38(7):413–6.

    Article  Google Scholar 

  6. Gelinas L, Lynch HF, Bierer BE, Cohen IG. When clinical trials compete: prioritising study recruitment. J Med Ethics. 2017;43(12):803–9.

    Article  Google Scholar 

  7. Wertheimer A. Non-completion and informed consent. J Med Ethics. 2014;40(2):127–30.

    Article  Google Scholar 

  8. Janiaud P, Axfors C, Ioannidis JPA, Hemkens LG. Recruitment and results reporting of COVID-19 randomized clinical trials registered in the first 100 days of the pandemic. JAMA Netw Open. 2021;4(3): e210330.

    Article  Google Scholar 

  9. Cunniffe NG, Gunter SJ, Brown M, Burge SW, Coyle C, De Soyza A, et al. How achievable are COVID-19 clinical trial recruitment targets? A UK observational cohort study and trials registry analysis. BMJ Open. 2020;10(10): e044566.

    Article  Google Scholar 

  10. Franks L, Liu H, Elkind MSV, Reilly MP, Weng C, Lee SM. Misalignment between COVID-19 hotspots and clinical trial sites. J Am Med Inform Assoc. 2021;28(11):2461–6.

    Article  Google Scholar 

  11. Hutchinson N, Klas K, Carlisle BG, Kimmelman J, Waligora M. How informative were early SARS-CoV-2 treatment and prevention trials ?A longitudinal cohort analysis of trials registered ClinicalTrials.gov. 2022. PloS ONE. https://doi.org/10.1371/journal.pone.0262114.

  12. U.S. Census Bureau. Annual and cumulative estimates of resident population change for the United States, Regions, States, District of Columbia, and Puerto Rico and Region and State Rankings: April 1, 2020 to July 1, 2021 (NST-EST2021-CHG). https://www.census.gov/data/tables/time-series/demo/popest/2020s-national-total.html. Accessed 10 June 2022.

  13. Huber P. Robust regression: asymptotics, conjectures and Monte Carlo. Ann Stat. 1973;1(5):799–821.

    Article  Google Scholar 

  14. Maechler M, Rousseeuw P, Croux C, Todorov V, Ruckstuhl A, Salibian-Barrera M, Verbeke T, Koller M, Conceicao EL, Anna di Palma M. robustbase: Basic robust statistics. R package version 0.95–0. 2022. http://robustbase.r-forge.r-project.org/. Accessed 16 Sept 2022.

  15. R Core Team. R: A language and environment for statistical computing. Vienna: R foundation for statistical computing; 2013.

    Google Scholar 

  16. Tran G, Harker M, Chiswell K, Unger JM, Fleury ME, Hirsch B, et al. Feasibility of cancer clinical trial enrollment goals based on cancer incidence. JCO Clin Cancer Inform. 2020;4:35–49.

    Article  Google Scholar 

  17. Kouzy R, Abi Jaoude J, Garcia Garcia CJ, El Alam MB, Taniguchi CM, Ludmir EB. Characteristics of the multiplicity of randomized clinical trials for Coronavirus disease 2019 launched during the pandemic. JAMA Netw Open. 2020;3(7): e2015100.

    Article  Google Scholar 

  18. Naci H, Kesselheim AS, Rottingen JA, Salanti G, Vandvik PO, Cipriani A. Producing and using timely comparative evidence on drugs: lessons from clinical trials for covid-19. BMJ. 2020;371: m3869.

    Article  Google Scholar 

  19. Moreland A, Herlihy C, Tynan MA, et al. Timing of state and territorial COVID-19 stay-at-home orders and changes in population movement—United States, March 1–May 31, 2020. MMWR Morb Mortal Wkly Rep. 2020;69:1198–203. https://doi.org/10.15585/mmwr.mm6935a2.

    Article  CAS  Google Scholar 

  20. Hutchinson N, Carlisle BG, Klas K, Polak M, Kimmelmen J, Waligora M. The informativeness of trials in COVID-19: lessons learned from the Coronavirus pandemic. OSF; 2022. Available from: https://osf.io/fp726.

Download references

Acknowledgements

We thank Lucja Zaborowska for her important contribution to the data extraction for this project.

Funding

This study was funded by the National Science Center, Poland, UMO-2020/01/0/HS1/00024 (www.ncn.gov.pl). Authors received the funding: MW, KK. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Author information

Authors and Affiliations

Authors

Contributions

NH: Conceptualization, Data Curation, Formal Analysis, Methodology, Writing of the original draft, Writing—review and editing. BC: Conceptualization, Formal Analysis, Methodology, Writing—review and editing. KK: Conceptualization, Data Curation, Methodology, Writing—review and editing. MW: Conceptualization, Funding Acquisition, Methodology, Project Administration, Supervision, Writing—review and editing. JK: Conceptualization, Writing- review and editing. MP: Formal Analysis, Methodology, Writing—review and editing. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Marcin Waligora.

Ethics declarations

Ethics approval and consent to participate

Our study relied on publicly accessible data and was not subject to Institutional Review Board approval.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hutchinson, N., Klas, K., Carlisle, B.G. et al. Competition for recruitment in SARS-CoV-2 Trials in the United States: a longitudinal cohort analysis. BMC Res Notes 15, 368 (2022). https://doi.org/10.1186/s13104-022-06263-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13104-022-06263-1

Keywords

  • Clinical trials
  • Patient-participant recruitment
  • Competition
  • SARS-CoV-2
  • COVID-19
  • United States