Skip to main content

What discontinued trials teach us about trial registration?

Abstract

Objective

Trial registries were set up to improve transparency, remove duplication, improve awareness and avoid waste. Many trials never reach the point of patient enrolment due to a myriad of reasons. The aim of this study was to investigate the reasons for and characteristics of discontinuation of trials.

Results

A total of 163 discontinued trials were identified and compared to completed trials. A Survey was designed to further explore the nature and conduct of the trial. No differences in registered and categorised information was observed between discontinued and completed trials. Most trials discontinue due to patient or participant recruitment issues, often related to funding. Substantial changes to procedures or the protocol or changes to recruitment strategy were also commonly cited reasons. Survey information was available for 21 discontinued and 28 completed trials and no obvious differences could be identified. Our findings highlight the underlying problem of lack of detail, suboptimal recording, dated information and incomplete reporting of trials within a trial registry which hampers sharing and learning. To date, important progress has been made by the implementation of standards and the requirement of trials to be registered. Our review identifies areas where further improvements can be made.

Introduction

Clinical trials are costly, time consuming, resource intensive and burdensome. Apart from the financial and resource implications, there is an ethical imperative not to expose patients to trials that are not viable or necessary. It is therefore of importance that all learnings from clinical trials are taken into account and not wasted [1]. Trial registries were set up to improve transparency, remove duplication improve awareness of trials and avoid waste in RCTs. The value and use of registries relies on complete, accurate, up to date and easily accessible information which includes the publication of protocols at study inception [2, 3]. This provides researchers with a comprehensive overview of current and previous research to avoid biased evidence, unjustified research, waste, and actual patient harm from unnecessary trials or a delay in access to beneficial treatments [4, 5].

The International Standard Randomised Controlled Trial Number (ISRCTN) register (www.ISRCTN.com) was set up in 2000. The ISRCTN register is an online searchable registry of clinical research studies which provides a unique identification number for each registered trial linked to a record of the study. Key information is recorded for each registered trial (e.g. title, primary contact, experimental hypotheses, primary and secondary outcomes, trial design, intervention). Most funding agencies and sponsors require the registration of any trial in public registries.

According to the revised Declaration of Helsinki “every clinical trial must be registered in a publicly accessible database before recruitment of first subject” [6]. Similarly, the International committee of medical Journal Editors (ICMJE) requires registration of trial methodology to protect patient interests and confidentiality and prevent unethical conduct. However, no further guidelines or requirements in relation to the results of a RCT are provided by the ICMJE [7]. While the quality of the information provided at registration was found to have improved 2004 to 2009, gaps in the provided information remained [1] including the failure to update procedures, data and results [8].

To highlight some of the issues with waste in trial research, a series of five reviews in the Lancet ‘Research: increasing value, reducing waste’, outlined how research can be made more efficient through the improved use and sharing of information [2, 9,10,11,12]. This series provided a list of 17 recommendations to help increase value, and covers issues relating to funders, regulators, journals, academic institutions and researchers. One of its suggestions to support successful replication of basic research and its application in health care, called for more ‘research on research’ [2].

An area for potential research on research which received little attention, is trials that are stopped before completion. Since the start of the ISRCTN registry, of the 14,000 registered studies, about 3% are classified as stopped i.e. discontinued. Considering that clinical trials can cost between €600,000 and €1.5 M, the waste of time and money is substantial [13].

Main text

Database review

Through the open access ISRCTN registry, we identified all RCT registered since 2000. On request, the ISRCTN registry provided us with a file including all online published information on all trials registered since 2000. Within this file, we identified 163 discontinued RCTs registered since 2000. To explore potential differences with completed trials we selected the next completed trial immediately registered after the discontinued as the comparison.

The information extracted from the ISRCTN registry included: the registration number, title of the trial, date of registration and overall trial status (stopped/completed). Additional information was extracted from the ISRCTN online database on age group; gender; contact person; type of study (prospective/retrospective); condition category, date registration and last edited; participant inclusion criteria; funder name; ethics approval; study design, trial setting, type and phase; start and end date; reason abandoned (if applicable); target number of participants; countries of recruitment, organisation; sponsor type. When an ISRCTN registered trial is reported as discontinued, the reason for abandonment is requested by the ISRCTN team and added to the database if obtained.

Four researchers reviewed and extracted information from 163 discontinued and 163 completed trials listed in the ISRCTN registry, immediately following the discontinued trial.

Survey

A Survey Monkey questionnaire to obtain more information on each discontinued and completed trial was designed. The survey link was sent to the email of the contact listed on the ISRCTN online database. A mailing was send to all emails which included two reminders as well as searches to update email addresses of bounced mails.

Results

For the database review, a total of 163 discontinued trials and 163 completed trials were evaluated. The condition category of the trials is shown in Table 1.

Table 1 Overview of the clinical area in discontinued and completed trials

There were no differences observed between discontinued and completed trial. This included the study design, which was mainly interventional (as opposed to observational), the secondary study design (e.g. cluster, cross over and parallel) and the setting (for most trials this was hospital (68%)).

The type of trial was generally testing different treatments but discontinued trials were significantly (p = 0.02) more often treatment trials (Table 2). Completed trials were more likely to involve diagnostic, prevention, quality of life, or ‘other types’ of trials.

Table 2 Overview of the type of discontinued and completed trials

The duration between the registration of the trial and discontinuation or completion showed no difference. However, the time period represents the period from registration unit the trial was stopped for discontinued trials while it represents the period from registration until completion for the completed trials.

Evaluation of the reasons for discontinuation

The reason to discontinue a trial is reported to the ISRCTN registry as part of the process to update the database (Table 3). Most trials discontinue due to patient or participant recruitment issues (37%), often related to funding (17%) as increasing efforts to recruit patients generally has financial implications. Second to recruitment/funding issues, substantial changes to procedures or the protocol are indicated as reason for discontinuation (9%), changes to recruitment strategy or treatment were most commonly cited changes. Other issues recorded were problems with the purchase or delivery of the trial product, staff (including recruitment) and PI/internal issues, information indication equipoise, unattainable objectives, adverse events, DSMB (Data and Safety Monitoring Board) decision, regulatory issues, and superseded technology. For four trials the reason for discontinuation was not known.

Table 3 List of key reasons for discontinuation

Survey

The email survey was completed for 21 discontinued and 28 completed trials. The results of the survey provided little new information on the reason why the trial was discontinued beyond the database review due to the low number of respondents.

Limitations

Our results indicate that the main reason for discontinuation is patient/participant recruitment, reflected in inflated target sample size estimates and unrealistic recruitment goals. Recruitment issues can delay trials and are directly related to funding. As previously identified, failure to meet recruitment targets due to overoptimistic or inaccurate estimates of recruitment [14] which could be anticipated by conducting a pilot or feasibility study [15]. Delays, due to changes in the protocol, or recruitment issues, also have an impact on researcher retention, often reported to the ISRCTN as another reason to discontinue a trial.

The existing information contained in the ISRCTN database is not detailed or specific enough to allow for the identification of reasons why a trial was discontinued. The information that was available was also difficult to interpret, similar to previous findings, as most data fields did not consist of standardised categories [8]. To allow more in-depth understanding of what factors are associated with completed trials, more information is necessary on actual recruitment, participation of a clinical trial unit or an (independent) trial steering committee. Such information can also help funders to assess if a study is delivering on its objectives.

The International Standards for Clinical Trial Registries (2012[16] and 2018[17]) designed a minimum dataset (Trial Registration Data Set) covering 20 items [18]. Our findings provide evidence of the underlying problem of lack of detail, suboptimal recording, dated information and incomplete reporting of trials which hampers sharing and learning.

Today, funding agencies as well as publishers, request the trial registration number. Information required to register a trial is rarely linked to the primary report once the research is complete resulting in the drop in the relevance of registries after the trial obtains a registration number. However, funding agencies require proof of ethical approval before funding is transferred while journal editors require checklists for the CONSORT [19]., QUOROM [20], STROBE [21] and STARD [22] statements. Consideration should be given to opportunities to bundle all this information as part of an integrated registration process necessary as part of the publication of the trial results.

An addition to these requirements could include the submission of a protocol as part of registration [23]. However, “research is not a car factory” [10] and it is important to acknowledge that adjustments to the design process are expected. Including a protocol should therefore not be a static document upload but allow to make adjustments as part of a well-documented process. Part of this process is already required by journals and included as ‘deviations from the protocol’ in the manuscript.

There are several parties responsible in ensuring access to complete and meaningful information on RCTs conducted throughout the world; the registry, the registrant and other stakeholders such as journal editors, ethical committees and funding agencies [18]. Enforcing measures designed to improve the quality of registration are suggested for journal editors and legislators, but little is known on if or how this is done [16].

Based on our study and to boost learning from others and other trials, we suggest following improvements:

Registries should consider to:

  • Provide a downloadable template at the moment of registration which is required when publishing the results of the trial, similar to evidence of ethical approval or statements

  • Improve the standardisation of the information provided through the use of standardised categories

  • Allow registries to be fully searchable using standardised reporting.

  • Extend the information gathered with up to date ethical approval information, CONSORT [19]., QUOROM [20], STROBE [21] and STARD [22] statements, actual recruitment information and other trial procedures

Funding agencies should consider to:

  • Encourage and fund feasibility studies to support realistic recruitment targets

Journal editors should consider to:

  • Request an up to date trial registry template as part of the manuscript submission

Our review of discontinued trials showed that not enough progress has been made to allow learning from other research. Five targeted groups (funders, regulators, journals, academic institutions and researchers) have been identified to play a part in increasing research value. We argue there is a sixth group: the trial registry should provide a common template with up-to-date information to be submitted as part of the publication of the trial results documenting each stage of the research process, from conception, through registration and publication.

Availability of data and materials

All data and material is available at request by mailing the corresponding author.

Abbreviations

DSMB:

Data and Safety Monitoring Board

ICMJE:

International committee of medical Journal Editors

ISRCTN:

International Standard Randomised Controlled Trial Number

RCT:

Randomised Clinical Trial

References

  1. 1.

    Viergever RF, Karam G, Reis A, Ghersi D. The quality of registration of clinical trials: still a problem. PLoS ONE. 2014;9(1):e84727.

    Article  Google Scholar 

  2. 2.

    Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gülmezoglu AM, Howells DW, Ioannidis JPA, Oliver S. How to increase value and reduce waste when research priorities are set. The Lancet. 2014;383(9912):156–65.

    Article  Google Scholar 

  3. 3.

    Begley CG, Ellis LM. Raise standards for preclinical cancer research. Nature. 2012;483(7391):531–3.

    CAS  Article  Google Scholar 

  4. 4.

    Cooper NJ, Jones DR, Sutton AJ. The use of systematic reviews when designing studies. Clin Trials. 2005;2(3):260–4.

    Article  Google Scholar 

  5. 5.

    Goudie AC, Sutton AJ, Jones DR, Donald A. Empirical assessment suggests that existing evidence could be used more fully in designing randomized controlled trials. J Clin Epidemiol. 2010;63(9):983–91.

    Article  Google Scholar 

  6. 6.

    Goodyear MD, Krleza-Jeric K, Lemmens T. The Declaration of Helsinki. BMJ. 2007;335(7621):624–5.

    Article  Google Scholar 

  7. 7.

    Aslam A, Imanullah S, Asim M, El-Menyar A. Registration of clinical trials: is it really needed? North Am J Med Sci. 2013;5(12):713–5.

    Article  Google Scholar 

  8. 8.

    Viergever RF, Li K. Trends in global clinical trial registration: an analysis of numbers of registered clinical trials in different parts of the world from 2004 to 2013. BMJ Open. 2015;5(9):e008932.

    Article  Google Scholar 

  9. 9.

    Moher D, Shamseer L, Cobey KD, Lalu MM, Galipeau J, Avey MT, Ahmadzai N, Alabousi M, Barbeau P, Beck A, et al. Stop this waste of people, animals and money. Nature. 2017;549(7670):23–5.

    CAS  Article  Google Scholar 

  10. 10.

    Moher D, Glasziou P, Chalmers I, Nasser M, Bossuyt PM, Korevaar DA, Graham ID, Ravaud P, Boutron I. Increasing value and reducing waste in biomedical research: who’s listening? Lancet. 2016;387(10027):1573–86.

    Article  Google Scholar 

  11. 11.

    Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, Michie S, Moher D, Wager E. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383(9913):267–76.

    Article  Google Scholar 

  12. 12.

    Ioannidis JP, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, Schulz KF, Tibshirani R. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014;383(9912):166–75.

    Article  Google Scholar 

  13. 13.

    Hind D, Reeves BC, Bathers S, Bray C, Corkhill A, Hayward C, Harper L, Napp V, Norrie J, Speed C, et al. Comparative costs and activity from a sample of UK clinical trials units. Trials. 2017;18(1):203.

    Article  Google Scholar 

  14. 14.

    Duley L, Gillman A, Duggan M, Belson S, Knox J, McDonald A, Rawcliffe C, Simon J, Sprosen T, Watson J, et al. What are the main inefficiencies in trial conduct: a survey of UKCRC registered clinical trials units in the UK. Trials. 2018;19(1):15.

    Article  Google Scholar 

  15. 15.

    Eldridge SM, Lancaster GA, Campbell MJ, Thabane L, Hopewell S, Coleman CL, Bond CM. Defining Feasibility and Pilot Studies in Preparation for Randomised Controlled Trials: Development of a Conceptual Framework. PLoS ONE. 2016;11(3):e0150205.

    Article  Google Scholar 

  16. 16.

    World Health Organization. International standards for clinical trial registries. Geneva: World Health Organization; 2012.

    Google Scholar 

  17. 17.

    World Health Organization. International standards for clinical trial registries: the registration of all interventional trials is a scientific, ethical and moral responsibility, version 3.0. Geneva: World Health Organization; 2018.

    Google Scholar 

  18. 18.

    WHO Trial Registration Data Set. http://www.who.int/ictrp/network/trds/en/. Accessed May 2019.

  19. 19.

    Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG. CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869.

    Article  Google Scholar 

  20. 20.

    Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement Quality of Reporting of Meta-analyses. Lancet. 1999;354(9193):1896–900.

    CAS  Article  Google Scholar 

  21. 21.

    Vandenbroucke JP, von Elm E, Altman DG, Gotzsche PC, Mulrow CD, Pocock SJ, Poole C, Schlesselman JJ, Egger M. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. Int J Surg. 2014;12(12):1500–24.

    Article  Google Scholar 

  22. 22.

    Cohen JF, Korevaar DA, Altman DG, Bruns DE, Gatsonis CA, Hooft L, Irwig L, Levine D, Reitsma JB, de Vet HC, et al. STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open. 2016;6(11):e012799.

    Article  Google Scholar 

  23. 23.

    Chan AW, Song F, Vickers A, Jefferson T, Dickersin K, Gotzsche PC, Krumholz HM, Ghersi D, van der Worp HB. Increasing value and reducing waste: addressing inaccessible research. Lancet. 2014;383(9913):257–66.

    Article  Google Scholar 

Download references

Acknowledgements

We would like to acknowledge the help provided by the ISRCTN identifying and downloading the trial information

Funding

The authors have no funding to declare.

Author information

Affiliations

Authors

Contributions

AV conceived and designed the study, extracted and analysed the information from the ISRCTN, and drafted the manuscript. KL organised the survey. All authors reviewed the information extracted from the ISRCTN, read drafts of the manuscript and approved the final manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Akke Vellinga.

Ethics declarations

Ethics approval and consent to participate

Ethical approval was obtained from the NUI Galway Research Ethics Committee. Consent to participate was not applicable.

Consent for publication

Not applicable.

Competing interests

The authors have no conflict of interest to declare.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Vellinga, A., Lambe, K., O’Connor, P. et al. What discontinued trials teach us about trial registration?. BMC Res Notes 14, 47 (2021). https://doi.org/10.1186/s13104-020-05391-w

Download citation