Skip to main content

Improving medical research in the United Kingdom

Abstract

Poor quality medical research causes serious harms by misleading healthcare professionals and policymakers, decreasing trust in science and medicine, and wasting public funds. Here we outline underlying problems including insufficient transparency, dysfunctional incentives, and reporting biases. We make the following recommendations to address these problems: Journals and funders should ensure authors fulfil their obligation to share detailed study protocols, analytical code, and (as far as possible) research data. Funders and journals should incentivise uptake of registered reports and establish funding pathways which integrate evaluation of funding proposals with initial peer review of registered reports. A mandatory national register of interests for all those who are involved in medical research in the UK should be established, with an expectation that individuals maintain the accuracy of their declarations and regularly update them. Funders and institutions should stop using metrics such as citations and journal’s impact factor to assess research and researchers and instead evaluate based on quality, reproducibility, and societal value. Employers and non-academic training programmes for health professionals (clinicians hired for patient care, not to do research) should not select based on number of research publications. Promotions based on publication should be restricted to those hired to do research.

Introduction

The UK invests substantial sums in medical research and the international reputation of the sector is vital to the government’s industrial strategy [1]. Unfortunately, systemic problems undermine the rigour of medical research and lead to costly research waste [2].

In this commentary, we propose straightforward measures to reduce waste in medical research that will safeguard investments and ensure the UK remains a productive setting for researchers committed to genuine scientific discovery.

Main text

What are the causes of the reproducibility crisis in medical research?

While fraud is an important problem that leads to non-reproducible research, many of the issues that undermine reproducibility do not involve deliberate misconduct [3, 4]. Here, we concentrate on identifying and suggesting remedies to the systemic threats to research rigour and reproducibility.

Lack of transparency for methods and data

Although most UK medical research is funded through taxation and charitable donations, there are remarkably few requirements placed on researchers to share data, adequately describe methods, or provide their full results. Understanding exactly how studies were carried out requires full transparency of study methods and analytic code (where applicable) to verify results and conclusions, ideally with full access to the data. Without providing this information, the validity of published results can only be taken on trust.

Some journals now have policies requiring authors to make their data, analytic code, and protocols available. In practice, these requirements are routinely ignored [5,6,7]. Considerations of patient privacy or commercial confidentiality are not insurmountable barriers to adequate transparency. Research participants should be offered the opportunity to consent to the sharing of their data, or data can be de-identified and provided via third-party, securely-managed access. The OpenSafely project has provided a successful model throughout the COVID-19 pandemic for safe and secure access to UK patient records for emergency epidemiological research [8]. While individual electronic health records cannot be shared in this instance, the OpenSafely project has made transparency into data access, code, and analysis plans central to the project. The Vivli platform provides another model for securely sharing data collected from clinical research studies and numerous platforms exist for sharing non-sensitive data [9].

Consistent and mandatory sharing of data, code, and protocols via journals is one route to improved transparency. However, there is a strong case that all study documentation for publicly funded research should be reported in the public realm at study completion (regardless of article publication status). The establishment of a single national repository would provide a much more complete and useful record of experiments, as well as reduce duplication in reporting requirements that researchers currently face. The Health Research Authority has recently announced plans for all clinical trials to be automatically registered following ethical approval, in partnership with the ISRCTN registry [10]. This infrastructure could form the nucleus of a more ambitious research repository, hosting comprehensive documentation for all study types.

Proposed measure 1 Journals and funders should ensure authors fulfil their obligation to share study protocols and, as far as possible, analytical code and research data regardless of the study’s ultimate publication status in a journal.

Reporting and publication biases

It is well established that results which are ‘positive’ or novel are much more likely to be published [11] leading to a distorted public record. One study found that 96% of research findings were ‘statistically significant’, a mathematical impossibility [12]. Where unpublished results are taken into account, the evidence of benefit for interventions may diminish or disappear altogether [11, 13].

We can monitor whether clinical trials have been published because of legal and ethical requirements that clinical trials are registered prior to commencement [14]. However, clinical trials are a minority of all medical studies. For other study types, including observational studies that have informed healthcare decision-making throughout the coronavirus pandemic, there is little expectation that researchers pre-register their analysis plans and hypotheses. This makes it impossible to monitor what research is planned, if research plans have been followed, if all results are published, and if interpretation involves unscientific “spin” (Table) [15]. Pre-registering study plans for any research type is now straightforward and free to researchers [16, 17]. Requiring pre-registration for medical research can and should be used to promote transparency and accountability.

Although we believe pre-registration enhances transparency and should be more widely used, even when research is pre-registered, it is common to find unacknowledged deviations between the proposed research and the published manuscript [6]. While registration makes these deviations detectable, detection is onerous and journals often fail to take action when alerted to these issues [6]. Related ‘questionable research practices’, like manipulating analyses to generate statistically significant results (‘p-hacking’) and amending study hypotheses retrospectively to suit the results found (Hypothesising After the Result is Known, or ‘HARKing’) can lead to biased findings when research is not pre-registered or the registrations are not checked against final reports [18].

Registered reports are a publishing format with two methods for preventing these poor research practices. In a registered report, reviewers assess methods before data collection begins. When the researchers and reviewers agree that the design is appropriate, the researchers are given in-principle approval for publication, regardless of the study findings, as long as the proposed methodology is adhered to [19].

Promoting publishing via registered reports addresses both outcome reporting and publication biases for almost all types of research [19]. Since journals decide on publication based on the methods before the results are known, researchers have less incentive to strain to produce eye-catching or positive results. Instead, methodologically sound research enters the record without bias regarding what those results might say. Early evidence suggests that registered reports do improve the quality and rigor of proposed study designs [20, 21]. Unfortunately, medicine lags behind other disciplines like psychology in adopting registered reports, with just over 1% of medical journals offering the format in 2020 [22].

Funders should either incentivise publication via registered report, which will increase their uptake by more journals, or establish their own publication platforms which prioritise the registered report format. The NIHR and Wellcome Trust have both established open and peer-reviewed publication platforms for their funded research [23, 24]. Novel funding pathways are demonstrating that it is possible to integrate peer reviews for registered reports in grant applications which could improve the efficiency of academic discovery and dissemination [25, 26].

Proposed measure 2 Funders and medical journals should incentivise the uptake of registered reports.

Lack of transparency on conflicts of interest

Researchers potential conflicts of interest are difficult to ascertain. Conflict of interest statements in publications are brief and often omit important conflicts, even major financial conflicts like sources of funding or relationships with industry [27]. The RetractionWatch project has logged numerous instances of problematic findings, and eventual retractions of articles, due to important undisclosed conflicts of interest [28]. In the UK, there are currently voluntary registers for those who have received payments from the pharmaceutical industry and for doctors [29, 30]. However, because there is little incentive for individuals to make such declarations, these registers are greatly underutilised. Only 0.002% of doctors’ registered with the General Medical Council (GMC) were listed on the doctors’ voluntary register in 2020 [31]. Patients, the public and other scientists are entitled to be able to check whether a researcher has a conflict of interest and to understand what potential biases might impact a study.

There have been calls for the GMC to curate a registry of doctors’ interests [32, 33]. While such a register would mark an important advance in transparency, it would not cover non-medically qualified researchers. Therefore the UK’s research regulator, the Health Research Authority (HRA), along with the GMC, could be tasked with creating a central register, similar to the US OpenPayments database, to index all medical researchers’ interests using the unique identity numbers (ORCiD) which are already required by institutions and funders [34, 35] Expectations of accurate and up-to-date declarations could be encouraged by employers during researchers’ appraisals, although a legal mandate for regulators to ask for such declarations may require legislation [36].

Proposed measure 3 a mandatory national register of interests for all those who are involved in medical research in the UK should be established, with an expectation that individuals maintain the accuracy of their declarations.

Dysfunctional incentives and research culture

Decisions about who to hire, fund and promote in academia are often informed using reductive, simple metrics such as citations, the journal impact factor of publications or grant income [37, 38]. Such metrics perversely incentivise researchers to generate a high quantity of publications which are perceived to be exciting or newsworthy, rather than prioritising high quality reproducible research that actually benefits patients and the public. Practices that support high quality research by improving transparency and reducing bias, such as registering studies and publishing all results, are not typically used to appraise performance in academia [39]. Initiatives which seek to mitigate dysfunctional incentives and promote practices that are conducive to reproducible research are becoming more widely established throughout the UK. These include the UK Reproducibility Network (UKRN), the San Francisco Declaration on Research Assessment (DORA), Résumé for Researchers, and the Concordat to Support the Career Development of Researchers [40,41,42,43,44]. Such efforts should be supported and expanded.

Proposed measure 4 funders and institutions should stop using metrics such as citations and journal’s impact factor to assess research and instead evaluate based on quality, reproducibility and societal value.

Some clinical professionals are incentivised to undertake research because publication is used as a selection criteria for training programmes and for professional promotion [45]. Such dysfunctional incentives promote research that is undertaken solely for career advancement, by individuals who may lack methodological expertise and commitment to produce high-quality reproducible research. There is no persuasive evidence that authoring scientific publications improves the clinical performance of healthcare professionals. Instead we should aspire to create a system of medical research that produces “less research, better research, and research done for the right reasons” [46].

Proposed measure 5 employers and training programmes for health professionals should remove incentives to publish from their selection procedures.

Outlook

In this commentary we have set out recommendations to improve the transparency and reproducibility of medical research. Not all of these would be easily implemented, and further evidence is needed to articulate their value and best practice. Moreover sincere engagement from funders, government, journals, researchers, and their employing institutions is required. Elsewhere, we highlighted three actions that warrant prioritisation as readily implementable measures with high potential for impact [22, 47]. These are:

  1. (1)

    Mandatory registration of interests for all people and institutions who conduct and publish health research on a single on-line platform accessible to all;

  2. (2)

    Prioritisation by journals and funders of publication of research via Registered Reports; and.

  3. (3)

    Public pre-registration before data collection of the study design of all publicly-funded medical research, along with protocols, analytic code and, where possible, research data and results.

Table 1 outlines how these actions, along with other proposed measures outlined in this commentary could address some of the problems in medical research.

Table 1 Problems in medical research and how they can be mitigated by authors’ proposed strategy

There is growing acknowledgement that systemic problems pervade health research. But as long as we fail to take action, medicine’s reproducibility crisis will persist. The recommendations in this commentary aim to advance beyond identification of the problems to meaningful, achievable reform.

Availability of data and materials

Not applicable.

Abbreviations

ISRCTN:

International standard randomised controlled trial number

UKRN:

United Kingdom reproducibility network

References

  1. Life Sciences Industrial Strategy: UK Government. Available from: https://www.gov.uk/government/publications/life-sciences-industrial-strategy.

  2. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. The Lancet. 2009;374(9683):86–9.

    Article  Google Scholar 

  3. Fanelli D. How many scientists fabricate and falsify research? a systematic review and meta-analysis of survey data. PLoS ONE. 2009;4(5): e5738.

    Article  Google Scholar 

  4. John LK, Loewenstein G, Prelec D. Measuring the prevalence of questionable research practices with incentives for truth telling. Psychol Sci. 2012;23(5):524–32.

    Article  Google Scholar 

  5. Wallach JD, Boyack KW, Ioannidis JPA. Reproducible research practices, transparency, and open access data in the biomedical literature, 2015–2017. PLoS Biol. 2018;16(11): e2006930.

    Article  Google Scholar 

  6. Goldacre B, Drysdale H, Dale A, Milosevic I, Slade E, Hartley P, et al. COMPare: a prospective cohort study correcting and monitoring 58 misreported trials in real time. Trials. 2019;20(1):118.

    Article  Google Scholar 

  7. Naudet F, Sakarovitch C, Janiaud P, Cristea I, Fanelli D, Moher D, et al. Data sharing and reanalysis of randomized controlled trials in leading biomedical journals with a full data sharing policy: survey of studies published in <em>The BMJ</em> and <em>PLOS Medicine</em>. BMJ. 2018;360: k400.

    Article  Google Scholar 

  8. OpenSAFELY. Available from: https://www.opensafely.org/.

  9. Vivli: A global research data sharing platform. Available from: https://vivli.org/.

  10. New partnership guarantees full picture of UK clinical trials Health Research Authority. Available from: https://www.hra.nhs.uk/about-us/news-updates/new-partnership-guarantees-full-picture-uk-clinical-trials/.

  11. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358(3):252–60.

    CAS  Article  Google Scholar 

  12. Chavalarias D, Wallach JD, Li AHT, Ioannidis JPA. Evolution of reporting p values in the biomedical literature, 1990–2015. JAMA. 2016;315(11):1141–8.

    CAS  Article  Google Scholar 

  13. Tamiflu Campaign. Available from: https://www.bmj.com/tamiflu.

  14. Clinical Trials Registration: International Committee of Medical Journal Editors. Available from: http://www.icmje.org/about-icmje/faqs/clinical-trials-registration/.

  15. Yavchitz A, Boutron I, Bafeta A, Marroun I, Charles P, Mantz J, et al. Misrepresentation of randomized controlled trials in press releases and news coverage: a cohort study. PLoS Med. 2012;9(9):e1001308-e.

    Article  Google Scholar 

  16. Open Science Framework. Available from: https://osf.io.

  17. International Prospective Register of Systematic Reviews (PROSPERO). Available from: https://www.crd.york.ac.uk/PROSPERO/.

  18. Munafò MR, Nosek BA, Bishop DVM, Button KS, Chambers CD, Percie du SN N, et al. A manifesto for reproducible science. Nat Hum Behav. 2017;1(1):0021.

    Article  Google Scholar 

  19. Registered Reports: Peer review before results are known to align scientific values and practices.: Centre for Open Science. Available from: https://cos.io/rr/.

  20. Soderberg CK, Errington TM, Schiavone SR, Bottesini J, Thorn FS, Vazire S, et al. Initial evidence of research quality of registered reports compared with the standard publishing model. Nat Hum Behav. 2021;5(8):990–7.

    Article  Google Scholar 

  21. Scheel AM, Schijen MRMJ, Lakens D. An excess of positive results: comparing the standard psychology literature with registered reports. Adv Methods Pract Psychol Sci. 2021;4(2):25152459211007468.

    Google Scholar 

  22. Bradley SH, De Vito NJ, Lloyd KE, Richards GC, Rombey T, Wayant C, et al. Reducing bias and improving transparency in medical research: a critical overview of the problems, progress and suggested next steps. J R Soc Med. 2020;113(11):433–43.

    Article  Google Scholar 

  23. Journals Library: National Intstiute for Health Research. Available from: https://www.journalslibrary.nihr.ac.uk/journals/.

  24. Wellcome Open Research. Available from: https://wellcomeopenresearch.org/.

  25. First Funding Cycle of the Drug Discovery Initiative Registered Report (DDIRR) Awards Announced [Available from: https://www.ctf.org/news/first-funding-cycle-of-the-drug-discovery-initiative-registered-report-ddir.

  26. Registered Reports funding partnership: Cancer Research UK; 2022. Available from: https://www.cancerresearchuk.org/funding-for-researchers/how-we-deliver-research/positive-research-culture/registered-reports.

  27. Wayant C, Turner E, Meyer C, Sinnett P, Vassar M. Financial conflicts of interest among oncologist authors of reports of clinical drug trials. JAMA Oncol. 2018;4(10):1426–8.

    Article  Google Scholar 

  28. Failure to Disclose COI. Available from: https://retractionwatch.com/category/by-reason-for-retraction/failure-to-disclose-coi/.

  29. Disclosure UK: Association of the British Pharmaceutical Industry. Available from: https://www.abpi.org.uk/our-ethics/disclosure-uk/

  30. Who pays this doctor?. Available from: http://www.whopaysthisdoctor.org/.

  31. Bradley SH, DeVito NJ, Lloyd K, Richards GC, Rombey T, Wayant C, et al. Reducing bias and improving transparency in biomedical and health research: A critical overview of the problems, progress so far and suggested next steps [preprint] 2020.

  32. Cumberlege BJ. 2020 First Do No Harm: The report of the independent medicines and medical devices safety review.

  33. Rimmer A. Briefing: why do we need a mandatory register of doctors’ interests? BMJ. 2021;373:n1280.

    Article  Google Scholar 

  34. ORCID: Open Researcher and Contributor ID. Available from: https://orcid.org/.

  35. Open Payments. Available from: https://www.cms.gov/openpayments.

  36. Coombes R. Doctors’ duty to declare their interests should be enforced, says GMC. BMJ. 2021;373:n1329.

    Article  Google Scholar 

  37. Rice DB, Raffoul H, Ioannidis JPA, Moher D. Academic criteria for promotion and tenure in biomedical sciences faculties: cross sectional analysis of international sample of universities. BMJ. 2020;369: m2081.

    Article  Google Scholar 

  38. Khan H, Almoli E, Franco MC, Moher D. Open science failed to penetrate academic hiring practices: a cross-sectional study. J Clin Epidemiol. 2022;144:136–43.

    Article  Google Scholar 

  39. Van Calster B, Wynants L, Riley RD, van Smeden M, Collins GS. Methodology over metrics: current scientific standards are a disservice to patients and society. J Clin Epidemiol. 2021. https://doi.org/10.1016/j.jclinepi.2021.05.018.

    Article  PubMed  PubMed Central  Google Scholar 

  40. San Francisco Declaration on Research Assessment resource library. Available from: https://sfdora.org/resource-library/.

  41. The UK Reproducibility Network (UKRN). Available from: https://www.ukrn.org/.

  42. San Francisco Declaration on Research Assessment. Available from: https://sfdora.org/.

  43. Résumé for Researchers: The Royal Society. Available from: https://royalsociety.org/topics-policy/projects/research-culture/tools-for-support/resume-for-researchers/.

  44. The Concordat to Support the Career Development of Researchers: vitae. Available from: https://www.vitae.ac.uk/policy/concordat.

  45. UK Foundation Programme 2022 Applicants’ Handbook. Available from: https://nwpgmd.nhs.uk/sites/default/files/UKFP%202022_Applicants%27%20Handbook_FINAL_0.pdf.

  46. Altman DG. The scandal of poor medical research. BMJ. 1994;308(6924):283–4.

    CAS  Article  Google Scholar 

  47. The Declaration to Improve Health RESearch. Available from: www.improvehealthresearch.com.

  48. Yavchitz A, Boutron I, Bafeta A, Marroun I, Charles P, Mantz J, et al. Misrepresentation of randomized controlled trials in press releases and news coverage: a cohort study. PLoS Med. 2012;9(9): e1001308.

    Article  Google Scholar 

Download references

Acknowledgements

David Mellor (Center for Open Science) contributed to the written evidence submitted to the parliamentary select committee inquiry on reproducibility and research integrity on which this commentary is based. We would also like to thank Till Bruckner (TranspariMED) for feedback provided on that submission.

Funding

Not applicable.

Author information

Affiliations

Authors

Contributions

SHB authored the first draft of the paper, NJD, KEL, PL and JEB contributed to subsequent revisions of the text. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Stephen H. Bradley.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

Stephen Bradley: I am employed as a General Practitioner for one day a week. I receive funding for PhD study from CanTest collaborative (Cancer Research UK, C8640/A23385). The publication costs of a collection of essays on health inequalities which I co-edited for the Fabian Society was funded by the Association of the British Pharmaceutical Industry and Lloyds Pharmacies, I received no direct funding or payment for this. I sit on the NIHR’s Health Services & Delivery research prioritisation committee (unpaid aside from reimbursement of travel expenses). I am a co-investigator on a study which is funded by Yorkshire Cancer Research (Patient-centred models for surveillance and support of cancer survivors with bowel and breast cancer). I am a member of the steering group of a campaign to improve health research (the Declaration to Improve Health Research). I have previously received funding from the Mason Medical Foundation to undertake a study on chest x-ray and lung cancer diagnosis.

Nicholas DeVito: I am a doctoral student at the DataLab (soon to be the Bennett Institute for Applied Data Science supported by the Peter Bennett Foundation) and the Centre for Evidence-Based Medicine at the University of Oxford and I am supported in my studies by a studentship from the Naji Foundation. I have been employed on grants in the last three years from the Laura and John Arnold Foundation, the Good Thinking Society, and the German Federal Ministry of Education and Research (BMBF). I have also received grant support from the Fetzer Franklin Memorial Fund.

Kelly Lloyd: I am supported by an Economic and Social Research Council studentship [grant number ES/P000745/1]. I am a member of a steering group of a campaign to improve health research (the declaration to improve health research). Patricia Logullo: I am a postdoctoral meta-researcher at the University of Oxford and a member of the UK EQUATOR Centre, an organisation that promotes the use of reporting guidelines, and I am personally involved in the development of some new reporting guidelines or their extensions. I receive funding from Cancer Research UK and NIHR Biomedical Research Centre for my research work. I am also a member of the Oxford-Brazil EBM Alliance, a not-for-profit organisation interested in disseminating evidence-based medicine principles (unpaid).

Jessica Butler: I am employed by the University of Aberdeen where I currently receive funding for medical research from the Health Foundation and from Wellcome Trust. I am an honorary analyst for NHS Grampian (unpaid). I am on the editorial board of Scientific Reports and Scientific Data (unpaid). I am a member of the UK Reproducibility Network and the Association of Professional Healthcare Analysts.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Bradley, S.H., DeVito, N.J., Lloyd, K.E. et al. Improving medical research in the United Kingdom. BMC Res Notes 15, 165 (2022). https://doi.org/10.1186/s13104-022-06050-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13104-022-06050-y

Keywords

  • Reproducibility
  • Research integrity
  • Research quality
  • Research waste
  • Medical research
  • Reporting biases