- Regular article
- Open Access
Non-compliance with randomised allocation and missing outcome data in randomised controlled trials evaluating surgical interventions: a systematic review
BMC Research Notes volume 8, Article number: 403 (2015)
Randomised controlled trials are widely acknowledged as the gold standard in medical research although their validity can be undermined by non-compliance with the randomly allocated treatment and missing data. Due to the nature of the intervention, surgical trials face particular threat to compliance and data collection. For example, ineligibility for the intervention may only become apparent once the operation has commenced. It is unclear how such cases are reported and handled.
The objective was to assess non-compliance and missing data in reports of trials of surgical interventions.
Searches for reports of trials involving at least one surgical procedure and published in 2010 were carried out in the Medical Literature Analysis and Retrieval System Online (MEDLINE®). Data on missing data, non-compliance and methods of handling missing data were extracted from full texts. Descriptive data analyses were carried out on the data.
Forty-five (55 %) studies reported non-compliance with treatment allocation and 52 (63 %) reported primary outcome missing data. The median levels of non-compliance and missing data were 2 % [IQR (0, 5), range (0–29)] and 6 % [IQR (0, 15), range (0–57)], respectively. Fifty-two (63 %) studies analysed as randomised, 17 (21 %) analysed per protocol and 3 (4 %) analysed as treated. Complete case analysis was the most common method used to deal with missing data, 35/52 (67 %).
The reporting of non-compliance to allocation and the handling of missing data were typically suboptimal. There is still room for improvement on the use of the CONSORT statement particularly in accounting for study participants. Transparency in reporting would facilitate evidence synthesis.
A randomised controlled trial (RCT) is the optimal type of experimental design  for evaluating causal effect of treatments. High quality RCTs with very low risk of bias serve as first level evidence in evidence-based medicine . However, the theoretical advantages of the RCT design can be undermined by non-compliance to the treatment and group allocation and missing data, potentially introducing bias and therefore applying the findings becomes problematic. Over recent years there has been growing awareness of the importance of these issues and methodological development [3, 4]. Trials of surgical interventions have been criticised in part due to concerns about their ability to account for the complexity of surgery, the role and influence of the surgeon along with potentially strong patient preference and challenges in outcome measurement. Difficulties include achieving adequate sample size and potentially higher levels of non-compliance because participants are at liberty to seek their preferred treatment option outside the trial . Empirical evidence suggests that surgical expertise (in terms of both individual and surgical community learning curves) may cause deviation from allocated procedures [6–8].
Unlike pharmacological trials where temporary or permanent discontinuation of study drug constitutes non-compliance , surgical trials are particularly prone to other forms of non-compliance as a result of the challenges involved in their design and implementation. Causes of non-compliance and missing data in surgical trials include changes in treatment option at the surgeon’s discretion or for other clinical indications; failure of surgeons to adhere strictly to protocol by performing the same procedure but with modifications because of variable surgical expertise or in an attempt to tailor the intervention to the individual needs of the patient; change to the alternative surgical procedure if the surgeon is not in equipoise in terms of skill; failure to receive the allocated treatment because surgery was deemed unnecessary or impossible after gaining surgical access; refusal of surgery whereby a participant does not give consent for surgery but is otherwise happy to continue in the study; death before surgery; and withdrawal of consent for participation in the trial after randomisation but before surgery is received. Treatment changes may be to another treatment being evaluated within the trial or to treatments outwith the trial protocol. It is unclear how non-compliance and missing data are reported and accounted for in surgical trials. Randomisation must be accompanied by analysis of data that include unbiased measurements and ignorable missing values for attribution to be made ; therefore, inappropriate methods of handling missing data could lead to false conclusions. Different methods of handling missing data are employed in different situations and the effect of different approaches on the estimates should be considered because different methods can produce results that vary not only in magnitude but also in direction [11, 12]. The main aims of this study were to assess the level of non-compliance and missing data in surgical trials, and also to describe the methods used to account for this. Additionally, we aimed to show whether the level of these occurrences varied according to the type of surgical trial (see below for definitions).
Systematic searches for relevant surgical trials published in 2010 in the general medical (GMJs) and surgical journals (SJs) were carried out in MEDLINE® from inception till 31st March 2011 (see Additional file 1). Six GMJs (The British Medical Journal, Journal of the American Medical Association, The Lancet, New England Journal of Medicine, Archives of Internal Medicine and Annals of Internal Medicine) and 12 SJs (British Journal of Surgery, American Journal of Surgery, Annals of Surgery, Archives of Surgery, World Journal of Surgery, International Journal of Surgery, The Journal of Bone and Joint Surgery (American), The Journal of Bone and Joint Surgery (British), British Journal of Urology International, Journal of Urology, Obstetrics and Gynecology, American Journal of Obstetrics and Gynecology and British Journal of Obstetrics and Gynaecology International) were searched. Primary reports of two-arm randomised controlled trials involving at least one surgical procedure were included. Studies comparing the timing of administration of the same surgical procedure were excluded because the timing of intervention was not the unit of comparison in this review. Those comparing alternatives within the same participant (within-patient randomisation) were excluded because it would be impossible to assess differential attrition between groups. Secondary reports of the same study as well as pilot and feasibility studies were excluded because the aim was to assess primary reports on the same basis. Non-randomised studies, systematic reviews and non-English language papers were excluded.
Cook  defines surgical interventions as ‘those which involve physically changing body tissues and organs through manual operation such as cutting, abrading, suturing or the use of lasers’. Wente et al.  described two types of surgical trials: (1) those focusing on surgical procedures and (2) those focusing on other aspects of surgery such as anaesthesia or pharmacological and adjuvant treatments in surgical patients. The first group, which this review will be examining, can be further divided into four types of comparisons: (1) those comparing surgical procedure with placebo surgery, for example arthroscopic debridement versus sham surgery for osteoarthritis; (2) those comparing similar surgical procedures, for example use of reamed or unreamed nails for fixation of tibial fractures; (3) those comparing substantially different surgical procedures, for example open versus laparoscopic hernial repair; and (4) those comparing surgical with non-surgical management, for example surgery versus medicine in the treatment of gastroesophageal reflux disease or surgery versus physiotherapy for the management of anterior cruciate ligament injury.
The percentage of participants with non-compliance with treatment allocation and missing primary outcome data and the primary and secondary methods for handling missing data in the primary analysis of the primary outcome were determined for each study. For the purpose of this study, missing data included any primary outcome data that were not collected for randomised participants and any data excluded from analysis, for example, participants who were randomised and subsequently excluded due to ineligibility (post randomisation exclusions). Methods for handling missing data were determined either from the text or denominators in the tables and figures. The occurrence and level of non-compliance were deduced from the methods of analysis reported when these were not reported in the text. A definition of the terms related to the methods of analysis that were reviewed are presented in Table 1.
All titles and abstracts and full texts were screened by two independent reviewers. Data were double extracted by two independent reviewers using a data extraction form that was designed for the review and which had been piloted on 10 % of the included studies. Discrepancies were resolved by discussion or arbitration by the third reviewer. Two independent reviewers ran descriptive statistical analyses (mean, median, frequency and proportion) on the data using Stata 13 (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, TX: StataCorp LP). The review protocol was not registered and it is available from the authors.
Eighty-two studies were included from 131 full texts screened. Figure 1 outlines the identification of potentially eligible studies and the reasons for exclusion while Table 2 summarises their characteristics.
Description of included studies
Of 82 included studies, 49(60 %) compared similar surgical procedures [13–61], 17 compared substantially different procedures [62–78], 15 compared surgical and non-surgical management [79–93] and one compared surgery with placebo . Most reports were of single centre RCTs, the median number of participants recruited was 108 [IQR 65, 200].
Forty-five (55 %) studies reported non-compliance with treatment allocation; it was unclear in 11 (13 %) studies. In the 68 studies where the level of non-compliance could be derived (including 26 studies with zero non-compliance) the median percentage was 2 % [IQR (0, 5 %), range (0–29 %)], see Table 3. The reported analysis strategies were 52 (63 %) analysed as randomised, 17 (21 %) per-protocol and 3 (4 %) as-treated. The method of primary analysis was unclear in 10 (12 %) studies. Of the 31 studies which explicitly stated that the analysis was by intention-to-treat, 20 (65 %) included all participants in the analysis and analysed as randomised.
Fifty-two (63 %) studies reported having missing primary outcome data. In 7 (9 %) studies, it was unclear whether there were missing data. The median level of missing primary outcome data (including studies with zero missing data) was 6 % [IQR (0, 15), range (0–57)], see Table 3. The most common analysis strategy when missing data were present was complete case analysis, 35/52 (67 %). Methods used to account for missing data were survival type analysis to deal with censoring (4, 8 %); baseline or last observation carried forward (4, 8 %); imputing best case scenario (3, 6 %); and imputing best and worst case scenarios (2, 4 %). Multiple imputation and mixed models were used by 4 (8 %) studies (Table 4).
Non-compliance, missing data and types of surgical trials
Clear reporting of the occurrence or otherwise of non-compliance was observed in 40/49 (82 %), 14/17 (82 %) and 13/15 (87 %) comparisons between similar surgical procedures, substantially different surgical procedures and between surgical and non-surgical management, respectively. The median levels were 1 % [IQR (0, 3), range (0–3)], 4 % [IQR (1, 10), range (0–29)], and 4 % [IQR (0, 6), range (0–10)], respectively.
The level of missing data could be derived (including those with zero missing data) in 45/49(92 %), 15/17 (88 %) and 13/15 (87 %) comparisons between similar surgical procedures, substantially different surgical procedures and between surgical and non-surgical management, respectively. The medians were 5 % [IQR (0, 13), range (0–57)], 0 % [IQR (0, 14), range (0–27)], and 14 % [IQR (6, 17), range (0–46)], respectively.
The CONSORT statement  and the CONSORT statement extension for RCTs of non-pharmacological treatments  provide a means of ensuring transparency in reporting RCTs as well as non-pharmacological RCTs which have peculiar challenges in specific areas such as blinding, standardisation of interventions and care provider expertise.
This review shows that non-compliance and missing data commonly occur in surgical trials but the reported levels are typically low where reported. The occurrence and level of non-compliance were comparable among the three types of trials (82–87 %, 1–4 %). The occurrence of missing data was similar (88–92 %) across the types of surgical trials while the level varied from 0 % between substantially different surgical procedures, to 5 % between similar surgical procedures and 14 % between surgical and non-surgical management. In about a sixth of studies, the extent of reporting was not sufficient to allow assessment of the occurrence of non-compliance and missing data. A recent study  found an overall level of loss to follow-up (6 %, IQR 2–14 %) across studies of pharmacological versus surgery/invasive procedure versus other interventions which is similar to that found in our study.
It is well known that many cases of non-compliance result in missing data and this was also observed in the current study. We suspected that the occurrence of non-compliance and hence missing data would be higher when surgical procedures were compared with non-surgical interventions such as medicine, active surveillance and physiotherapy due to patient preference particularly when blinding is impractical or between substantially different surgical procedures such as open versus laparoscopic procedures which may be due to differences in surgical skill, experience and flexibility in unexpected situations such as rare anatomical findings. Our findings suggest that non-compliance is more likely in RCTs of surgical versus non-surgical management. We have not identified similar studies with which to compare our findings.
In most of the studies where it was impossible to ascertain whether a type of non-compliance or missing data occurred, this was because the authors had not fully accounted for all randomised participants and/or given clear reasons for non-compliance and/or exclusions from analysis. When the levels of non-compliance and missing data were unquantifiable, this was because the authors reported that there were missing data and non-compliance but we could not quantify the numbers with missing data or non-compliance because the authors had presented the results in percentages and either the number randomised into each group had not been reported or the number analysed was unclear. On occasion, it was observed that only the number of participants included in the analysis was reported while the numbers randomised into treatment groups were not reported. In addition, many authors reported the overall number of participants with non-compliance and/or missing data as a proportion of all participants in the study rather than as a proportion of the participants in each intervention group as stipulated by the CONSORT statement [96, 98].
Some non-compliance and missing data are almost inevitable, therefore it is important that investigators define adequately what the intervention is and specify reasons for non-compliance and address missing data. Reporting the timing of non-compliance or loss to follow-up was often ambiguous thus making it difficult to ascertain whether or not patients allocated to surgical interventions received the intervention or the period of time that patients who were allocated to medical interventions defaulted from treatment. Akl et al.  found that the accuracy of abstracted information varied from 84 to 100 % depending on the clarity of reporting in reports of RCTs of pharmacological versus surgery/invasive procedure versus other interventions. In order for readers not to misjudge poor reporting as poor methodology, clarity and transparency in reporting is essential. Furthermore, the reason for missingness is important in choosing an appropriate method for handling missing data. When data are missing completely at random (MCAR), for example, if a participant moves abroad and outcome data cannot be collected, the reason for missingness is not influenced by the prognosis or the study, neither is it related to the missing value itself  and as such, the difference between the observed data and the hypothetically complete data is random and exclusion of missing data would not introduce bias but there will be loss of power [99, 100]. When data are missing at random (MAR), the difference between the observed data and the hypothetically complete data is systematic. The reason for missingness on that variable cannot be explained by the missing value itself but rather by the information available on other variables  and as such, deletion of missing data would not be justified. For example, if it is known that older participants tend to ignore a certain question in a survey, one could explain that the reason for missingness is the age of the participant but this reason is non-informative or ignorable because it still does not explain why older participants ignore the question. In such situations, there are methods that could handle the missing data and a model is not required . Missing not completely at random (MNAR) is when data is missing in a systematic fashion, the probability of missingness depends on the missing value itself [99, 100] and the reason for missingness is informative or non-ignorable because it is related to the prognosis. In this situation a model is required to explain the missingness and handle it in the analysis . Authors need to investigate and report reasons for missingness so as to justify the appropriateness of the methods of handling missing data as well as the possible implications missing data may have on the findings.
There appears to be a lack of consistency in the terminologies used in reporting missing data and non-compliance. However, a better understanding of the purpose of the CONSORT flow chart and strict adherence to the guidance template would definitely give some clarity in the numbers randomised, treated, followed up and analysed; this finding is supported by previous studies [97, 101, 102]. Akl et al.  showed that 87 % of reviewed studies explicitly reported information on loss to follow-up or presented it in a CONSORT flow diagram or both. The current review found that 91 % of studies reported the occurrence or otherwise of missing data explicitly, using a CONSORT flow chart or the reviewers obtained relevant information by calculating denominators from raw figures and percentages in result tables.
About two-thirds of reviewed studies analysed as randomised while some performed per-protocol analysis and few analysed as treated. Exclusion of data could introduce bias or give an estimate that is applicable to the sub-population of participants that completed the study. The choice between intention-to-treat (ITT) and per-protocol depends on the objective of the study, which is whether it sets out to establish equivalence of two interventions or the superiority of one over the other. In order to avoid presenting a harmful or equivalent as harmless, equivalent or superior, a per-protocol analysis is more appropriate as an ITT analysis is conservative. In addition, authors may use a per-protocol analysis to handle non-compliance  but an ITT analysis is the standard analysis and should be presented as well . Of 17 studies that reported per-protocol analysis only, only one  was an equivalence trial and the authors also presented an ITT analysis. Previous studies [102, 106] demonstrated that reporting of an ITT analysis was far more common (48, 83 %) than its use (13, 22 %) while the current review showed that more authors are beginning to use the term properly as 65 % of studies reporting an ITT analysis were found to have analysed all randomised participants as-randomised.
The vast majority of studies with missing data used simple methods of handling missing data such as complete case analysis and simple imputation despite the widespread availability of more robust methods. While levels of missing data were generally low, some studies had large levels for which the use of more complex methods could be particularly useful.
This review is limited by the sample size and the limited range of sources of the included studies and as such our findings may not be generalizable to all journals. In addition, the extent to which compliance and missing data could be evaluated was dependent upon the standard of reporting, which was suboptimal.
Recommendations for authors
Based on our findings, we recommend that authors should account for all participants who were assessed for eligibility at every stage of the trial and the numbers should also be clearly documented in the text as well as in the CONSORT flow chart. Authors should endeavour to investigate the reasons for missing data and non-compliance with treatment allocation for all randomised participants and report these reasons with the number of participants in the text as well as in the CONSORT flow chart. It is crucial that this is done per treatment arm. Authors should also report the time in the course of the trial when the missing data occurred and indicate this at the appropriate stage in the CONSORT flow chart. Finally, the analysis strategy should be described clearly rather than described with terms such as “intention-to-treat” which could vary in meaning.
Non-compliance and missing data commonly occur in surgical trials but typically low levels. Reporting of non-compliance and missing data is sub-optimal and there is room for improvement in reporting standards. Authors should be encouraged to use more robust methods of handling missing data. Lack of transparency in reporting may hinder users of research findings from making valid decisions and could raise questions about the integrity of a trial.
CONsolidated Standards Of Reporting Trials
General Medical Journal
Medical Literature Analysis and Retrieval System Online
randomised controlled trial
Stolberg HO, Norman G, Trop I. Fundamentals of clinical research for radiologist: randomised controlled trials. AJR. 2004;183:1539–44.
Harbour R, Miller J. A new system for grading recommendations in evidence based guidelines. BMJ. 2001;323(7308):334–6.
White I. Uses and limitations of randomization-based efficacy estimators. Stat Methods Med Res. 2005;14:327–47.
White I, Horton N, Carpenter J, Pocock S. Strategy for intention to treat analysis in randomised trials with missing outcome data. BMJ. 2011;342:d40. doi:10.1136/bmj.d40.
Ergina PL, Cook JA, Blazeby JM, Boutron I, Clavien PA, Reeves BC, et al. Challenges in evaluating surgical innovation. Lancet. 2009;374:1097–104.
Cook JA. The challenges faced in the design, conduct and analysis of surgical randomised controlled trials. Trials. 2009;10:9. doi:10.1186/1745-6215-10-9.
SPRINT Investigators. Randomized trial of reamed and unreamed intramedullary nailing of tibial shaft fractures. J Bone Joint Surg Am. 2008;90:2567–78.
Wente M, Seiler C, Uhl W, Bchler M. Perspectives of evidence-based surgery. Dig Surg. 2003;20(4):263–9.
Snapinn SM, Jiang Q, Iglewicz B. Informative non-compliance in endpoint trials. Curr Control Trials Cardiovasc Med. 2004;5(1):5.
Lachin JM. Statistical considerations in the intent-to-treat principle. Control Clin Trials. 2000;21(3):167–89.
Montori VM, Guyatt GH. Intention-to-treat principle. CMAJ. 2001;165(10):1339–41.
Wittes J. Missing inaction: preventing missing outcome data in randomized clinical trials. J Biopharm Stat. 2009;19(6):957–68.
Abo-hashem AA, Sarhan A, Aly AM. Harmonic scalpel compared with bipolar electro-cautery hemorrhoidectomy: a randomized controlled trial. Int J Surg. 2010;8(3):243–7.
Barton TM, Gleeson R, Topliss C, Greenwood R, Harries WJ, Chesser TJ. A comparison of the long gamma nail with the sliding hip screw for the treatment of AO/OTA 31-A2 fractures of the proximal part of the femur: a prospective randomized trial. J Bone Joint Surg Am. 2010;92(4):792–8.
Basha SL, Rochon ML, Quinones JN, Coassolo KM, Rust OA, Smulian JC. Randomized controlled trial of wound complication rates of subcuticular suture vs staples for skin closure at cesarean delivery. Am J Obstet Gynecol. 2010;203(3):285.e1–e8. doi:10.1016/j.ajog.2010.07.011.
Basu M, Duckett J. A randomised trial of a retropubic tension-free vaginal tape versus a mini-sling for stress incontinence. BJOG. 2010;117(6):730–5.
Bevis PM, Windhaber RA, Lear PA, Poskitt KR, Earnshaw JJ, Mitchell DC. Randomized clinical trial of mesh versus sutured wound closure after open abdominal aortic aneurysm surgery. Br J Surg. 2010;97(10):1497–502.
Botker HE, Kharbanda R, Schmidt MR, Bottcher M, Kaltoft AK, Terkelsen CJ, et al. Remote ischaemic conditioning before hospital admission, as a complement to angioplasty, and effect on myocardial salvage in patients with acute myocardial infarction: a randomised trial. Lancet. 2010;375(9716):727–34.
Can MF, Sevinc MM, Hancerliogullari O, Yilmaz M, Yagci G. Multicenter prospective randomized trial comparing modified Limberg flap transposition and Karydakis flap reconstruction in patients with sacrococcygeal pilonidal disease. Am J Surg. 2010;200(3):318–27.
Chalmers RT, Darling Iii RC, Wingard JT, Chetter I, Cutler B, Kern JA, et al. Randomized clinical trial of tranexamic acid-free fibrin sealant during vascular surgical procedures. Br J Surg. 2010;97(12):1784–9.
Chen Q, Zhang L, Fan QL, Zhou J, Peng YB, Wang Z. Bipolar transurethral resection in saline vs traditional monopolar resection of the prostate: results of a randomized trial with a 2-year follow-up. BJU Int. 2010;106(9):1339–43.
Choi WC, Lee S, Seong SC, Jung JH, Lee MC. Comparison between standard and high-flexion posterior-stabilized rotating-platform mobile-bearing total knee arthroplasties: a randomized controlled study. J Bone Joint Surg Am. 2010;92(16):2634–42.
Davila-Perez R, Bracho-Blanchet E, Tovilla-Mercado JM, Hernandez-Plata JA, Reyes-Lopez A, Nieto-Zermeno J. Unnecessary gastric decompression in distal elective bowel anastomoses in children: a randomized study. World J Surg. 2010;34(5):947–53.
Degiuli M, Sasako M, Ponti A, Italian Gastric Cancer Study G. Morbidity and mortality in the Italian Gastric Cancer Study Group randomized clinical trial of D1 versus D2 resection for gastric cancer. Br J Surg. 2010;97(5):643–9.
Demey G, Servien E, Pinaroli A, Lustig S, Ait Si Selmi T, Neyret P. The influence of femoral cementing on perioperative blood loss in total knee arthroplasty: a prospective randomized study. J Bone Joint Surg Am. 2010;92(3):536–41.
Farrell SA, Gilmour D, Turnbull GK, Schmidt MH, Baskett TF, Flowerdew G, et al. Overlapping compared with end-to-end repair of third- and fourth-degree obstetric anal sphincter tears: a randomized controlled trial. Obstet Gynecol. 2010;116(1):16–24.
Garcia-Pagan JC, Caca K, Bureau C, Laleman W, Appenrodt B, Luca A, Abraldes JG, Nevens F, Vinel JP, Mossner J, Bosch J, Early TIPS, (Transjugular Intrahepatic Portosystemic Shunt) Cooperative Study Group. Early use of TIPS in patients with cirrhosis and variceal bleeding. N Engl J Med. 2010;362(25):2370–9.
Geavlete B, Multescu R, Dragutescu M, Jecu M, Georgescu D, Geavlete P. Transurethral resection (TUR) in saline plasma vaporization of the prostate vs standard TUR of the prostate: ‘the better choice’ in benign prostatic hyperplasia? BJU Int. 2010;106(11):1695–9.
Gong Y, Xu J, Shao J, Cheng H, Wu X, Zhao D, et al. Prevention of seroma formation after mastectomy and axillary dissection by lymph vessel ligation and dead space closure: a randomized trial. Am J Surg. 2010;200(3):352–6.
Haskal ZJ, Trerotola S, Dolmatch B, Schuman E, Altman S, Mietling S, et al. Stent graft versus balloon angioplasty for failing dialysis-access grafts. N Engl J Med. 2010;362(6):494–503.
Hay GC, Kampshoff J, Kuster MS. Lateral subvastus approach with osteotomy of the tibial tubercle for total knee replacement: a two-year prospective, randomised, blinded controlled trial. J Bone Joint Surg Br. 2010;92(6):862–6.
Hove LM, Krukhaug Y, Revheim K, Helland P, Finsen V. Dynamic compared with static external fixation of unstable fractures of the distal part of the radius: a prospective, randomized multicenter study. J Bone Joint Surg Am. 2010;92(8):1687–96.
Iglesia CB, Sokol AI, Sokol ER, Kudish BI, Gutman RE, Peterson JL, et al. Vaginal mesh for prolapse: a randomized controlled trial. Obstet Gynecol. 2010;116(2 Pt 1):293–303.
Khafagy W, Omar W, El Nakeeb A, Fouda E, Yousef M, Farid M. Treatment of anal fistulas by partial rectal wall advancement flap or mucosal advancement flap: a prospective randomized study. Int J Surg. 2010;8(4):321–5.
Lambert EH, Walsh R, Moreno MW, Gupta M. Effect of escalating versus fixed voltage treatment on stone comminution and renal injury during extracorporeal shock wave lithotripsy: a prospective randomized trial. J Urol. 2010;183(2):580–4.
Lee PC, Lo C, Lai PS, Chang JJ, Huang SJ, Lin MT, et al. Randomized clinical trial of single-incision laparoscopic cholecystectomy versus minilaparoscopic cholecystectomy. Br J Surg. 2010;97(7):1007–12.
Li JP, Chu JP, Oh P, Li Z, Chen W, Huang YH, et al. Characterizing clinicopathological findings of transarterial chemoembolization for Wilms tumor. J Urol. 2010;183(3):1138–44.
McCalden RW, Charron KD, Yuan X, Bourne RB, Naudie DD, MacDonald SJ. Randomised controlled trial comparing early migration of two collarless polished cemented stems using radiostereometric analysis. J Bone Joint Surg Br. 2010;92(7):935–40.
Mishra S, Sabnis RB, Kurien A, Ganpule A, Muthu V, Desai M. Questioning the wisdom of tubeless percutaneous nephrolithotomy (PCNL): a prospective randomized controlled study of early tube removal vs tubeless PCNL. BJU Int. 2010;106(7):1045–8.
Muzi MG, Milito G, Cadeddu F, Nigro C, Andreoli F, Amabile D, et al. Randomized comparison of Limberg flap versus modified primary closure for the treatment of pilonidal disease. Am J Surg. 2010;200(1):9–14.
Nystrom PO, Qvist N, Raahave D, Lindsey I, Mortensen N, Stapled or Open Pile Procedure (STOPP) trial study group. Randomized clinical trial of symptom control after stapled anopexy or diathermy excision for haemorrhoid prolapse. Br J Surg. 2010;97(2):167–76.
Ohye RG, Sleeper LA, Mahony L, Newburger JW, Pearson GD, Lu M, et al. Comparison of shunt types in the Norwood procedure for single-ventricle lesions. N Engl J Med. 2010;362(21):1980–92.
Omonbude D, El Masry MA, O’Connor PJ, Grainger AJ, Allgar VL, Calder SJ. Measurement of joint effusion and haematoma formation by ultrasound in assessing the effectiveness of drains after total knee replacement: a prospective randomised study. J Bone Joint Surg Br. 2010;92(1):51–5.
Parker MI, Pryor G, Gurusamy K. Cemented versus uncemented hemiarthroplasty for intracapsular hip fractures: a randomised controlled trial in 400 patients. J Bone Joint Surg Br. 2010;92(1):116–22.
Penninx JP, Mol BW, Engels R, van Rumste MM, Kleijn C, Koks CA, et al. Bipolar radiofrequency endometrial ablation compared with hydrothermablation for dysfunctional uterine bleeding: a randomized controlled trial. Obstet Gynecol. 2010;116(4):819–26.
Perera D, Stables R, Thomas M, Booth J, Pitt M, Blackman D, et al. Elective intra-aortic balloon counterpulsation during high-risk percutaneous coronary intervention: a randomized controlled trial. JAMA. 2010;304(8):867–74.
Persson P, Brynhildsen J, Kjolhede P, Hysterectomy Multicentre Study Group in South-East, Sweden. Short-term recovery after subtotal and total abdominal hysterectomy: a randomised clinical trial. BJOG. 2010;117(4):469–78.
Rane A, Bradoo A, Rao P, Shivde S, Elhilali M, Anidjar M, et al. The use of a novel reverse thermosensitive polymer to prevent ureteral stone retropulsion during intracorporeal lithotripsy: a randomized, controlled trial. J Urol. 2010;183(4):1417–21.
Reid K, Pockney P, Pollitt T, Draganic B, Smith SR. Randomized clinical trial of short-term outcomes following purse-string versus conventional closure of ileostomy wounds. Br J Surg. 2010;97(10):1511–7.
Richter HE, Albo ME, Zyczynski HM, Kenton K, Norton PA, Sirls LT, et al. Retropubic versus transobturator midurethral slings for stress incontinence. N Engl J Med. 2010;362(22):2066–76.
Rogers SJ, Cello JP, Horn JK, Siperstein AE, Schecter WP, Campbell AR, et al. Prospective randomized trial of LC + LCBDE vs ERCP/S + LC for common bile duct stone disease. Arch Surg. 2010;145(1):28–33.
Sari S, Erbil Y, Sumer A, Agcaoglu O, Bayraktar A, Issever H, et al. Evaluation of recurrent laryngeal nerve monitoring in thyroid surgery. Int J Surg. 2010;8(6):474–8.
Schatloff O, Lindner U, Ramon J, Winkler HZ. Randomized trial of stone fragment active retrieval versus spontaneous passage during holmium laser lithotripsy for ureteral stones. J Urol. 2010;183(3):1031–5.
Shepherd AC, Gohel MS, Brown LC, Metcalfe MJ, Hamish M, Davies AH. Randomized clinical trial of VNUS ClosureFAST radiofrequency ablation versus laser for varicose veins. Br J Surg. 2010;97(6):810–8.
Simpson DJ, Kendrick BJ, Hughes M, Glyn-Jones S, Gill HS, Rushforth GF, et al. The migration patterns of two versions of the Furlong cementless femoral stem: a randomised, controlled trial using radiostereometric analysis. J Bone Joint Surgery Br. 2010;92(10):1356–62.
Smolders JM, Hol A, Rijnders T, van Susante JL. Changes in bone mineral density in the proximal femur after hip resurfacing and uncemented total hip replacement: a prospective randomised controlled study. J Bone Joint Surg Br. 2010;92(11):1509–14.
Sovik TT, Taha O, Aasheim ET, Engstrom M, Kristinsson J, Bjorkman S, et al. Randomized clinical trial of laparoscopic gastric bypass versus laparoscopic duodenal switch for superobesity. Br J Surg. 2010;97(2):160–6.
Stassano P, Di Tommaso L, Monaco M, Mastrogiovanni G, Musumeci A, Contaldo A, et al. Left heart pump-assisted myocardial revascularization favorably affects neutrophil apoptosis. World J Surg. 2010;34(4):652–7.
Subramonia S, Lees T. Randomized clinical trial of radiofrequency ablation or conventional high ligation and stripping for great saphenous varicose veins. Br J Surg. 2010;97(3):328–36.
Tani M, Kawai M, Hirono S, Ina S, Miyazawa M, Shimizu A, et al. A prospective randomized controlled trial of internal versus external drainage with pancreaticojejunostomy for pancreaticoduodenectomy. Am J Surg. 2010;199(6):759–64.
van der Gaag NA, Rauws EA, van Eijck CH, Bruno MJ, van der Harst E, Kubben FJ, et al. Preoperative biliary drainage for cancer of the head of the pancreas. N Engl J Med. 2010;362(2):129–37.
Basiri A, Zare S, Tabibi A, Sharifiaghdas F, Aminsharifi A, Mousavi-Bahar SH, et al. A multicenter, randomized, controlled trial of transureteral and shock wave lithotripsy—which is the best minimally invasive modality to treat distal ureteral calculi in children? J Urol. 2010;184(3):1106–9.
Brott TG, Hobson RW 2nd, Howard G, Roubin GS, Clark WM, Brooks W, et al. Stenting versus endarterectomy for treatment of carotid-artery stenosis. N Engl J Med. 2010;363(1):11–23 (Erratum appears in N Engl J Med. 2010;363(2):198).
Darai E, Dubernard G, Coutant C, Frey C, Rouzier R, Ballester M. Randomized trial of laparoscopically assisted versus open colorectal resection for endometriosis: morbidity, symptoms, quality of life, and fertility. Ann Surg. 2010;251(6):1018–23.
Gervaz P, Inan I, Perneger T, Schiffer E, Morel P. A prospective, randomized, single-blind comparison of laparoscopic versus open sigmoid colectomy for diverticulitis. Ann Surg. 2010;252(1):3–8.
Ghezzi F, Uccella S, Cromi A, Siesto G, Serati M, Bogani G, et al. Postoperative pain after laparoscopic and vaginal hysterectomy for benign gynecologic disease: a randomized trial. Am J Obstet Gynecol. 2010;203(2):118.e1–8. doi:10.1016/j.ajog.2010.04.026.
Guo JJ, Tang N, Yang HL, Tang TS. A prospective, randomised trial comparing closed intramedullary nailing with percutaneous plating in the treatment of distal metaphyseal fractures of the tibia. J Bone Joint Surg Br. 2010;92(7):984–8.
Hessman O, Westerdahl J, Al-Suliman N, Christiansen P, Hellman P, Bergenfelz A. Randomized clinical trial comparing open with video-assisted minimally invasive parathyroid surgery for primary hyperparathyroidism. Br J Surg. 2010;97(2):177–84.
Huang J, Yan L, Cheng Z, Wu H, Du L, Wang J, et al. A randomized trial comparing radiofrequency ablation and surgical resection for HCC conforming to the Milan criteria. Ann Surg. 2010;252(6):903–12.
Itani KM, Hur K, Kim LT, Anthony T, Berger DH, Reda D, et al. Comparison of laparoscopic and open repair with mesh for the treatment of ventral incisional hernia: a randomized trial. Arch Surg. 2010;145(4):322–8.
Langeveld HR, van’t Riet M, Weidema WF, Stassen LP, Steyerberg EW, Lange J, et al. Total extraperitoneal inguinal hernia repair compared with Lichtenstein (the LEVEL-Trial): a randomized controlled trial. Ann Surg. 2010;251(5):819–24.
Naik R, Jackson KS, Lopes A, Cross P, Henry JA. Laparoscopic assisted radical vaginal hysterectomy versus radical abdominal hysterectomy—a randomised phase II trial: perioperative outcomes and surgicopathological measurements. BJOG. 2010;117(6):746–51.
Nicholson ML, Kaushik M, Lewis GR, Brook NR, Bagul A, Kay MD, et al. Randomized clinical trial of laparoscopic versus open donor nephrectomy. Br J Surg. 2010;97(1):21–8.
Ozmen M, Zulfikaroglu B, Ozalp N, Moran M, Soydinc P, Ziraman I. Femoral vessel blood flow dynamics following totally extraperitoneal vs Stoppa procedure in bilateral inguinal hernias. Am J Surg. 2010;199(6):741–5.
Penn HA, Gatti JM, Hoestje SM, DeMarco RT, Snyder CL, Murphy JP. Laparoscopic versus open pyeloplasty in children: preliminary report of a prospective randomized trial. J Urol. 2010;184(2):690–5.
Pospischill M, Kranzl A, Attwenger B, Knahr K. Minimally invasive compared with traditional transgluteal approach for total hip arthroplasty: a comparative gait analysis. J Bone Joint Surg Am. 2010;92(2):328–37.
van Santvoort HC, Besselink MG, Bakker OJ, Hofker HS, Boermeester MA, Dejong CH, et al. A step-up approach or open necrosectomy for necrotizing pancreatitis. N Engl J Med. 2010;362(16):1491–502.
Wulker N, Lambermont JP, Sacchetti L, Lazaro JG, Nardi J. A prospective randomized study of minimally invasive total knee arthroplasty compared with conventional surgery. J Bone Joint Surg Am. 2010;92(7):1584–90.
Frobell RB, Roos EM, Roos HP, Ranstam J, Lohmander LS. A randomized trial of treatment for acute anterior cruciate ligament tears. N Engl J Med. 2010;363(4):331–42.
Kelly T, Suddes J, Howel D, Hewison J, Robson S. Comparing medical versus surgical termination of pregnancy at 13–20 weeks of gestation: a randomised controlled trial. BJOG. 2010;117(12):1512–20.
Klazen CA, Lohle PN, de Vries J, Jansen FH, Tielbeek AV, Blonk MC, et al. Vertebroplasty versus conservative treatment in acute osteoporotic vertebral compression fractures (Vertos II): an open-label randomised trial. Lancet. 2010;376(9746):1085–92.
Kuck KH, Schaumann A, Eckardt L, Willems S, Ventura R, Delacretaz E, et al. Catheter ablation of stable ventricular tachycardia before defibrillator implantation in patients with coronary heart disease (VTACH): a multicentre randomised controlled trial. Lancet. 2010;375(9708):31–40.
Leon MB, Smith CR, Mack M, Miller DC, Moses JW, Svensson LG, et al. Transcatheter aortic-valve implantation for aortic stenosis in patients who cannot undergo surgery. N Engl J Med. 2010;363(17):1597–607.
Moosmayer S, Lund G, Seljom U, Svege I, Hennig T, Tariq R, et al. Comparison between surgery and physiotherapy in the treatment of small and medium-sized tears of the rotator cuff: A randomised controlled study of 103 patients with one-year follow-up. J Bone Joint Surg Br. 2010;92(1):83–91.
Nasr M, Ezzat H, Elsebae M. Botulinum toxin injection versus lateral internal sphincterotomy in the treatment of chronic anal fissure: a randomized controlled trial. World J Surg. 2010;34(11):2730–4.
O’Brien PE, Sawyer SM, Laurie C, Brown WA, Skinner S, Veit F, et al. Laparoscopic adjustable gastric banding in severely obese adolescents: a randomized trial. JAMA. 2010;303(6):519–26.
Rompe JD, Cacchio A, Weil L Jr, Furia JP, Haist J, Reiners V, et al. Plantar fascia-specific stretching versus radial shock-wave therapy as initial treatment of plantar fasciopathy. J Bone Joint Surg Am. 2010;92(15):2514–22.
Schouten R, Rijs CS, Bouvy ND, Hameeteman W, Koek GH, Janssen IM, et al. A multicenter, randomized efficacy study of the EndoBarrier Gastrointestinal Liner for presurgical weight loss prior to bariatric surgery. Ann Surg. 2010;251(2):236–43.
Sciurba FC, Ernst A, Herth FJ, Strange C, Criner GJ, Marquette CH, et al. A randomized study of endobronchial valves for advanced emphysema. N Engl J Med. 2010;363(13):1233–44.
Symplicity HTN-2 I, Esler MD, Krum H, Sobotka PA, Schlaich MP, Schmieder RE, et al. Renal sympathetic denervation in patients with treatment-resistant hypertension (The Symplicity HTN-2 Trial): a randomised controlled trial. Lancet. 2010;376(9756):1903–9.
Verze P, Imbimbo C, Cancelmo G, Creta M, Palmieri A, Mangiapia F, et al. Extracorporeal shockwave lithotripsy vs ureteroscopy as first-line therapy for patients with single, distal ureteric stones: a prospective randomized study. BJU Int. 2010;106(11):1748–52.
Wilber DJ, Pappone C, Neuzil P, De Paola A, Marchlinski F, Natale A, et al. Comparison of antiarrhythmic drug therapy and radiofrequency catheter ablation in patients with paroxysmal atrial fibrillation: a randomized controlled trial. JAMA. 2010;303(4):333–40.
Willits K, Amendola A, Bryant D, Mohtadi NG, Giffin JR, Fowler P, et al. Operative versus nonoperative treatment of acute Achilles tendon ruptures: a multicenter randomized trial using accelerated functional rehabilitation. J Bone Joint Surg Am. 2010;92(17):2767–75.
Chitale S, Morsey M, Swift L, Sethia K. Limited shock wave therapy vs sham treatment in men with Peyronie’s disease: results of a prospective randomized controlled double-blind trial. BJU Int. 2010;106(9):1352–6.
Altman D, Schulz K, Moher D, Egger M, Davidoff F, Elbourne D. The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Int Med. 2001;134:663–94.
Boutron I, Moher D, Altman DG, Schulz K, Ravaud P, for the CONSORT group. Extending the CONSORT Statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Ann Int Med. 2008;148(4):295–309.
Akl E, Briel M, You J, Sun X, Johnston B, Busse J, et al. Potential impact on estimated treatment effects of information lost to follow-up in randomised controlled trials (LOST-IT): systematic review. BMJ. 2012;344:e2809. doi:10.1136/bmj.e2809.
Moher D, Hopewell S, Schulz K, Montori V, Gøtzsche P, Devereaux P, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869. doi:10.1136/bmj.c869.
Baraldi AN, Enders CK. An introduction to modern missing data analyses. J Sch Psychol. 2010;48:5–37.
Howell DC. The analysis of missing data. In: Outhwaite W, Turner S, editors. Handbook of social science methodology. London: Sage; 2007. p. 1–44.
Sinha S, Sinha S, Ashby E, Jayaram R, Grocott MPW. Quality of reporting in randomized trials published in high-quality surgical journals. J Am Coll Surg. 2009;209:565–71.
Dodd S, White I, Williamson P. Nonadherence to treatment proto col in published randomised controlled trials: a review. Trials 2012;13:84. doi:10.1186/1745-6215-13-84.
Little RJ, Long Q, Lin X. A comparison of methods for estimating the causal effect of a treatment in randomized clinical trials subject to noncompliance. Biometrics. 2008;65(2):640–9.
Snapinn S. Noninferiority trials. Curr Control Trials Cardiovasc Med. 2000;1(1):19–21.
Richter HE, Albo ME, Zyczynski HM, Kenton K, Norton PA, Sirls LT, et al. Retropubic versus transobturator midurethral slings for stress incontinence. N Engl J Med. 2010;362(22):2066–76.
Hollis S, Campbell F. What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ. 1999;319(7211):670–4.
TEA was the primary reviewer and coordinated all aspects of the study including data extraction, analysis and wrote the manuscript. GMAC extracted and analysed data as a second reviewer, and provided critical comments on the manuscript. JAC extracted data also as a second reviewer, acted as an arbiter when there were discrepancies, provided expert advice on methodological challenges in conducting surgical trials and provided critical comments on the manuscript. All authors read and approved the final manuscript.
JAC held MRC training (reference number: G0601938) and methodology (reference number: G1002292) fellowships while this research was undertaken. The Health Services Research Unit is core funded by the Chief Scientist Office of the Scottish Government Health and Social Care Directorates. The authors accept full responsibility for this manuscript.
Compliance with ethical guidelines
Competing interests The authors declare that they have no competing interests.
About this article
Cite this article
Adewuyi, T.E., MacLennan, G. & Cook, J.A. Non-compliance with randomised allocation and missing outcome data in randomised controlled trials evaluating surgical interventions: a systematic review. BMC Res Notes 8, 403 (2015). https://doi.org/10.1186/s13104-015-1364-9
- Missing data
- Surgical trials