Research note | Open | Published:
The trials and tribulations of conducting an m-health pilot randomized controlled trial to improve oral cancer therapy adherence: recommendations for future multisite, non-drug clinical trials
BMC Research Notesvolume 12, Article number: 226 (2019)
Integrating mobile phone-based health (m-health) interventions into healthcare systems is one solution to improve access to services for the growing number of patients with chronic illness. Practical challenges such as poor recruitment and inadequate resource allocation can hamper the assessment of such interventions with clinical trial methodology. This paper highlights the challenges encountered during a pilot randomized controlled trial of an m-health medication adherence intervention and offers recommendations for future multi-site, non-drug clinical trials.
Eighteen patients were recruited to the study; eight were randomly allocated to the intervention arm. Intervention participants responded to their daily medication-reminder text messages, indicating that medication had been taken or not, and nurses were able to organize their calls around their workload. The trial closed prematurely primarily due to inadequate numbers of eligible patients; however, other potentially resolvable feasibility issues were identified. These included lack of infrastructure at study sites, poor screening data acquisition and management processes, and inexperience in conducting supportive care trials at participating sites. M-health intervention trials are designed to inform implementation of best supportive care practice. Adequate skills and infrastructure are research prerequisites that require careful consideration and sufficient investment for the successful execution of multi-site supportive care trials.
Trial registration Australian and New Zealand Clinical Trials Register: ACTRN12612000635864
Given the increasing number of people diagnosed with chronic disease [1, 2], the mounting pressure on health services and the fiscal restraints on health care , cost effective telehealth interventions integrated with targeted, direct clinical contact are urgently required . Mobile phone-based health (m-health) interventions have shown promise in improving oral drug adherence , however most trials have been methodologically flawed and none have yet been conducted in populations with cancer . Oral tyrosine kinase inhibitor therapy is the standard of care for patients with chronic phase myeloid leukemia (CML); however, continuous, daily dosing is required for an indefinite period, often lifelong, to ensure treatment efficacy [7, 8]. Thus, optimal medication adherence is critical for patients with CML, making this an important population in which to trial m-health interventions to improve medication adherence .
Therefore we developed the REMIND intervention package  to promote medication adherence and provide coaching in the self-management of side effects through mobile phone alerts, self-care advice, and nurse telephone consultations. REMIND consisted of two synergistically operating elements: nurse telephone consultations to promote medication adherence and provide coaching in adverse effect self-management; and a m-health system comprising individually-tailored mobile phone alerts, to which patients were to text responses indicating that medication had been taken or not, with tailored self-management advice based on self-reported, weekly symptom assessment and medication adherence.
To evaluate REMIND we commenced a pilot randomized-controlled trial. Eighteen patients were recruited before premature trial closure; the original target was 40 patients. This brief research note explores the feasibility issues encountered during the pilot testing of REMIND, as well as possible solutions.
This study was conducted across three cancer centres in Australia and received authorisation to begin from the respective Human Research Ethics Committees. A detailed description of participant selection criteria, outcome measures and data collection process are provided elsewhere . Consenting patients were randomised (1:1) using a computer-generated randomisation chart.
The recruitment target was 40 participants over approximately four months. However, after 12 months, only 18 participants had been recruited which prompted study discontinuation due to process and resource issues. Table 1 provides a summary of the problems encountered and suggested solutions.
Results and discussion
Despite the three participating sites initially confirming having sufficient CML patients to meet study timelines, most approached patients did not meet eligibility criteria. The overestimation of the number of patients meeting inclusion criteria, also known as Lasagna’s law [10, 11] is a very common phenomenon in clinical trials. To ensure adequate recruitment rates for medication adherence trials, patient inclusion criteria could be expanded to include other cancer types, other chronic disease types or clinical indications of unstable disease status. In addition, social networking could be employed to enable self-referral to trials targeting low-incidence diseases .
Inadequate infrastructure at some participating sites, and a lack of understanding among staff about the focus of supportive care interventions affected study conduct and data collection. Specifically, the study protocol required that patients be trained in the REMIND program by a study nurse at the beginning of the intervention. However, limited computer availability in the clinic area prevented some patients from being trained as per protocol at one site, resulting in patient non-adherence to the intervention.
Another issue was the poor maintenance of clinic screening logs which are designed to record the recruitment of a consecutive sample of patients to inform on the representativeness of participants within the clinical setting . Poor documentation of the screening and recruitment process significantly affected the quality of the information collected with only information from eligible participants’ being recorded. Without information about the total number of patients screened, ineligible, or declining participation, participants’ representativeness could not be established. Inadequate time and resource allocation may have prevented compliance with this process. Additionally, staff inexperience with supportive care studies may also have contributed to the problem. While they had extensive expertise in the management of clinical drug trials, some were unfamiliar with the purpose and priorities specific to supportive care trials, in particular aspects related to endpoints related to patient reported outcomes. From their experience with drug trial these outcomes were generally regarded as ‘an optional extra’. Specific training in the conduct of supportive care studies for clinical trial staff may be required and helpful to facilitate their conduct.
Pre-study (or initiation) visits to recruiting centres are conducted to assess suitability of infrastructure and staff availability. Additionally, regular site monitoring visits are important to ensure that trial conduct complies with protocols and good clinical practice (GCP) guidelines . The site-related issues outlined above may have been identified and addressed prior to trial initiation. However, budget restrictions only allowed for one site to receive an initiation visit, with no subsequent site monitoring visits. Therefore, the other two sites were initiated over the phone, and study monitoring was conducted over the phone and by emails. This method is not as effective as in-person visits, for which more time is allocated providing an opportunity to re-emphasise the importance of study processes, discuss study specificities, and resolve problems.
However, pre-study and regular site monitoring visits are costly , and adequate funding should be sought to conduct these visits. When necessary, study protocol and procedural compliance can be monitored through a combination of central and on-site monitoring . Just as appropriate expertise and investment is required to implement best supportive care practice , the same should be required for the conduct of these trials. It is critical that relevant GCP content is conveyed through staff training, particularly emphasising aspects of the study design or protocol which may be unfamiliar to staff members who are inexperienced with supportive care trials. This could entail training data managers from Clinical Trials units or collaborating with other hospital departments, such as psychology, who may be more familiar with supportive care research methods.
Tracking medication adherence
In conjunction with self-reported medication adherence, we attempted to objectively assess adherence via BCR-ABL blood levels, a routinely performed test to monitor therapeutic drug response. However, the timing of BCR-ABL blood testing seldom coincided with baseline testing, which sometimes occurred 10 weeks prior to study commencement. Therefore, molecular testing was not a feasible endpoint. Other objective measures that were trialled in REMIND were pill counts and prescription refills. However, patients often forgot to return their used and unused medication, packs and prescription refill. Patients were also not consistently collecting their medication from the hospital, making tracking of prescription refills and pill counts an impossible task. Future studies should consider investing in medication adherence tracking devices such as electronic pill monitoring. Additionally, analyses of population-based health databases such as health insurances for medication claims could also provide an objective measure of medication adherence .
Online data collection greatly reduces the costs of mailing out surveys, paying staff to remind patients to return completed surveys, and entering paper questionnaire data. However, even in this small study, four out of the 18 patients encountered problems using the online questionnaire when the support of a data manager was not available. Furthermore, the intervention patients had two sets of usernames and passwords; one set for the online data collection platform and another for the self-reported, weekly side-effects assessment, which caused confusion. The use of electronic surveys embedded on a single software application to capture and manage clinical data (e.g. REDCap ) would eliminate these problems for patients able to complete online surveys. Educating patients less familiar with online questionnaires through adequately trained staff members would keep costs to a minimum and reduce the risk of missing or incorrect data.
Clinical trials are crucial for the generation of evidence to effectively manage disease. Behavioural trials differ fundamentally from drug trials in the nature of their interventions, but the methodologies to establish efficacy and effectiveness are the same . Interventions in behavioural trials are usually more complex to define and standardise than those of drug trials. A main reason is the variation in participants’ characteristics and preferences, which will influence their behaviour (e.g. medication adherence), and have an important impact on the estimated effect of the intervention . Institutions and organisations conducting clinical trials usually have staff experienced in managing industry sponsored trials. Clinical trial staff are regularly trained in the good conduct of these studies, and are frequently monitored for adequate data collection and reporting. Considerable time and money are invested to ensure robust infrastructure and rigorous trial conduct [22, 23]. The same rigour is difficult to maintain in publicly funded studies with leaner budgets [20, 24]. Failing to provide adequate training may lead to under-reporting, and possibly study discontinuation [10, 25].
Despite the great potential that technology presents in delivering timely and cost-efficient supportive care programs for people requiring ongoing treatment, adequate resources to evaluate the efficacy of these intervention is imperative. If clinical trials units are to be allocated funding to conduct supportive care studies, careful evaluation of staff experience with, and infrastructure available to accommodate, supportive care studies is essential. It is important to continue to consider how to improve support for non-drug clinical trial research.
A limitation of this study is the small sample of nurses and patients used to pilot test this intervention. As CML is a relatively rare disease, it was difficult and time consuming to recruit patients. Future trials should consider assessing the impact of telehealth packages upon oral medication adherence in different cancer types.
Chronic myeloid leukemia
Good clinical practice
- M-health intervention:
Mobile phone-based health intervention
AIHW. Risk factors contributing to chronic disease. Canberra: Australian Institute of Health and Welfare; 2012. p. 157.
WHO. The world health report 2002: reducing risks, promoting healthy life. Geneva: World Health Organisation; 2002.
AIHW. 25 years of health expenditure in Australia 1989–1990 to 2013–2014. Canberra: Australian Institute of Health and Welfare; 2016. p. 66.
Schofield P, Chambers S. Effective, clinically feasible and sustainable: key design features of psycho-educational and supportive care interventions to promote individualised self-management in cancer care. Acta Oncol. 2015;54(5):805–12.
Haynes RB, Ackloo E, Sahota N, McDonald HP, Yao X. Interventions for enhancing medication adherence. Cochrane Database Syst Rev. 2008;2:CD000011.
Mistry N, Keepanasseril A, Wilczynski NL, Nieuwlaat R, Ravall M, Haynes RB. Technology-mediated interventions for enhancing medication adherence. J Am Med Inform Assoc. 2015;22:e177.
Baccarani M, Cortes J, Pane F, Niederwieser D, Saglio G, Apperley J, et al. Chronic myeloid leukemia: an update of concepts and management recommendations of European LeukemiaNet. J Clin Oncol. 2009;27(35):6041–51.
Welch M, Kaled E. Ensuring optimal adherence to BCR-ABL1 tyrosine kinase inhibitor therapy for chronic myeloid leukemia. Community Oncol. 2013;10(5):138–46.
Pereira-Salgado A, Westwood JA, Russell L, Ugalde A, Ortlepp B, Seymour JF, et al. Mobile health intervention to increase oral cancer therapy adherence in patients with chronic myeloid leukemia (The REMIND System): clinical feasibility and acceptability assessment. JMIR mHealth uHealth. 2017;5:12.
Van der Wouden JC, Blankenstein AH, Huibers MJ, van der Windt DA, Stalman WA, Verhagen AP. Survey among 78 studies showed that Lasagna's law holds in Dutch primary care research. J Clin Epidemiol. 2007;60(8):819–24.
Harris EL, Fitzgerald JD. The principles and practice of clinical trials: based on a symposium organised by the Association of Medical Advisers in the Pharmaceutical Industry. London: Churchill Livingstone; 1970.
Palac HL, Alam N, Kaiser SM, Ciolino JD, Lattie EG, Mohr DC. A practical do-it-yourself recruitment framework for concurrent ehealth clinical trials: simple architecture (Part 1). JMIR. 2018;20(11):e11050.
Schulz KF, Altman DG, Moher D, Group C. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMC Med. 2010;8:18.
Carter A, Landier W, Schad A, Moser A, Schaible A, Hanby C, et al. Successful coordination and execution of nontherapeutic studies in a cooperative group setting: lessons learned from Children's Oncology Group studies. Cancer Epidemiol Biomarkers Prev. 2008;17(7):1665–733.
Pronker E, Geerts BF, Cohen A, Pieterse H. Improving the quality of drug research or simply increasing its cost? An evidence-based study of the cost for data monitoring in clinical trials. Br J Clin Pharmacol. 2011;71(3):467–70.
Bakobaki JM, Rauchenberger M, Joffe N, McCormack S, Stenning S, Meredith S. The potential for central monitoring techniques to replace on-site monitoring: findings from an international multi-centre clinical trial. Clin Trials. 2012;9(2):257–64.
Bosnjak S. The importance of clinical practice guidelines (CPGs) for the quality and development of supportive care in Central and Eastern European (CEE) countries. Support Care Cancer. 2003;11(12):775–9.
Lam WY, Fresco P. Medication adherence measures: an overview. Biomed Res Int. 2015;2015:12.
Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–81.
Glasgow RE, Lichtenstein E, Marcus AC. Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. Am J Public Health. 2003;93(8):1261–7.
Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P. Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Ann Intern Med. 2008;148(4):295–309.
Djurisic S, Rath A, Gaber S, Garattini S, Bertele V, Ngwabyt S-N, et al. Barriers to the conduct of randomised clinical trials within all disease areas. Trials. 2017;18(1):360.
Shanley TP, Calvin-Naylor NA, Divecha R, Wartak MM, Blackwell K, Davis JM, et al. Enhancing Clinical Research Professionals’ Training and Qualifications (ECRPTQ): recommendations for Good Clinical Practice (GCP) training for investigators and study coordinators. J Clin Transl Sci. 2017;1(1):8–15.
Hudson KL, Lauer MS, Collins FS. Toward a new era of trust and transparency in clinical trials. JAMA. 2016;316(13):1353–4.
Chen R, Desai NR, Ross JS, Zhang W, Chau KH, Wayda B, et al. Publication and reporting of clinical trial results: cross sectional analysis across academic medical centers. BMJ. 2016;352:i637.
PS lead the study conception and design in conjunction with PB, JS, SA and KG. JS and PS coordinated the recruitment network. LR coordinated data collection, interpreted study outcomes and drafted the manuscript. MP substantively reviewed the paper. All authors read and approved the final manuscript.
The authors would like to thank Dr. Lawrence Cavedon, Dr. David Yeung, Dr. Tony Mills, Dr. Constantine Tam, Sue Kirsa, Andrew Dunlevie.
The authors declare that they have no competing interests.
Availability of data and materials
Consent for publication
Ethics approval and consent to participate
Written informed consent were obtained from all study participants. The study was approved by the Peter MacCallum Cancer Centre Human research Ethics Committee (HREC E22-10).
Funding was provided by Australiasian Lymphoma and Leukaemia Group.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.