- Research note
- Open Access
The McMaster Toronto Arthritis patient preference questionnaire (MACTAR): a methodological study of reliability and minimal detectable change after a 6 week-period of acupuncture treatment in patients with rheumatoid arthritis
BMC Research Notesvolume 10, Article number: 687 (2017)
The McMaster Toronto Arthritis patient preference questionnaire (MACTAR) is a semi-structured interview consisting of a baseline and a follow-up interview. The MACTAR baseline is reliable and valid, however the reliability of the MACTAR follow-up is scarcely described. The aim of this study was to describe aspects of reliability and ability to detect changes of the Swedish MACTAR follow-up following acupuncture treatment in individuals with rheumatoid arthritis.
The study was of Single Subject Experimental Design, with a 2-week non-interventional A-phase and a 6-week intervention B-phase. Eight individuals with RA, age 30–68 years, were included. MACTAR baseline was performed once followed by five assessments with MACTAR follow-up during the A-phase and another ten assessments during the B-phase. Reliability statistics were calculated for measurements 1–3 during the A-phase and the ability to detect effects of acupuncture treatment was tested by celeration lines in the B-phase. The MACTAR follow-up was highly reliable (ICC = 0.7–0.9, SEM = 2.3–4.3, and SDD = 6.2–11.7). Visual and statistical analyses indicated that the MACTAR follow-up could detect effects on individual- and group levels after acupuncture treatment, indicating that the MACTAR follow-up seems to be reliable and is able to detect effects of acupuncture treatment in RA.
Rheumatoid arthritis (RA) is an inflammatory rheumatic condition with polyarthritis leading to pain, swollen and stiff joints, fatigue, and disability [1,2,3]. Acupuncture might reduce pain and reduce inflammation in patients with RA .
Reliable and valid clinical outcome measures are a prerequisite for assessment of outcome and effects of treatments. Several patient-reported outcome measures (PROMs) are developed and/or validated for patients with RA, but very few focus on patient preference . PROMs with pre-defined questions/items might not be relevant for all individuals with RA [6,7,8]. A patient preference instrument could be more sensitive to detect changes than recommended PROMs [6, 9].
The objective of this study was to establish the reliability of the Swedish McMaster Toronto Arthritis patient preference questionnaire (MACTAR) follow-up interview and to describe its ability to detect changes after a 6 week-period of acupuncture treatment in patients with RA.
The MACTAR was the first patient preference instrument developed for patients with RA . It was revised into a semi-structured baseline and follow-up interview in the Netherlands , and is sensitive to change following both medical treatment and exercise in RA [6, 11] and in chronic low back pain . The MACTAR is a valid measure for myositis and RA [13, 14] and for hip- and knee osteoarthritis . However, the MACTAR follow-up has not previously been evaluated for all aspects of reliability, as to sensitivity to change or to ability of detecting changes following acupuncture treatment in RA.
This is a single subject experimental design (SSED) study which in contrast to an open label design with group analysis allows each patient to be their own control by including a non-interventional A-phase followed by an interventional B-phase . Patients were assessed systematically three times a week during the 2-week A-phase and twice a week during the 6-week B-phase.
All patients with RA, referred to acupuncture treatment for pain, at Danderyd Hospital, Stockholm (n = 10), during August 2006 to January 2007, who fulfilled the inclusion criteria were eligible and were invited to participate. Inclusion criteria; RA diagnosis according to the ACR criteria , diagnosis duration > 12 months, ≤ 70 years of age, unchanged medication during the past 3 months. Exclusion criteria; any contra-indication for acupuncture treatment; received acupuncture treatment during the past 6 months, not understanding the Swedish language. All 10 patients accepted participation initially, however, two patients chose to abort participation, due to lack of time or starting a new medical treatment at the time of inclusion. Eight patients entered and completed the study and their demographic data is presented in Tables 1 and 2.
The MACTAR is a semi-structured interview assessing activity limitation, consisting of a baseline interview and a follow-up interview. Both interviews contain pre-defined questions on general health, physical function, social function and emotional function which are rated according to degree of disease-impact in daily life on a five-grade Likert Scale from 1 (poor health) to 5 (good health) [9, 13]. Patients are also asked to state five activities of daily living that are limited due to RA, and then to rank the five activities starting with the most important to improve. In the follow-up interview, patients rate if their ability to perform their five activities has improved, deteriorated or not changed at all. Patients also rate if their general health, physical-, social-, or emotional function has changed due to the treatment. MACTAR total score varies from 21 (severe limitations) to 77 (no limitation).
Pain during the last week was assessed on a Visual Analogue Scale (VAS pain), 0 (no pain) − 100 (worst imaginable pain) . Patients’ global well-being during the last week (PGA) was rated on a VAS, 0 (best well-being possible) − 100 (poor well-being) .
At the initial visit, patients were assessed using the MACTAR baseline interview which took between 20 and 45 min to complete. Five telephone interviews were then scheduled during the following 2 weeks during the A-phase. The MACTAR follow-upwas used during these telephone interviews, and then throughout the rest of the study. The participants were encouraged to set aside 15 min in private during the telephone interviews. After completing these initial six interviews (A-phase with one baseline and five follow-up interviews) the acupuncture treatment was introduced twice a week during the first 4 weeks, and then once a week during the following 2 weeks (B-phase). Each patient received 10 acupuncture treatments. The MACTAR follow-up interview was longitudinally compared to the VAS pain and to PGA. Thus the MACTAR follow-up interview, VAS pain and PGA were performed at every treatment visit. One physical therapist performed all acupuncture treatments, and another physical therapist administered all assessments. Both physical therapists had vast experience of acupuncture treatment and of using the included assessment methods.
Due to the type of data, non-parametric statistics were used in all statistical analyzes and data on group level are presented as median and range. Intra Class Correlation Coefficients were calculated between the first three measurements during the A-phase (A1 vs A2, A1 vs A3, and A2 vs A3), as well as the standard error of the measurement (SEM), the coefficient of variation expressed as percentage of the mean (CV%), and the smallest detectible difference (SDD). Bland and Altman methods were used to assess possible systematic disagreement between the test occasions . Calculations included the mean difference between the measures, the standard deviation of the differences (SD difference) and the 95% limits of agreement: mean ± 2 SD difference. Intra Class Correlation coefficients of ICC > 0.75 were considered to reflect “good” to “excellent” correlations . To assess sensitivity to change, two different procedures were undertaken. Firstly, to analyze changes in the MACTAR follow-up interview during the B-phase compared to the A-phase, two median values (one median of the first three assessments and one median of the remaining three assessments) from the six A-phase assessments were calculated, and a celeration line was drawn through these median values continuing through the 10 B-phase assessments. A majority of assessment points during the B-phase above or below the celeration line indicate a change in activity limitation . A classic power analysis based on mean values and sample size are not applicable for a SSED. Instead, the number of measurement points in both A- and B-phases and the natural variation during the A-phase indicates how many patients need to be included. A large A-phase measurement point variation requires a large change during the B-phase to indicate a true change. In SSED design results can be calculated for one patient, but the replication of results in a small number of additional patients is essential in SSED [22, 23]. Secondly, the Friedman’s ANOVA test was performed to analyze changes during the B-phase on the MACTAR follow-up interview, VAS pain and PGA, with the Wilcoxon signed rank test as after test. The level of significance was set to p < 0.05. SPSS for Windows, version 22, was used in all analyses. Statsoft, Statistica (version 12) was used to create the Bland and Altman plots.
All eight participants completed the 10 acupuncture treatments and all assessments throughout both the A-phase and the B-phase.
The ICC between A1 and A2 was 0.747, with SEM 4.21, CV% 8.60 and SDD 11.68. For the measures A1–A3, ICC was 0.697 with SEM 4.16, CV% 8.50 and SDD 11.50 and for the A2–A3 the ICC was 0.878 with SEM 2.25, CV% 4.50 and SDD 6.24, indicating good to excellent reliability.
Figure 1a–c with the Bland–Altman plots shows the difference between the occasions plotted against the mean of the measurement points A1–A2, A1–A3, and A2–A3. There was no systematic disagreement between the test occasions.
The celeration line analyses indicated that the MACTAR follow-up interview could detect changes after treatment, as all participants except one had a majority of assessment points above or below the celeration line in the B-phase (Additional file 1: Figure S1). Analysis on group level revealed a statistically significant improvement in the MACTAR follow-up interview at B10 compared to B1 (p = 0.02), while VAS pain and PGA remained unchanged (Table 2).
The present study indicated good to excellent reliability and ability to detect changes over time for the MACTAR follow-up without systematic disagreement between the test occasions. Best reliability, i.e., high ICC, low SEM, CV% and low SDD were found for measurements A2 and A3, which implies that it is preferable to exclude the first session of the MACTAR follow-up interview in clinical daily routines. The MACTAR follow-up interview was able to detect effect of acupuncture treatment, while measures of pain and well-being remained unchanged suggesting that the patient-preference MACTAR is a valuable addition to predefined PROMs.
The celeration line analysis indicated a change in MACTAR score in seven participants, which supports the statistically significant change on group level. These changes were not mirrored in the VAS pain or PGA, which could indicate that the MACTAR follow-up interview was more sensitive to change as it captures values that are important to the patient beyond those assessed using general PROM’s. The MACTAR follow-up interview was highly responsive following both medical treatment and exercise in patients with RA [6, 9]. Significant within-group improvement in MACTAR follow-up was evident also in an exercise study in patients with myositis .
Activities once identified as important to improve by using the MACTAR, might lose relevance as seasons and other life factors change [6, 12, 13]. Although during a short time-span, our study was performed during fall and winter when patients with RA often experience day-to-day variations due to for example infections or weather changes . In some patients, this led to large variations in the A-phase assessment of patient preference, VAS pain and PGA. Assessments during warmer seasons might have resulted in smaller symptom variation further improving reliability and sensitivity to change of the MACTAR follow-up. A SSED design might not be optimal in RA-patients, however a similar study protocol was successfully performed in patients with other inflammatory conditions [26, 27].
The first MACTAR interview was performed during the first study visit, while the following five A-phase interviews were performed over the telephone. One advantage was that telephone interviews required no time for traveling to and from the clinic, which probably enabled a more diverse group of full-time workers and severely disabled patients to participate. Our study included both younger and older men and women with various RA-duration, which strengthened external validity of our results. However, non-verbal communications are lost and participants might have had difficulties to find a secluded space to avoid distractions during the interviews. However, a pilot telephone interview performed with one participant before the first study visit did not reveal any test–retest variations. Although RA is not a rare condition, a SSED design was chosen to be able to study the natural variability of patient preference on an individual and on a group level. In order to account for this natural variability, a relative large number of subjects were included for this SSED compared to common study sizes in SSED . The SSED has a relative low evidence value and a larger study with another design is therefore needed to confirm our results. In the present study, patients were treated with manual or electrical acupuncture based on clinical status and indication. The choice of stimulation module might not be important as the main purpose of this study was to evaluate measurement properties of the MACTAR.
In conclusion, the MACTAR follow-up interview seems to be reliable and may be able to detect changes in activity limitations following acupuncture treatment in patients with established RA. This implies that the MACTAR could be a valuable addition to already established outcome measures. The patient-preference focus will enhance patient relevance and patients’ participation in clinical care. For optimal precision, we suggest a learning occasion before the first MACTAR follow-up assessment.
The main limitations of this study are the smaller sample-size SSED and that assessments were only performed during the colder season with more day-to-day symptom variation which might have resulted in lower validity and sensitivity to change of the MACTAR.
McMaster Toronto Arthritis patient preference questionnaire
Health Assessment Questionnaire
Arthritis Impact Measurement Scale
patient-reported outcome measure
single subject experimental design
Visual Analogue Scale
patient’s global well-being
standard error of the measurement
the coefficient of variation expressed as percentage of the mean
smallest detectible difference
Kay J, Upchurch KS. ACR/EULAR 2010 rheumatoid arthritis classification criteria. Rheumatology. 2012;51(suppl 6):vi5–9.
Staud R. Peripheral and central mechanisms of fatigue in inflammatory and noninflammatory rheumatic diseases. Curr Rheumatol Rep. 2012;14:539–48.
Gaudin P, Leguen-Guegan S, Allenet B, et al. Is dynamic exercise beneficial in patients with rheumatoid arthritis? Joint Bone Spine. 2008;75:11–7.
Wang C, de Pablo P, Chen X, et al. Acupuncture for pain relief in patients with rheumatoid arthritis: a systematic review. Arthritis Rheum. 2008;59:1249–56.
Kilic L, Erden A, Bingham CO III, Gossec L, et al. The reporting of patient-reported outcomes in studies of patients with rheumatoid arthritis: a systematic review of 250 articles. J Rheumatol. 2016;43:1300–5.
Verhoeven AC, Boers M, van der Linden S. Responsiveness of the core set, response criteria, and utilities in early rheumatoid arthritis. Ann Rheum Dis. 2000;59:966–74.
Hewlett S, Smith AP, Kirwan JR. Values for function in rheumatoid arthritis: patients, professionals, and public. Ann Rheum Dis. 2001;60:928–33.
Wright JG, Young NL. A comparison of different indices of responsiveness. J Clin Epidemiol. 1997;50:239–46.
Verhoeven AC, Boers M, van der Liden S. Validity of the MACTAR questionnaire as a functional index in a rheumatoid arthritis clinical trial. J Rheumatol. 2000;27:2801–9.
Tugwell P, Bombardier C, Buchanan WW, et al. The MACTAR patient preference disability questionnaire—an individualized functional priority approach for assessing improvement in physical disability in clinical trials in rheumatoid arthritis. J Rheumatol. 1987;14:446–51.
de Jong Z, Munneke M, Zwinderman AH, et al. Is a long-term high-intensity exercise program effective and safe in patients with rheumatoid arthritis? Results of a randomized controlled trial. Arthritis Rheumatol. 2003;48:2415–24.
Sanchez K, Papelard A, Nguyen C, et al. McMaster-Toronto Arthritis patient preference disability questionnaire sensitivity to change in low back pain: influence of shifts inpriorities. PLoS ONE. 2011;6:e20274.
Alemo Munters L, van Vollenhoven RF, Alexanderson H. Patient preference assessment reveals disease aspects not covered by recommended outcomes in polymyositis and dermatomyositis. ISRN Rheumatol. 2011. https://doi.org/10.5402/2011/463124.
Alemo Munters L, Brodin N, Löfgren E, et al. Disabilities of importance for patients to improve—using a patient preference tool in rheumatoid arthritis. Disabil Rehabil. 2014;36:1762–7.
Barten DJ, Pisters MF, Takken T, Veenhof C. Validity and responsiveness of the Dutch McMaster Toronto Arthritis patient preference questionnaire (MACTAR) in patients with osteoarthritis of the hip or knee. J Rheumatol. 2012;39:1064–73.
Zhan S, Ottenbacher KJ. Single subject research designs for disability research. Disabil Rehabil. 2001;23:1–8.
Arnett FC, Edworthy SM, Bloch DA, et al. The American Rheumatism Association 1987 revised criteria for the classification of rheumatoid arthritis. Arthritis Rheumatol. 1988;31:315–24.
Wilkie D, Lovejoy N, Dodd M, Tesler M. Cancer pain intensity measurement: concurrent validity of three tools—finger dynamometer, pain intensity number scale, visual analogue scale. Hosp J. 1990;6:1–13.
Jones S, Steiner A, Garrett L, et al. The bath ankylosing spondylitis patient global score (BAS-G). J Rheumatol. 1996;35:66–7.
Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1(8476):307–10.
Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater-reliability. Psychol Bull. 1979;86:420–8.
Backman CL, Harris SR, Chrisholm JM, et al. Single-subject research in rehabilitation. A review of studies using AB, withdrawal, multiple baseline, and alternating treatments design. Arch Phys Med Rehabil. 1997;78:145–53.
Byiers BJ, Reichle J, Symons FJ. Single-subject experimental design for evidence-based practice. Am J Speech Lang Pathol. 2012;31:397–414.
Munters AL, Dastmalchi M, Andgren V, et al. Endurance exercise improves health and may reduce disease activity in patients with established polymyositis and dermatomyositis. A multicenter randomized controlled trial with a 1-year open extension follow-up. Arthritis Care Res. 2013;65:1959–68.
Cutolo M. Circadian and circannual rhythms in RA. Nat Rev Rheumatol. 2011;7:500–2.
Clarke-Jenssen AC, Frediksen PM, Lilleby V, et al. Effects of supervised aerobic exercise in patients with systemic lupus erythematosus: a pilot study. Arthritis Rheum. 2005;53:308–12.
Sandström S, Röstlund S, Boström C, et al. Improved muscle function following whole-body vibration for patients with polymyositis and dermatomyositis. A pilot study. Ann Rheum Dis. 2011;70(Suppl3):776.
NB and HA made substantial contribution to conception and design, performed statistical analysis, interpreted data, drafted the manuscript and approved the final version of the manuscript. WG performed statistical analysis, interpreted data, revised the manuscript for important intellectual content and approved the final version of the manuscript. SS and EL collected all clinical data, performed statistical analysis, interpreted data and approved the final version of the manuscript. All authors read and approved the final manuscript.
We would like to thank all patients who participated in this study.
The authors declare that they have no competing interests.
Availability of data and materials
The data sets generated and analysed during the current study are available from the corresponding author on reasonable request.
Consent for publication
Ethics approval and consent to participate
This study was approved by the Regional Ethics committee in Stockholm (2005/1518-31/2). Patients signed an informed consent before entering the study.
This study was funded by Centre for Care Science (Centrum för Vårdvetenskap), and Stockholm City Council (Stockhoms Läns Landsting) and none of them has had influence of the interpretation of data or the final conclusions drawn.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.