Catching and monitoring clinical innovation through performance indicators. The case of the breast-conserving surgery indicator

Breast cancer is the most common cancer in women worldwide, with an estimated number of 464,000 cases in Europe in 2012 [1]. Due to its burden and related impact on healthcare services, quality of care assessment in breast cancer has become a central issue for policy makers and clinicians, to ensure that patients receive the highest standards of care [2–4]. Therefore quality targets and accreditation requirements for hospitals and breast units have been identified [5–7].

The European Society of Breast Cancer Specialist (EUSOMA) has defined quality indicators, linked to specific targets, in order to routinely measure and monitor the capacity of the breast units to ensure high-quality clinical outcomes [5, 6]. One of these indicators is the “Proportion of women with invasive breast cancer with a size <3 cm who underwent breast-conserving surgery (BCS)” and the target suggested by EUSOMA for this indicator is 70% (optimum level 80%) [5].

Recent European studies based on registries linked with claims data or hospital discharge records (HDRs) showed proportions of conservative interventions below these targets and with stable or decreasing trends [2, 8–10]. A recent study from Catalunia based on HDRs showed a significant increase in the percentage of patients treated with breast-conserving surgery (from 67.9% in 2005 to 74.0% in 2011). Of note, in this study, a decreasing trend was observed in high-tech hospitals, contrary to that observed in hospitals of medium and low complexity [11]. The BCS indicator computed using data from the nationwide mammography screening in Denmark ranged from 69.7 to 86.5% [12].

A large variability in BCS was also observed in Italy. This indicator, based on HDRs, is included in the performance evaluation system developed by the Management and Health Laboratory of the Scuola Superiore Sant’Anna of Pisa, Tuscany [13]. The Italian Parliamentary Commission inquiring on the efficiency and effectiveness of the National Health System [14] and, successively, a network of ten Italian Regions used the BCS indicator to measure the quality of surgery in breast cancer care. Moreover, a normative resolution of Tuscany Region, one of the ten Regions in the network, introduced the BCS indicator to monitor the performance of the breast units [15]. To this purpose, using routine administrative data may have several advantages, as they are readily available and can be used on a systematic basis [16]. In Tuscany Region, where the regional governance uses performance indicators, including the BCS indicator, to measure quality of care and set targets to guide the quality improvement process, health professionals are systematically involved in selecting and refining performance indicators. Indeed indicators should be used to guide the care improvement process and should be able to change the clinicians and healthcare organizations behavior [17, 18].

In the case of breast cancer surgery, the Tuscan clinicians agreed to use the BCS indicator. Inter-hospital comparison revealed a large variability among providers (from 65.9 to 90.3%) and low values for the three Tuscan teaching hospitals, surprising professionals themselves. Indeed, the finding of slightly lower proportions of BCS in high-complexity hospitals, which usually have high caseloads, is apparently counterintuitive and in contrast with the literature reporting that higher volumes are positively associated with better practices and outcomes [19] and in particular with a better chance of receiving a conservative intervention [20–22].

A possible source of variability in the BCS indicator is variability in coding diagnoses and procedures in the hospital discharge records database. Moreover, the case mix may be a possible, obvious, explanation of the differences between hospitals. Last but not least, in the recent years many changes occurred in the breast cancer care management, in particular the combination of breast oncologic and reconstructive surgery has revolutionized the surgical approach to breast cancer.

The aim of this paper is to examine the concurrent validity of the BCS indicator to determine whether it reflects accurately the ongoing choices in surgical practice, in particular in the teaching hospitals.

This was done by analyzing the possible reasons of the low proportions of BCS in two high-volume teaching hospitals of Tuscany Region, as measured by the traditional BCS indicator, to determine whether the BCS indicator is up-to-date and valid to monitor appropriateness and quality of breast cancer care and to guide the quality improvement process.

A large number of cases classified as “mastectomies” in HDRs (68.9% at Florence TH and the 74.2% at Pisa TH) proved to be reconstructive procedures (including nipple-sparing, skin-sparing and skin-reducing mastectomies) in the SR. Thus, after applying the new algorithm we reclassified HDR breast cancer surgical procedures into three categories: conservative, reconstructive and traditional mastectomy. Interventions with both conservative and reconstructive codes were recoded as reconstructive; those with both conservative and mastectomy codes were recoded as mastectomy and lastly interventions with both mastectomy codes and 85.33 or 85.35 were recoded as reconstructive.

In order to examine the breast surgery practice of the two hospitals in detail, the three indicators were calculated for five age groups. Results indicate that reconstructive interventions were more common among women under 40 years as compared with those in other age groups, at Florence TH and Pisa TH (Fig. 1).

On the contrary traditional mastectomies were more common among older patients, being performed in 29 and 33% of patients >70 years. After adjusting for age in a logistic regression model, the percentages of reconstructive interventions at Florence and Pisa were respectively 22 and 34% and those of breast-conserving interventions and 63 and 53%.

Breast-conserving surgery followed by adjuvant radiotherapy has been the gold standard for invasive and unifocal breast cancer since the 1980s. The indications of the Italian Ministry of Health, issued in 2010 (http://www.salute.gov.it/imgs/C_17_pubblicazioni_1700_allegato.pdf), read: ‘Surgery is based on partial resection or quadrantectomy when the size of the tumor is within the anatomical limits of a conservative surgery. The ratio between the breast volume and the size of the excision should be favorable to the complete removal of the neoplasm with an acceptable esthetic result.’

Thus, in our algorithm we labeled as conservative all local interventions including lumpectomy quadrantectomy or mammoplasty reduction.

However, in recent years, the preoperative genetic screening and diagnostic imaging, the development of new surgical techniques and the higher attention on the esthetic and psychological outcomes have produced the spread of reconstructive procedures [29, 30]. Therefore, updating quality of care indicators is necessary in order to make them more sensitive to clinical practice changes and technical progress.

Our results indicate that the apparently low percentage of conservative procedures in teaching hospital is due to the inclusion of reconstructive procedures among mastectomies.

The update of the coding algorithm allowed us to establish that traditional mastectomies were about 10% in both hospitals, in line with EUSOMA standards.

We argue that the breast-conserving procedures indicator should be complemented with the reconstructive procedures indicator to provide a more accurate description of the ongoing breast cancer surgery practice in the hospital and to monitor inter-hospital variability.

The present study shows how immediate breast reconstruction is an increasingly common practice in breast cancer surgery, performed in 16% (Florence TH) and 38.3% (Pisa TH) of women, with a peak at younger ages. Our results are consistent with Yang et al., who reported that the percentage of immediate breast reconstruction declines with age, and has a maximum in the age group <40 years [31].

We found that the large difference in the percentage of reconstructive procedures between the two study hospitals was only partially accounted by the lower mean age of women undergoing surgery at Pisa TH, suggesting that other clinical factors, organizational factors or patients’ decisions play an important role. One explanation is that the diagnostic procedures adopted in the two hospitals differed: MRI and preoperative genetic screening was more frequent in Pisa than in Florence, as reported by the breast unit teams of the two hospitals. The more frequent use of MRI facilitates the identification of multifocal tumors and might have led to the surgeon’s decision to undertake reconstructive surgery instead of a conservative procedure.

Our study has several strengths that should be considered. First, the Italian National Health System has a universal coverage; therefore, we could assume that individual economic resources did not account for the variability of the indicator among hospitals. Second, results are based on HDRs, a population-based database, that has been increasingly used to assess quality of care at national level and proved to have a good reliability for breast cancer cases [32]. The development and update of measures to monitor and assess the quality of healthcare pathways, such as cancer care, are crucial steps for the quality improvement and have to be shared with health professionals. Indeed, the improvement strategies are more effective when the professionals involved in the care pathway take part in the systematic data analysis by means a peer-review process. This approach makes health professionals more aware of the quality of services they contribute to produce and more likely to adopt or implement quality improvement strategies [28].

Concerning limitations, the use of HDRs has also shortcomings. Because of the administrative nature of this database, there could be some variability in ICD-9-CM coding practices between hospitals and professionals and miscoding can be possible. Moreover, administrative database does not include clinical information on tumor size or staging. Therefore, comparison between our indicators and EUSOMA indicators might be hindered by the difference in the denominator definition. In fact, when we calculated our indicators using the same criteria as EUSOMA to exclude tumors >3 cm, we found some discrepancies with those calculated using only HDR data, but differences were not significant. Lastly, an easier access to plastic surgery for patients admitted to teaching hospital could be a bias for breast surgery indicators when comparing teaching and non-teaching hospital. However working on the systematic review of coding procedures and disseminating the updated algorithms among professionals can reduce mistakes in HDR coding and, in turn, partial out some of the unwarranted variability in results.

Future studies are needed to examine in deeper detail the impact that clinical, organizational and preference related reasons have on the surgeon’s and women’s choice of the surgical procedure and to observe whether the monitoring of the three procedures over time will allow reducing the variability in the surgical practice across hospitals.

The engagement of professionals in the revision process of the BCS indicator allowed to detect effectively the “bug” of the BCS indicator and to modify rapidly the algorithm used for its calculation. The update of the criteria for monitoring the breast cancer surgical practices allowed (I) catching clinical innovation, (II) ensuring that measurement and targets were set in a correct way, and (III) addressing the improvement process. Considering how strong the effect of control measures in guiding professionals’ behavior can be, it is important to avoid misleading indicators where maybe the targets are achieved but the real point is missed [33]. This is possible if professionals themselves consider these indicators as a work tool. Lastly, only the synergic work among professionals, managers and researchers can ensure the use of a common language that is the first step of a common path towards the improvement of quality of the health care services.