Effective implementation of research into practice: an overview of systematic reviews of the health literature

Searches

We searched Medline and the Cochrane Database of Reviews 1998-2009 using the search strategy employed by Grimshaw et al. [4]. Searches from 1966-July 1998 were completed by Grimshaw et al for their earlier review [4]. Full details of the data extraction process are given in Figure 1.

Inclusion criteria

Systematic reviews are conducted to a set of consistent, transparent quality standards. As such, only systematic reviews were included in the review. In line with Franke et al. [5] reviews were considered to be systematic reviews if they met at least two of the following criteria: search terms were included; search included Pubmed/Medline; the methodological quality of the studies was assessed as part of the review. We included reviews that focused on the implementation of research evidence into practice. Study populations included healthcare providers and patients. Numerous interventions were assessed including clinical guidelines, audit and feedback, continuing professional education, financial incentives, use of opinion leaders and multifaceted interventions. Some systematic reviews included comparisons of different interventions, other reviews compared one type of intervention against a control group. Outcomes related to improvements in process or patient well-being. Numerous individual study types (RCT, CCT, BA, ITS) were included within the systematic reviews (see Additional file 1).

Exclusion criteria

We excluded systematic reviews that did not look explicitly at interventions designed to get research evidence into practice [6]. However, this is far from straightforward in the field of healthcare where the principle of evidence-based practice is widely acknowledged and tools to change behaviour such as guidelines are often seen to be an implicit codification of evidence, despite the fact that this is not always the case [7]. Systematic reviews that explored changes in provider behaviour, but did not state that the changes were research based were excluded. Systematic reviews were excluded that made no mention of research evidence as were papers that were unclear about the use of research evidence [8, 9]. Studies that focused on evidence-based interventions, but failed to report on the evidence base were also excluded. One systematic review was also excluded as it focused on changing patient rather than provider behaviour [10] and a second was excluded as it had no demonstrable outcomes [11].

Audit and feedback

One study looked at the effects of audit and feedback [22]. Prescribing and preventive care seem most likely to be altered by these approaches. More complex areas such as disease management, adherence to guidelines and diagnosis appear less effected by audit and feedback. The authors suggest that this may be due to the differences in complexity of the levels of decision making for clinicians in these respective facets of care.

Computerised decision support

Two studies focused on computerised decision support. One review suggested that research evidence in the form of computer guidance may give clinicians greater confidence when prescribing and lead to more effective prescribing practice. A second review lamented the lack of high-quality primary studies demonstrating improvements in patient outcomes and the poor descriptive value of many studies which make learning lessons for implementation difficult [13]. However, the authors cautioned that the findings were based on a small number of studies and that the overall quality of these was low [12].

Use of opinion leaders

One review looked at the role of local opinion leaders [14]. The authors suggest that opinion leaders can successfully promote evidence-based practice, however, the difficulty of identifying opinion leaders and the labour intensive nature of assessing their impact may limit the use of opinion leaders as a knowledge transfer intervention.

Multifaceted interventions

The majority of the reviews incorporated studies that focused on more than one intervention type across a variety of clinical areas. Examples of the interventions in one multi-faceted approach included: physician and public education, physician peer review and incentive payments to physicians and hospitals. The most consistent message is that interventions designed to promote the use of evidence in policy are more effective when delivered as part of multifaceted intervention that combine different approaches [16, 18–21, 23, 24], though the effect is characterised as small to moderate [17].

A further rationale for multifaceted interventions is that practitioners respond differently to varying types of interventions. For example, one of the reviews [18] investigated whether particular interventions were effective in promoting the use of evidence in obstetrics. They concluded that, in obstetrics, nurses were more receptive to educational strategies than physicians, whilst audit and feedback are effective for both groups.

Overall the reviews suggest that active interventions, such as opinion leaders [14] and reminders and feedback [22] are more effective than passive approaches, such as information campaigns.

Comparison with previous reviews

We know from the literature on the challenges involved in promoting Evidence-Based Medicine that the principles are not universally embedded in mechanisms such as guidelines and educational materials designed to promote clinical practice and service improvement [31]. It is therefore important that evaluations of strategies to change provider behaviour either only focus on changes that are evidenced based (not ones that are politically or financially driven) or are explicit about whether the changes are evidenced based or not.

In reporting the findings of existing primary studies, the systematic reviews point to two issues that warrant further investigation. Firstly, in order to improve the impact of research on health policy and practice, it is essential that theories are developed that reflect the diverse mechanisms involved in implementation [6]. It can be concluded from the reviews reported here that implementation of evidence into practice requires complex interventions that need to consider issues of context and process. For example, many of the systematic reviews [16, 18–21, 23, 24] highlight the importance of multifaceted interventions to promote implementation of evidence into practice. One of the papers [18] signals the importance of considering what implementation mechanisms might be most effective in particular clinical contexts. Therefore, systematic reviews of effectiveness studies alone may not be sufficiently sensitive to deliver all the learning necessary to improve the use of research evidence in clinical practice and service improvement. A deeper understanding may be gained by complementing these studies with the findings from social science research that considers the important issues of context and process [32, 33]. This review identified a much smaller literature focusing explicitly on getting research evidence into practice] [12–24]. This result suggests that further studies should explore whether the nature of the behaviour change being sought (either evidence based or not) has an impact on the degree of change that occurs.

However, the existence of a relatively large, rigorously evaluated set of interventions to promote the use of research evidence provides a vital tool (albeit not the only tool) in meeting the challenge of promoting better use of evidence in practice to improve patient care. Greater transparency and precision about the degree to which interventions are designed to promote evidence-based clinical practice and service improvement will further enhance our understanding of the progress made towards evidence-based medicine.

Limitations and strengths of this study

There are some limitations to conducting overview reviews of systematic reviews. Firstly, there are concerns about double counting individual studies included in different reviews. In this overview we have checked for this and found surprisingly little overlap. Secondly, in reviews of reviews the studies identified are unlikely to have been published in the last few years, given the fact that they have been published in both an original paper and then identified and included in a published review. Thus a review of reviews is less likely to include the very latest research as this would not be captured in existing reviews. This might have particular implications for interventions based on new technologies such as electronic reminders for clinicians. We made best efforts to overcome this by running the searches again at the end of 2009 and incorporated 2 additional studies [13, 20]. Finally, the reviewers are situated at some distance from the original studies and rely on summaries produced by others of existing primary studies. A further limitation related to the selection of systematic reviews that looked explicitly at interventions designed to get research evidence into practice. A number of systematic reviews were excluded as they were not explicit in their inclusion criteria that the studies selected were focused on promoting the use of evidence in practice. Others were excluded as they were not explicit in the main body of the text that the systematic review was focused on promoting the use of evidence in practice. These omissions may relate to reporting bias rather than the systematic reviews themselves.

However there is a considerable efficiency gain in doing a review of reviews, particularly as so much synthesis work has been done in the field already. We can learn from a wide body of work by reviewing 13 reviews of 313 individual studies. Furthermore, a coherent and tested set of interventions emerge that are highly consistent with previous studies [1].