Using evidence-based guidelines to inform service provision: a structured mapping exercise within the National Health Service Diabetes Prevention Programme in England

Abstract

Objective

The National Health Service (NHS) in England planned a national diabetes prevention programme (NHS DPP) with phased implementation. Evidence-based guidelines and service specifications support efficient and effective translation of research into practice. We aimed to evaluate the use of a structured mapping exercise to appraise how evidence, service specification and early phase practice could inform recommendations to guide subsequent implementation of the NHS DPP.

Results

The mapping exercise facilitated comparison and appraisal of key components from different documentary sources (evidence-based NICE guidelines, service specification, and provider documents). Key components were categorised into (A) pathways into programmes, (B) intervention content (C) inequalities and (D) quality assurance and staff training. We identified where key components were the same (accordance), where they varied (discrepancies) and where they were lacking (discontinuities), across the documentary sources. For example there was discrepancy in intervention duration and discontinuity in intervention enrolment procedures. This mapping exercise was useful to compare the fidelity in translation of evidence-based guidance into service specification and programme documents, thus identifying where future service implementation might be improved. This method may be applicable for use with other health conditions where research evidence requires translation into real world population programmes.

Introduction

The NHS 5 year forward view in England emphasised the need for ‘a radical upgrade in prevention and public health’ and included a plan for a national diabetes prevention programme (NHS DPP) [1].

The NHS DPP in England, for individuals at high risk of developing type 2 diabetes (T2D), was planned to be rolled out in phases (i) demonstrator site phase (seven sites in England), (ii) wave 1 (four procured providers in 27 sites across England, permitting 20,000 referrals in 2016/17) and (iii) wave 2 (nationally to the whole country by 2020 with an expected 100,000 referrals available each year). The stated objectives were reduction in incidence of T2D, blood glucose parameters and weight [2].

The NHS DPP service specification [2] was developed by NHS England using research evidence reviews and reports [3, 4], input from an Expert Reference group, a User Involvement group and analysis of the Health Survey for England data. The demonstrator site phase relied mostly on applications from local health economies, where relevant services were already being delivered, and was intended to inform subsequent implementation of the NHS DPP.

National Institute for Health and Care Excellence (NICE) guidelines are created to improve outcomes for those using health services [5]. However, evidence-based guidelines do not necessarily result in the anticipated change in practice. Where guidance is available there are often gaps between evidence-based principles, contractual agreements around intervention commissioning and actual provision of services and interventions [6].

Translation of research into practice involves making sure research findings about effective treatments reach populations that can benefit and are implemented as intended [7]. Reflection on the guidelines available and how these are implemented in practice is necessary to make best use of the recommendations in an applied setting [8].

Summary of the process evaluation of the demonstrator and wave 1 phases of the NHS DPP are reported elsewhere [9].

We aimed to appraise how evidence informed practice to guide subsequent implementation of the NHS DPP through a structured mapping exercise [10].

Main text

Methods

To conduct the mapping exercise we reviewed and extracted data from all the relevant evidence/documentary sources. The documentary sources used within the mapping method were:

• 1. NICE guidelines—PH38 preventing T2D guidance for individuals at high risk [11].

• 2. The draft NHS DPP service specification (demonstrator site phase).

• 3. The final NHS DPP service specification [2] (wave 1 phase).

• 4. All of the seven demonstrator site applications and Memoranda of Understanding (MoU) submitted to become part of the NHS DPP demonstrator site phase. Any provided baseline documentation from the seven sites.

• 5. All of the four procurements and Memoranda of Understanding (MoU) submitted to become a provider for the NHS DPP wave 1 phase. Any provided baseline documentation from the four providers was reviewed.

Data was extracted from the above documentary sources in relation to Key components. Components related to the whole of the programme were extracted to enable the complete T2D prevention pathway to be reviewed and synthesised. These included:

• A. Pathways into the programmes (identification, recruitment, referral, enrolment)

• B. Intervention content (intervention components using existing reporting frameworks and taxonomies [12,13,14])

• C. Inequalities using PROGRESS equality indicators (place of residence, race/ethnicity/language, occupation, gender/sex, religion, education, socioeconomic status, social capital) [15]

• D. Quality assurance and staff training (fidelity measures, resources, staffing, training requirements)

Information was extracted on staff or health care professional involvement at each stage of the programme and also areas of responsibility, i.e. training of delivery staff.

Structured mapping was used to collate the evidence and enable comparison of the findings across the different documentary sources. Initially we used a spreadsheet to facilitate the mapping process and we used recommendations in NICE guidance (PH38) to identify key components [16]. The extracted data were then organised into tables (Table 1).

Table 1 Mapping of intervention content

The mapping exercise drew on Structured Mapping Theory, which describes the use of mapping and how evaluation of the analogy gives a measure of the quality of match between the base and a target [10]. Critical appraisal identified whether key components across and between the documentary sources were in:

• 1. Accordance—components that were common and reported across all documentary sources, e.g. the format of the intervention (face-to-face group sessions).

• 2. Discrepancies—components that varied across documentary sources, e.g. duration or intensity of the intervention.

• 3. Discontinuities—components that did not appear across all documentary sources, e.g. intervention enrolment procedures.

We used the Accordance, Discrepancies, Discontinuities (ADD) ‘ADD-Fuse’ method outlined above, which was developed during the NHS DPP demonstrator and wave 1 phase evaluation projects, to facilitate critical appraisal. Critical appraisal identified where programmes or specifications consistently met the desired criteria or where differences or gaps were present and therefore where improvements could be recommended. Recommendations were formulated from the appraisal process and provided to the NHS DPP management team to inform subsequent phases (Additional file 1). Using this mapping exercise on two different phases of the NHS DPP showed how the programmes and service specifications progressed between these phases.

The mapping exercise was completed independently by two reviewers with expertise in behaviour change interventions and checked by a third reviewer in both phases, any disparities were resolved through discussion. However, we found that the clear specification of key components and the agreed classification as Accordance, Discrepancy and Discontinuity for each key component across each documentary source led to a high degree of consistency between reviewers. The data collection and methodology are summarised in a flow chart (Fig. 1).

Fig. 1

Flow chart of data collection and methodology: ADD-Fuse method applied to the NHS Diabetes Prevention Programme in England

Results

Table 1 provides an example of how the mapping exercise was conducted.

Table 1 illustrates how the mapping exercise facilitated the identification of key components, actors and responsibilities within the NICE guidelines, NHS DPP service specification and NHS DPP provider documentation (the applied setting/context). Tables were then used to compare and contrast across the different documentary sources.

We described this method as identifying Accordance, Discrepancies and Discontinuities (the ADD-Fuse method), which was used to highlight the key commonalities, differences and gaps between the documentary sources (Table 2).

Table 2 Using Accordance, Discrepancies and Discontinuities (ADD-Fuse) to identify service specification recommendation

Table 2 illustrates how the identification of accordance, discrepancies and discontinuities (ADD-Fuse method) led to the formulation of recommendations for improvements in relation to the NHS DPP service specification, the planned implementation of the DPPs (provider documents) or both. Recommendations were provided to the NHS DPP Management team and responses to the recommendations were received from the Management team back to the research team (Additional file 1). This method identified key components in the service specification that impact on implementation.

Accordance

In both the demonstrator and wave 1 phases, the format of the intervention was in accordance with the NHS DPP service specification (in person group sessions).

Discrepancy

The reporting of the content of the face-to-face sessions, the level of detail on outcomes, mechanisms of action and techniques used varied greatly between providers. As outlined in Tables 1 and 2 discrepancies were identified in the duration and intensity of the intervention provided in both phases (demonstrator and wave 1). One wave 1 provider did not meet the required standard for duration and intensity, which varied across the four providers. This variation poses an issue for outcome evaluation across the provider interventions. Therefore monitoring of patient contacts is vital to ensure clarity in intervention provision and the impact of this on intervention outcomes.

Discontinuity

A gap (discontinuity) in the draft NHS DPP service specification was identified at the demonstrator phase for the description of behaviour change techniques (BCTs) when compared with recommendation s in NICE guidelines (Table 1). However, by wave 1 more detailed BCT description was requested in the NHS DPP service specification, all providers incorporated the recommended BCTs, and most used additional evidence-based techniques for sustained behaviour change. Detail on additional contact with patients (i.e. telephone support, text messages or social media contact) outside of the standard in group sessions was an identified discontinuity in wave 1 provider documents. The remote contact and materials used, including digital components, should be described with the same level of detail as the other components, including reference to the specific behavioural outcomes, theoretical basis and techniques used. While this level of detail was recommended in the NHS DPP national service specification, the providers did not provide it in such detail.

Discussion

Key findings

Evidence-based documentary sources were used to examine incorporation of evidence in the planned context of the NHS DPP programme. Comparison identified accordance, discrepancies and discontinuities (ADD-Fuse method). Different components, actors and responsibilities that may impact the implementation and evaluability of the NHS DPP were revealed. This process identified recommendations (Additional file 1), informing subsequent phases of the NHS DPP, as to where further clarification and consideration was required to either improve the service specification and/or support the transition of evidence into practice.

Comparison with other studies

Evidence-based lifestyle interventions to prevent or treat diabetes have been shown to be effective and have the potential to reduce morbidity and mortality rates [17,18,19,20]. A difficulty in translating DPP’s into practice is the need to adapt to all patients, clinicians or setting needs. As all local services need to adapt for the diverse UK population it is vital to monitor intended variations as well as unintended variations that occur during implementation, highlighting the importance of process evaluations [21]. A previous review identified translational strategies and cultural adaptations were frequency used to in order for DPP’s to reach diverse populations and those from disadvantaged socioeconomic backgrounds, e.g. adapting materials (including information on local foods or traditional physical activities), reducing the frequency of classes or using community health workers to deliver classes. This review stated how adaptations often go unreported and supports the use of a structured approach to documenting translation, as offered in this current manuscript, to facilitate identification of implementation and effectiveness [22].

Strengths

Mapping two stages of the NHS DPP (demonstrator phase and wave 1) made it possible to trace the progression of a new service as the phases were rolled out in England and observe changes in the NHS DPP service specification over time. The mapping exercise evaluated the programme as a whole, informing on wider aspects of a health improvement programme that could be improved, which would not be assessed if focused solely on the intervention.

Implications

Variance in delivered programmes is likely to have an impact on the assessed outcomes. This structured mapping exercise has utility for implementation science and real-world programmes in explaining differences in outcomes based on specific components of the interventions and how each programme is implemented in relation to the service specification. This method could also enable the identification of key areas that require improvement. The mapping exercise examined the progression of a national programme rollout, identifying how the service specification developed from a draft to a final document (e.g. incorporating greater detail on the inclusion of BCTs and addressing inequalities). This mapping exercise could be utilised in further rollout of the NHS DPP. This process could be used for the development of future service specifications and in the reporting of behaviour change programmes. Fidelity measures need to be established in order to judge whether implementation of a programme meets the required standards.

Conclusion

A mapping exercise was applied in the context of the NHS DPP in England. Using NICE guidelines allowed the service specification and provider documents to be examined in relation to the evidence base. This enabled identification of whether the implementation of a new health care programme may experience problems owing to shortcomings in the service specification or whether problems lie within the transition from evidence into practice. We suggest the method may be applicable for use within other disease or health conditions where research evidence requires translation into real world population programmes.

Limitations

The strength of the evidence base varies across different health conditions and therefore using a mapping exercise like this may not be applicable to programmes that do not have existing evidence-based guidelines and where the evidence is minimal or of poor quality.

The NHS DPP explicitly entered other sources of evidence into the specification development (users, experts, new evidence syntheses) and this has implications for how closely the programme tracks the research evidence. There are of course reasons for doing this, for example practicality and funding can impact greatly, but this brings risks that the key components that make an intervention effective become diluted.

Since the mapping exercise additional evidence has become available, in particular the 2017 update to the NICE PH38 guidelines [23]. Data extraction relied on information provided from demonstrator site and wave 1 providers.