Absence of carious lesions at margins of glass-ionomer cement and amalgam restorations: An update of systematic review evidence

Literature search, review and quality assessment of trials

Included trials that passed the exclusion criteria were accepted for further quality assessment and data extraction. Review, data extraction and quality assessment of the accepted trials was undertaken independently by two reviewers (SM and VY). Differences were resolved through discussion and consensus.

In contrast to the original published systematic review [7], quality assessment of accepted trials was undertaken on the basis of availability of evidence indicating successful prevention of selection- and detection/performance bias from start to end of each trial. The new criteria (Table 1) differ from those used in the first review [7]. It has been argued that the inclusion of bias-preventing measures (e.g. randomisation, blinding/masking) into the trial methodology only demonstrates an attempt to reduce bias risk but does not carry proof in itself that such attempt was indeed successful and that it is far more important to judge trial quality according to evidence that indicates to what extent such attempt has succeeded [9]. Against this background the quality criteria were adjusted accordingly. Thus, where trials only reported that randomisation was conducted or included a detailed description of the randomisation process, this was not considered adequate if they failed to provide any evidence that randomisation was indeed effective throughout the trial.

Potential attrition- and publication bias was not investigated in the original systematic review [7]. This update used RevMan Version 4.2 statistical software by The Nordic Cochrane Centre, The Cochrane Collaboration (Copenhagen; 2003) in conducting sensitivity analysis in order to investigate potential risk of attrition bias in trials. To investigate publication bias, a funnel plot was generated, using the datasets from the included clinical trials. The standard error (SE) of the Mean differences was plotted on the Y-axis, and the log of the Relative Risk (RR) on the X-axis, using MIX Version 1.7 meta-analysis software [13]. In addition, the Egger's linear regression method [14] was used to calculate an intercept with 95% Confidence Interval (CI) with statistical significance set at α = 0.05.

Data extraction and analysis

All data concerning primary and secondary outcomes of accepted trials were extracted as single dichotomous datasets, containing the number of observed effects (n) and the total number of evaluations (N) for both the control and the test groups. The Cochrane RevMan, Version 4.2 software package was used in computing the Relative Risk (RR, 95% CI). Statistical significance was set at α = 0.05.

Meta-analysis, using RevMan Version 4.2 statistical software by The Nordic Cochrane Centre, The Cochrane Collaboration (Copenhagen; 2003), was considered for datasets only if they complied with the previously published criteria for clinical homogeneity [7]. The percentage of total variations across datasets (I²) was used in assessing statistical heterogeneity [15]. Statistical significance for assessing statistical heterogeneity was set at α = 0.10. A fixed-effects model was used for meta-analysis under condition of statistical homogeneity of datasets. Pooled datasets were assigned Mantel-Haenszel weights directly proportionate to their sample sizes.

Cumulative meta-analysis, using MIX Version 1.7 meta-analysis software [13], was performed for datasets of consecutive follow-up periods, which also showed clinical and statistical homogeneity, in order to investigate whether a chronological trend of the available evidence might be observed.

Literature search

In addition to the 9 trials included in the original meta-analysis [16–24], one trial [25] was identified during the new database search and a further 11 [26–36] from the hand-search and reference check were included (Figure 1). Of these 21 trials, 11 were excluded [16–19, 30–36], for reasons shown in Table 2. Ten trials passed the exclusion criteria and were accepted for data extraction and quality assessment [20–29]. Of the accepted trials two were of parallel group design [22, 24] and five were split-mouth studies [20, 23, 27–29]. Three trials used the tooth and not the patient as unit of randomisation [21, 25, 26] and can be regarded as of partial split-mouth design: in one trial 20 out of 50 patients received more than one type of restoration [21]; during one trial 45 patients were treated with 38 amalgam and 35 GIC restorations [25] and in one further trial 92 out of 666 patients received both types of materials [26].

Data extraction and analysis results

Of the primary datasets, 9 reflected results for restorations in primary teeth, with 3 datasets for single-surface restorations (DS 03,10,11); 2 datasets for multiple-surface restorations (DS 02,04) and 4 datasets (DS 01,07,08,14) for primary teeth in which single- and multiple-surface restorations were combined. Four datasets showed results in permanent teeth: 3 datasets for single surface restorations (DS 05,06,09) and one dataset for multiple surface restorations (DS 12).

The four datasets that showed secondary outcome results were related to caries progression (DS 13,15,16) and regression (DS 17) in approximal tooth surfaces adjacent to restoration surfaces of neighbouring teeth [26, 29].

Computed results for restorations in permanent teeth

Of the four datasets, two (DS 05, 06) were considered as fulfilling the criteria for clinical homogeneity [23, 24]. Additional analysis established a low statistical heterogeneity (I² = 56.7%, p = 0.13). For that reason the decision was made to pool both datasets, using a fixed-effects model. The meta-analysis results shown in Figure 2 suggest that margins of single-surface GIC restorations in permanent teeth had a 65% lower chance of developing carious lesions on restoration margins after 6 years than did similar teeth restored with amalgam (RR 0.35; 95% CI 0.19 - 0.65; p = 0.001). No difference was found between single-surface restorations after 1 year (DS 09: RR 0.56; 95% CI 0.25 - 1.24, p = 0.15) [27].

One dataset for multiple-surface restoration (DS 12) was identified. The results of this dataset indicate no difference between the two types of restoration after 2 years (RR 0.14; 95% CI 0.01 - 2.53; p = 0.18) [29].

Computed results for restorations in primary teeth

Of the nine datasets, two (DS 02,04) were found to fulfil the criteria for clinical homogeneity [21, 22]. Additional analysis established a low statistical heterogeneity (I² = 0%, p = 0.67). For that reason the decision was made to pool both datasets, using a fixed effect model. The meta-analysis results shown in Figure 3 indicate no difference between the types of multiple-surface restorations, with regard to the chance of developing carious lesions on margins after 3 years (RR 0.38; 95% CI 0.13 - 1.12; p = 0.08). All the other datasets also showed no difference between GIC and amalgam in this regard (Table 4).

In order to investigate whether a possible trend may be assumed in the comparisons of GIC and amalgam, the chronological results from 4 datasets concerning restorations (single- and multiple surface combined) after 1 year (DS 07), 2 years (DS 08), 3 years (DS 14) and 5 years (DS 01) were included in a cumulative meta-analysis [20, 25, 26]. The datasets were considered homogenous in all aspects except in their follow-up periods. In addition, a lack of statistical heterogeneity was established (I² = 5.94%; p = 0.36). The cumulative Relative Risk indicates no statistical significant difference between the two materials after 5 years (RR 0.75; 95% CI 0.45 - 1.23; p = 0.26). However, a shift was observed in the cumulative Relative Risk over time, with a continued reduction of the 95% confidence intervals in favour of GIC (Figure 4).

Quality assessment of trial results

Publication bias risk

Publication bias was investigated, using one funnel plot (Figure 5). The funnel plot covering data for caries progression showed an even distribution that did not suggest publication bias. Egger's linear regression method for the same datasets showed an intercept of 0.96 (95% CI -2.03 - 0.11; p = 0.07). The regression result was not statistically significant.

Selection-, Detection-/Performance bias risk

All of the accepted trials appear to be limited by risk of selection- and detection/performance bias. Bias or systematic error may affect studies, causing either an over- or an under-estimation of the treatment effect of an investigated clinical procedure. Overestimation has been observed to be the most common [37]. Kjaergard et al. reported a treatment effect overestimation of 48% caused by lack of random sequence allocation [38] and Egger et al. reported a treatment effect overestimation of 54% and 53% due respectively to lack of allocation concealment and lack of evaluator blinding [39].

It has been emphasized that selection bias can be successfully prevented only if the allocation sequence remains truly random and free from potential interference throughout the trial [9, 10]. For this reason it is important that trials include an effective process for concealing the random allocation sequence and that such process has indeed prevented direct observation and prediction of the random sequence allocation throughout each trial [9, 10]. Quality assessment in terms of the internal validity of trials should, therefore, be a measure of the result of random sequence allocation and allocation concealment, and not only of its reported attempt. All trials accepted in this systematic review failed to report not only on evidence of successful sequence allocation and allocation concealment results but also on necessary details about how sequence allocation and allocation concealment was attempted (Table 5). None of the trials therefore provide any guarantee that each patient had an equal chance to be allocated to either treatment group and thus their allocation may have favoured the outcome of one type of treatment above the other. One measure for testing whether random sequence allocation has been successful is testing whether covariates differ between treatment groups at baseline [9]. Only two articles, reporting different datasets from the same trial, had included such a test and reported on its outcome [22, 24]. The statistically non-significant results (p > 0.05) suggest a successful random allocation. However, doubt remains regarding potential bias risk, as other non- balanced covariates may exist that were not tested for and/or not reported.

From the onset, successful blinding or masking appeared not to have been possible in all trials, owing to the obvious differences between GIC and amalgam in their clinical appearance. For that reason the allocation to either treatment group was visible to patients, operators and evaluators. However, the difficulties of successful blinding still carry the danger of detection-/performance bias, which thus may have affected the results of the trials. Potential knowledge of superiority claims prior to the trial may have led patients to change their oral hygiene habits, operators to place restorations more carefully or evaluators to apply evaluation criteria more subjectively. This in turn may have favoured the outcome of one type of treatment above the other.

Attrition bias risk

Sensitivity analysis may be used to establish whether missing data could have affected the trial outcomes, by assuming that the numbers of restoration lost to evaluation were either failures or successes [40]. Comparison of the resulting analysis results with reported trial outcomes indicates whether different conclusion should be drawn. Sensitivity analysis was conducted for all datasets. The analysis results differed from reported outcomes of three datasets (DS 02,05,14) extracted from three trials [21, 23, 26]. According to the analysis results, multiple-surface restorations placed with GIC (DS 02) would have a 5-times higher chance than amalgam of developing caries on restoration margins after 3 years in primary teeth [21] and single-surface restorations placed with GIC (DS 05) would have only a 10% (instead of 65%) lesser chance of developing caries on margins than amalgam after 6 years in permanent teeth [23]. However, neither would affect the overall conclusions drawn from the meta-analyses (Figures 2 and 3): that single-surface GIC restorations have a lesser likelihood of developing caries on margins in permanent teeth after 6 years and that no difference can be found between the two materials regarding multiple-surface restorations after 3 years in primary teeth.

The outcome of dataset 14 would be affected by attrition bias, in that no significant differences between GIC and amalgam would be found if all patients lost to follow-up were assumed to show no caries progression in the adjacent teeth [26]. However, a statistically significant difference would remain in favour of GIC (p = 0.002) if all lost patients were assumed to show caries progression. It has been observed that trials with patient exclusion showed more beneficial test treatment effects than analyses based on randomisation of all, or most, patients [41]. This may be due to the possibility that patients drop out or discontinue treatment if the treatment proves unsuccessful. As no proof is available about how many patients who were unavailable for evaluation really did or did not have caries progression, the more conservative assumption would be that most of them had progressed caries. Against such a background, a remaining statistical significant difference between the treatment groups, in favour of GIC, seems more likely.

A run-in phase is considered to be a stage during a trial where all patients receive, for example, the test treatment and only those patients that respond well to the treatment are later used for random allocation in either the control or the test group [10]. Such practice would effectively exclude patients from the randomisation process and potentially favour, for example, the test group above the control. No such run-in phase was indicated in any of the accepted trials (Table 5).

Publication bias risk

Publication bias was investigated by generating a funnel plot (Figure 5). Publication bias is present when the results of published research differ from those of all the research that has been done [42]. Funnel plots are scatter graphs showing the size of studies on the Y-axis (large studies on top, small studies at the bottom) and the effect size, observed in these studies, on the X-axis. The effect sizes of larger studies have the tendency to cluster near the mean. Small studies have effect sizes that are dispersed across a wider range. Results of both types of studies, plotted on a scatter graph, give the shape of an inverted, in absence of publication bias symmetrical, funnel [43]. Publication bias affects a funnel plot in the form of a concentration of studies to only one side (asymmetry). Such asymmetry is created when particular smaller studies are published only when they show a larger than average effect. However, if the number of studies (n) is less than 10, any asymmetry may be due to chance and not to publication bias [44]. For that reason the decision was made to plot results of the 17 extracted datasets instead of those of published articles. Despite this departure from the common use of funnel plots, the use of datasets (instead of published studies) will also indicate potential publication bias when only datasets that show a larger than average effect are published and other datasets are not. In this update, the funnel plot concerning dichotomous RCT data on caries progression showed a symmetrical spread of dataset results (Figure 5). As the visual judgement of funnel plots is subjective, intercepts were calculated (95% CI), using Eggers regression [14]. The calculated intercept confirmed a non-significant result. Eggers regression is used to quantify bias captured by funnel plots [43]. However, its reported power is considered to be low unless in the presence of severe bias or a high number (n) of studies/datasets (n >10) [43, 44]. Therefore, the results of the calculated intercept concerning datasets for caries progression (n = 17) may be ascribed to lack of severe publication bias.

Data extraction and analysis results

The extended scope of this update did not change the overall results of the meta-analysis originally published [7]. However, it has to be noted that these results are limited by risk of selection- and detection-/performance bias. As the true extent of such bias impact remains unknown within the reviewed trials, the results need to be regarded with caution.

In eight out of the ten accepted trials that used split-mouth or partial split-mouth design (Table 3) a cross-over effect caused by fluoride, released from the GIC restoration, may have reduced the caries susceptibility of tooth margins surrounding the amalgam restorations and thus may have confounded the observed results towards a more equivocal outcome.

It also has to be noted that none of the accepted studies reported on fluoride exposure of subjects. It can be assumed that subjects may have been exposed to external fluoride sources and that this may have increased the caries resistance of teeth restored with amalgam, thus confounding the caries-preventive effect of GIC as suggested by Hara et al. [45]. In this context, it is worth pointing out that the only trial reporting on a statistically significant primary outcome in favour of GIC was conducted in a developing country in Africa [23], where the opportunities of exposure to external fluoride sources may be few. The hypothesis that GIC restorations may in general be less susceptible to recurrent caries is supported by the secondary trial outcomes (DS 13,15,16) [26, 29].

A significantly lower chance of caries development on GIC restoration margins was reported in the permanent dentition only [23]. In the primary dentition, the results for multiple-surface restorations after 3 years (Figure 3) suggest that neither of the materials is superior. The reasons for this remain unclear. It can be assumed that factors like the larger restoration surface, as well as the greater difficulties involved in placing restorations in children than in adults, may outweigh any caries-preventive properties of GIC in comparison to amalgam. However, under ideal trial conditions free of bias and confounder influence, GIC restorations may also have a lower chance of developing caries on restoration margins on a long-term basis in primary teeth. This statement should be regarded as a hypothesis requiring testing through high quality randomised control trials. Plausibility for this hypothesis is provided by the results of the included cumulative meta-analysis (Figure 4). Cumulative meta-analysis is used to provide insights into how much the efficacy of treatments, often reported as mean results with 95% confidence intervals, change over time as evidence accumulates [46]. It is the result of conducting a new meta-analysis each time a new set of evidence emerges [47]. A cumulative meta-analysis not only allows the evaluation of any additional contributions made by individual studies to the cumulatively combined results of preceding studies [48] but also allows observation of a trend in evidence direction over time. The forest plot in Figure 4 shows a steady shift in cumulative results, starting after 1 to 5 years, in favour of GIC. The observed shift is caused by a cumulative change in Relative Risk (from 4.36 to 0.75) and the narrowing of the cumulative 95% confidence interval (from 0.51 - 37.08 to 0.45 - 1.23).

Recommendations for further research

Systematic reviews have been reported as providing the highest form of clinical evidence [49]. However, the internal validity of such evidence can only be as good as the internal validity of the trials reviewed. Although the trials accepted in this update may be considered not much affected by attrition- and publication bias, their risk of selection- and detection/performance bias is high. Thus, further high quality randomised control trials (RCT) are needed in order to verify (or disprove) the currently available results. Such RCTs should adopt a parallel group design and include randomisation and allocation concealment methods that can effectively prevent direct observation and prediction of the allocation sequence. For this purpose the maximum randomisation method has been suggested [9]. Covariates of both treatment groups should be tested as to whether they differ at baseline (after randomization). Recently, the inclusion of the Berger-Exner test has been recommended for authors of trials needing to investigate whether selection bias has been introduced into their studies [9, 10]. Where bias risk has been found, such risk may be adjusted statistically [9]. In order to assure that the lack of blinding may not have led to the favouring of one treatment above another, trials should use and report on procedures and tests employed that may have limited, or at least monitored, potential bias risk. Future trials should also base their reporting on the CONSORT statement [50].