One could, for example, estimate the ratio of the effect size over τ, which can convey how many times larger the treatment effect is compared with the SD of the effect across studies. Dispersion in treatment effects is better reflected by τ because τ is the SD of the between-study effects.
8 With very large (highly precise) studies, even tiny differences in effect size may result in a high I 2, while with small (imprecise) studies, very different treatment effects can yield an I 2 of 0. The clinical interpretation of I 2 is ambiguous: a high I 2 does not necessarily imply that the study effects are dispersed over a wide range 6 and a low I 2 might correspond to high dispersion, 7 because I 2 depends on sample size of the included studies. 5 However, the information that can be directly retrieved from τ 2 and I 2 with respect to the variation in the effects is limited. 4 Therefore, summarising the findings of a meta-analysis in a single summary value sacrifices potentially informative variation. One of the main merits of a meta-analysis may even be that it reveals the variation of effects in different studies. We describe its merits and provide working examples to show how it can be calculated.īetween-study variation in the magnitude of treatment effects cannot be neglected. Reporting a prediction interval in addition to the summary estimate and CI will illustrate which range of true effects can be expected in future settings. The prediction interval presents the heterogeneity in the same metric as the original effect size measure, in contrast to τ 2 or I 2. Our objective in the current article is to show the potential advantages of obtaining and reporting the prediction interval routinely in meta-analyses because its clinical meaning is much more straightforward. 2, 3 However, neither of these two metrics can readily point to the clinical implications of the observed heterogeneity. Typically also some measure of the between-study heterogeneity is presented such as τ 2 or the inconsistency measure I 2. 1 Nevertheless, the usual reporting of a meta-analysis is focused on the summary effect size combined with a CI and p value. Interventions may have heterogeneous effects across studies because of differences in study populations, interventions, follow-up length or other factors like publication bias.
Inferences based on the prediction interval are only valid for settings that are similar (exchangeable) to those on which the meta-analysis is based. Further, the interval will be imprecise if the estimates of the summary effect and the between-study heterogeneity are imprecise, for example, if they are based on only a few, small studies. Limitations are that the calculations and inferences for the prediction interval are based on the normality assumption, which is difficult to ensure. Prediction intervals should be routinely reported to allow more informative inferences in meta-analyses. Completely opposite effects were not excluded in over 20% of those meta-analyses. This occurred in over 70% of statistically significant meta-analyses with heterogeneity of the Cochrane Database of Systematic Reviews. In case of heterogeneity, prediction intervals will show a wider range of expected treatment effects than CIs, and thus may lead to different conclusions. The prediction interval helps in the clinical interpretation of the heterogeneity by estimating what true treatment effects can be expected in future settings. In many meta-analyses, there is large variation in the strength of the effect.