A simple method to estimate prediction intervals and predictive distributions: Summarizing meta-analyses beyond means and confidence intervals.

Published on Apr 3, 2019in Research Synthesis Methods5.273
· DOI :10.1002/JRSM.1345
Chia‐Chun Wang1
Estimated H-index: 1
(NTU: National Taiwan University),
Wen-Chung Lee27
Estimated H-index: 27
(NTU: National Taiwan University)
Sources
Abstract
: A systematic review and meta-analysis is an important step in evidence synthesis. The current paradigm for meta-analyses requires a presentation of the means under a random-effects model; however, a mean with a confidence interval provides an incomplete summary of the underlying heterogeneity in meta-analysis. Prediction intervals show the range of true effects in future studies and have been advocated to be regularly presented. Most commonly, prediction intervals are estimated assuming that the underlying heterogeneity follows a normal distribution, which is not necessarily appropriate. In this article, we provide a simple method with a ready-to-use spreadsheet file to estimate prediction intervals and predictive distributions nonparametrically. Simulation studies show that this new method can provide approximately unbiased estimates compared with the conventional method. We also illustrate the advantage and real-world significance of this approach with a meta-analysis evaluating the protective effect of vaccination against tuberculosis. The nonparametric predictive distribution provides more information about the shape of the underlying distribution than does the conventional method.
📖 Papers frequently viewed together
References37
Newest
#1Jessica Gurevitch (SBU: Stony Brook University)H-Index: 55
#2Julia Koricheva (RHUL: Royal Holloway, University of London)H-Index: 69
Last. Gavin B. Stewart (Newcastle University)H-Index: 34
view all 4 authors...
Meta-analysis—the quantitative, scientific synthesis of research results—has been both revolutionary and controversial, with rapid advances and broad implementation resulting in substantial scientific advances, but not without pitfalls.
Source
#1Yusuke Yamaguchi (Astellas Pharma)H-Index: 4
#2Kazushi MaruoH-Index: 12
Last. Richard D Riley (Keele University)H-Index: 72
view all 4 authors...
Background In a random effects meta-analysis model, true treatment effects for each study are routinely assumed to follow a normal distribution. However, normality is a restrictive assumption and the misspecification of the random effects distribution may result in a misleading estimate of overall mean for the treatment effect, an inappropriate quantification of heterogeneity across studies and a wrongly symmetric prediction interval.
Source
#1Dean Langan (Ebor: University of York)H-Index: 12
#2Julian P T Higgins (UoB: University of Bristol)H-Index: 134
Last. Mark Simmonds (Ebor: University of York)H-Index: 26
view all 3 authors...
Random-effects meta-analysis methods include an estimate of between-study heterogeneity variance. We present a systematic review of simulation studies comparing the performance of different estimation methods for this parameter. We summarise the performance of methods in relation to estimation of heterogeneity and of the overall effect estimate, and of confidence intervals for the latter. Among the twelve included simulation studies, the DerSimonian and Laird method was most commonly evaluated. ...
Source
#1Adil Daud (UCSF: University of California, San Francisco)H-Index: 73
#2Jedd D. Wolchok (MSK: Memorial Sloan Kettering Cancer Center)H-Index: 152
Last. Omid HamidH-Index: 85
view all 24 authors...
PurposeExpression of programmed death-ligand 1 (PD-L1) is a potential predictive marker for response and outcome after treatment with anti–programmed death 1 (PD-1). This study explored the relationship between anti–PD-1 activity and PD-L1 expression in patients with advanced melanoma who were treated with pembrolizumab in the phase Ib KEYNOTE-001 study (clinical trial information: NCT01295827).Patients and MethodsSix hundred fifty-five patients received pembrolizumab10 mg/kg once every 2 weeks ...
Source
#1Joanna IntHout (Radboud University Nijmegen)H-Index: 27
#2John P. A. IoannidisH-Index: 205
Last. Jelle J. Goeman (Radboud University Nijmegen)H-Index: 40
view all 4 authors...
Objectives Evaluating the variation in the strength of the effect across studies is a key feature of meta-analyses. This variability is reflected by measures like τ 2 or I 2 , but their clinical interpretation is not straightforward. A prediction interval is less complicated: it presents the expected range of true effects in similar studies. We aimed to show the advantages of having the prediction interval routinely reported in meta-analyses. Design We show how the prediction interval can help u...
Source
#1Areti Angeliki Veroniki (St. Michael's Hospital)H-Index: 26
#2Dan JacksonH-Index: 37
Last. Georgia Salanti (UoI: University of Ioannina)H-Index: 77
view all 10 authors...
Meta-analyses are typically used to estimate the overall/mean of an outcome of interest. However, inference about between-study variability, which is typically modelled using a between-study variance parameter, is usually an additional aim. The DerSimonian and Laird method, currently widely used by default to estimate the between-study variance, has been long challenged. Our aim is to identify known methods for estimation of the between-study variance and its corresponding uncertainty, and to su...
Source
#1Sandip Pravin Patel (UCSD: University of California, San Diego)H-Index: 29
#2Razelle Kurzrock (UCSD: University of California, San Diego)H-Index: 137
The resurgence of cancer immunotherapy stems from an improved understanding of the tumor microenvironment. The PD-1/PD-L1 axis is of particular interest, in light of promising data demonstrating a restoration of host immunity against tumors, with the prospect of durable remissions. Indeed, remarkable clinical responses have been seen in several different malignancies including, but not limited to, melanoma as well as lung, renal and bladder cancer. Even so, determining which patients derive bene...
Source
Background The androgen-receptor isoform encoded by splice variant 7 lacks the ligand-binding domain, which is the target of enzalutamide and abiraterone, but remains constitutively active as a transcription factor. We hypothesized that detection of androgen-receptor splice variant 7 messenger RNA (AR-V7) in circulating tumor cells from men with advanced prostate cancer would be associated with resistance to enzalutamide and abiraterone. Methods We used a quantitative reverse-transcriptase–polym...
Source
#1John E. CornellH-Index: 44
#2Cynthia D. Mulrow (University of Texas Health Science Center at San Antonio)H-Index: 58
Last. Steven N. Goodman (Stanford University)H-Index: 91
view all 7 authors...
A primary goal of meta-analysis is to improve the estimation of treatment effects by pooling results of similar studies. This article explains how the most widely used method for pooling heterogeneous studies—the DerSimonian–Laird (DL) estimator—can produce biased estimates with falsely high precision. A classic example is presented to show that use of the DL estimator can lead to erroneous conclusions. Particular problems with the DL estimator are discussed, and several alternative methods for ...
Source
#1Joseph Cesario (MSU: Michigan State University)H-Index: 21
Concerns have been raised recently about the replicability of behavioral priming effects, and calls have been issued to identify priming methodologies with effects that can be obtained in any context and with any population. I argue that such expectations are misguided and inconsistent with evolutionary understandings of the brain as a computational organ. Rather, we should expect priming effects to be highly sensitive to variations in experimental features and subject populations. Such variatio...
Source
Cited By20
Newest
#2Alan McCallH-Index: 18
Last. Maarten van Smeden (UU: Utrecht University)H-Index: 21
view all 3 authors...
Abstract null null Objective null The Nordic hamstring exercise (NHE) is strongly recommended to reduce hamstring injuries in previous meta-analyses (50% reduction in risk of injury). To underline the importance and impact of adopting appropriate methodology for evidence synthesis, we revisited the study selection, re-analysed and updated the findings of the most recent meta-analysis. null null null Study design and setting null Only randomised control trials (RCT) using NHE as one of the preven...
Source
#1William L. Baker (UConn: University of Connecticut)H-Index: 35
#2Diana M Sobieraj (UConn: University of Connecticut)H-Index: 23
Source
#1Guo-Qiang Zhang (University of Gothenburg)H-Index: 1
#2Jin-Liang Chen (CQMU: Chongqing Medical University)
Last. Mary Ann Lumsden (Glasgow Royal Infirmary)H-Index: 34
view all 17 authors...
Background: There remains uncertainty about the impact of menopausal hormone therapy (MHT) on women’s health. A systematic, comprehensive assessment of the effects on multiple outcomes is lacking. We conducted an umbrella review to comprehensively summarize evidence on the benefits and harms of MHT across diverse health outcomes. Methods and findings: We searched MEDLINE, EMBASE, and 10 other databases from inception to November 26, 2017, updated on December 17, 2020, to identify systematic revi...
Source
#1Jadbinder Seehra (Guy's and St Thomas' NHS Foundation Trust)H-Index: 13
#2Daniel Stonehouse-Smith (Guy's and St Thomas' NHS Foundation Trust)
Last. Nikolaos Pandis (University of Bern)H-Index: 41
view all 3 authors...
BACKGROUND A prediction interval represents a clinical interpretation of heterogeneity. The aim of this study was to determine the prevalence of prediction interval reporting in orthodontic random effect meta-analyses. The corroboration between effect size estimates with 95% confidence intervals (CIs) and prediction intervals were also explored. MATERIALS AND METHODS Systematic reviews (SRs) published between 1 January 2010 and 31 January 2021 containing at least one random effects meta-analysis...
Source
#1Maya B. Mathur (Stanford University)H-Index: 20
#2Tyler J. VanderWeele (Harvard University)H-Index: 87
Meta-regression analyses usually focus on estimating and testing differences in average effect sizes between individual levels of each meta-regression covariate in turn. These metrics are useful but have limitations: they consider each covariate individually, rather than in combination, and they characterize only the mean of a potentially heterogeneous distribution of effects. We propose additional metrics that address both limitations. Given a chosen threshold representing a meaningfully strong...
Source
#1Conor Goold (University of Leeds)H-Index: 4
#2Ruth C. Newberry (NMBU: Norwegian University of Life Sciences)H-Index: 34
Predicting the behaviour of shelter dogs after adoption is an important, but difficult, endeavour. Differences between shelter and post-adoption environments, between- and within-individual heterogeneity in behaviour, uncertainty in behavioural predictions and measurement error all hinder the accurate assessment of future behaviour. This study integrates 1) a longitudinal behavioural assessment with 2) a novel joint hierarchical Bayesian mixture model that accounts for individual variation, miss...
Source
#1Maya B. Mathur (Stanford University)H-Index: 20
#2Jacob PeacockH-Index: 2
Last. Thomas N. Robinson (Stanford University)H-Index: 85
view all 7 authors...
Abstract Reducing meat consumption may improve human health, curb environmental damage, and limit the large-scale suffering of animals raised in factory farms. Most attention to reducing consumption has focused on restructuring environments where foods are chosen or on making health or environmental appeals. However, psychological theory suggests that interventions appealing to animal welfare concerns might operate on distinct, potent pathways. We conducted a systematic review and meta-analysis ...
Source
#1Maya B. Mathur (Stanford University)H-Index: 20
#2Tyler J. VanderWeele (Harvard University)H-Index: 87
Selective publication and reporting in individual papers compromise the scientific record, but are meta-analyses as compromised as their constituent studies? We systematically sampled 63 meta-analyses (each comprising at least 40 studies) in PLoS One, top medical journals, top psychology journals, and Metalab, an online, open-data database of developmental psychology meta-analyses. We empirically estimated publication bias in each, including only the peer-reviewed studies. Across all meta-analys...
Source
#1Lianne Siegel (UMN: University of Minnesota)H-Index: 3
#2M. Hassan Murad (Mayo Clinic)H-Index: 96
Last. Haitao Chu (UMN: University of Minnesota)H-Index: 48
view all 3 authors...
Often clinicians are interested in determining whether a subject's measurement falls within a normal range, defined as a range of values of a continuous outcome which contains some proportion (eg, 95%) of measurements from a healthy population. Several studies in the biomedical field have estimated reference ranges based on a meta-analysis of multiple studies with healthy individuals. However, the literature currently gives no guidance about how to estimate the reference range of a new subject i...
Source
This website uses cookies.
We use cookies to improve your online experience. By continuing to use our website we assume you agree to the placement of these cookies.
To learn more, you can find in our Privacy Policy.