Doctoral Dissertation Defense: April Albertine
Advisor: Dr. Bimal Sinha
Location
Mathematics/Psychology : 401
Date & Time
June 10, 2016, 10:00 am – 2:00 pm
Description
Title: Statistical meta-analysis methods for publication bias, effect size estimation, and synthetic data
Abstract
Meta-analysis is the process by which we combine the results of many studies that address the same research topic. As the body of scientific literature grows and we are presented with seemingly contradictory results on many research questions, the tools of meta-analysis are meant to combine all the available evidence to give a more accurate picture than any one study alone. In this dissertation we examine a number of aspects of statistical meta-analysis: publication bias, an application in synthetic data, and the per- formance of the traditional random effects estimator as compared to the Inverse Variance Heterogeneity (IVHet) estimator.
In statistical meta-analysis, publication bias may tilt the overall conclusion toward a significant result. We examine two approaches for accounting for this bias: the fail safe sample size, for which we provide a correction, and selection models, for which we explore a simplification. In a new approach which combines both of these methods, we use a variation on one type of selection model in order to make better assumptions about the unobserved studies in the fail safe sample size.
We further examine the role of publication bias in a meta-analysis on the impact of Conditional Cash Transfers (social programs meant to reduce extreme poverty in de- veloping nations), as well as provide two improvements to the meta-analysis. We found the original meta-analysis to include several reported effect sizes from the primary studies that were dependent on one another, and hence inappropriate for combining. Furthermore, some studies with multiple effect sizes exhibited within-study heterogeneity, indicating that a random effects method of combining the results is more appropriate than the fixed effect model originally used. The original estimate of the combined effect size was a 3.7 percentage point decrease in drop-out rate with a 95% large sample confidence interval of (-7.1, -0.3); we estimate a 4.0 percentage point decrease but with a wider confidence interval (-7.9,-0.1). Furthermore, publication bias does not seem to impact the results, as evidenced by selection modeling and the fail safe sample size.
An application of the conventional tools of meta-analysis is possible in the area of synthetic data in official statistics. When confidentiality concerns prohibit the publi- cation of survey microdata, synthetic data sets may be released in place of the original data. Multiple imputation is when more than one data set is generated for public release. One mechanism for generating synthetic data sets is posterior predictive sampling. In this setting, assuming normally distributed data, we examine and compare a variety of confidence intervals for the normal mean and variance. Using Monte Carlo simulation, we compare candidate confidence intervals originally formulated for the common mean problem in meta-analysis. For confidence intervals about the population mean, the nar- rowest intervals were based on the sum of the t-statistics corresponding to each synthetic data set. For confidence intervals about the variance, we find that intervals based on the arithmetic, geometric, and harmonic mean all perform reasonably well. An application using census data is provided.
Finally, we compare the traditional random effects model of meta-analysis with the IVHet estimator, which has recently been proposed as a remedy for the overdispersion often seen in traditional random effects estimation. This estimator is identical in form to the fixed effect estimator, but with a different model-derived variance. We show by simulation that under heterogeneity, estimators based on the RE model frequently have a lower MSE and better confidence interval coverage in a variety of settings. We also show that the IVhet estimator, like the traditional RE estimator, is not robust to the choice of estimator for the between-studies variance τ2 or to the choice of estimator for the standard error of the overall effect size.
Abstract
Meta-analysis is the process by which we combine the results of many studies that address the same research topic. As the body of scientific literature grows and we are presented with seemingly contradictory results on many research questions, the tools of meta-analysis are meant to combine all the available evidence to give a more accurate picture than any one study alone. In this dissertation we examine a number of aspects of statistical meta-analysis: publication bias, an application in synthetic data, and the per- formance of the traditional random effects estimator as compared to the Inverse Variance Heterogeneity (IVHet) estimator.
In statistical meta-analysis, publication bias may tilt the overall conclusion toward a significant result. We examine two approaches for accounting for this bias: the fail safe sample size, for which we provide a correction, and selection models, for which we explore a simplification. In a new approach which combines both of these methods, we use a variation on one type of selection model in order to make better assumptions about the unobserved studies in the fail safe sample size.
We further examine the role of publication bias in a meta-analysis on the impact of Conditional Cash Transfers (social programs meant to reduce extreme poverty in de- veloping nations), as well as provide two improvements to the meta-analysis. We found the original meta-analysis to include several reported effect sizes from the primary studies that were dependent on one another, and hence inappropriate for combining. Furthermore, some studies with multiple effect sizes exhibited within-study heterogeneity, indicating that a random effects method of combining the results is more appropriate than the fixed effect model originally used. The original estimate of the combined effect size was a 3.7 percentage point decrease in drop-out rate with a 95% large sample confidence interval of (-7.1, -0.3); we estimate a 4.0 percentage point decrease but with a wider confidence interval (-7.9,-0.1). Furthermore, publication bias does not seem to impact the results, as evidenced by selection modeling and the fail safe sample size.
An application of the conventional tools of meta-analysis is possible in the area of synthetic data in official statistics. When confidentiality concerns prohibit the publi- cation of survey microdata, synthetic data sets may be released in place of the original data. Multiple imputation is when more than one data set is generated for public release. One mechanism for generating synthetic data sets is posterior predictive sampling. In this setting, assuming normally distributed data, we examine and compare a variety of confidence intervals for the normal mean and variance. Using Monte Carlo simulation, we compare candidate confidence intervals originally formulated for the common mean problem in meta-analysis. For confidence intervals about the population mean, the nar- rowest intervals were based on the sum of the t-statistics corresponding to each synthetic data set. For confidence intervals about the variance, we find that intervals based on the arithmetic, geometric, and harmonic mean all perform reasonably well. An application using census data is provided.
Finally, we compare the traditional random effects model of meta-analysis with the IVHet estimator, which has recently been proposed as a remedy for the overdispersion often seen in traditional random effects estimation. This estimator is identical in form to the fixed effect estimator, but with a different model-derived variance. We show by simulation that under heterogeneity, estimators based on the RE model frequently have a lower MSE and better confidence interval coverage in a variety of settings. We also show that the IVhet estimator, like the traditional RE estimator, is not robust to the choice of estimator for the between-studies variance τ2 or to the choice of estimator for the standard error of the overall effect size.
Tags:
