Table of Contents

List of Figures
List of Tables


Acknowledgements


Preface


PART 1: INTRODUCTION


1 HOW A META-ANALYSIS WORKS


Introduction


Individual studies


The summary effect


Heterogeneity of effect sizes


Summary points


2 WHY PERFORM A META-ANALYSIS


Introduction


The SKIV meta-analysis


Statistical significance


Clinical importance of the effect


Consistency of effects


Summary points


PART 2: EFFECT SIZE AND PRECISION


3 OVERVIEW


Treatment effects and effect sizes


Parameters and estimates


Outline


4 EFFECT SIZES BASED ON MEANS


Introduction


Raw (unstandardized) mean difference D


Standardized mean difference, D and G


Response ratios


Summary points


5 EFFECT SIZES BASED ON BINARY DATA (2×2 TABLES)


Introduction


Risk ratio


Odds ratio


Risk difference


Choosing an effect size index


Summary points


6 EFFECT SIZES BASED ON CORRELATIONS


Introduction


Computing R


Other approaches


Summary points


7 CONVERTING AMONG EFFECT SIZES


Introduction


Converting from the log odds ratio to D


Converting from D to the log odds ratio


Converting from R to D


Converting from D to R


Summary points


8 FACTORS THAT AFFECT PRECISION


Introduction


Factors that affect precision


Sample size


Study design


Summary points


9 CONCLUDING REMARKS


Further reading


PART 3: FIXED-EFFECT VERSUS RANDOM-EFFECTS MODELS


10 OVERVIEW


Introduction


Nomenclature


11 FIXED-EFFECT MODEL


Introduction


The true effect size


Impact of sampling error


Performing a fixed-effect meta-analysis


Summary points


12 RANDOM-EFFECTS MODEL


Introduction


The true effect sizes


Impact of sampling error


Performing a random-effects meta-analysis


Summary points


13 FIXED EFFECT VERSUS RANDOM-EFFECTS MODELS


Introduction


Definition of a summary effect


Estimating the summary effect


Extreme effect size in large study


Confidence interval


The null hypothesis


Which model should we use?


Model should not be based on the test for heterogeneity


Concluding remarks


Summary points


14 WORKED EXAMPLES (PART 1)


Introduction


Worked example for continuous data (Part 1)


Worked example for binary data (Part 1)


Worked example for correlational data (Part 1)


Summary points


PART 4: HETEROGENEITY


15 OVERVIEW


Introduction


16 IDENTIFYING AND QUANTIFYING HETEROGENEITY


Introduction


Isolating the variation in true effects


Computing Q


Estimating tau-squared


The I 2 statistic


Comparing the measures of heterogeneity


Confidence intervals for T 2


Confidence intervals (or uncertainty intervals) for I 2


Summary points


17 PREDICTION INTERVALS


Introduction


Prediction intervals in primary studies


Prediction intervals in meta-analysis


Confidence intervals and prediction intervals


Comparing the confidence interval with the prediction
interval


Summary points


18 WORKED EXAMPLES (PART 2)


Introduction


Worked example for continuous data (Part 2)


Worked example for binary data (Part 2)


Worked example for correlational data (Part 2)


Summary points


19 SUBGROUP ANALYSES


Introduction


Fixed-effect model within subgroups


Computational models


Random effects with separate estimates of T 2


Random effects with pooled estimate of T 2


The proportion of variance explained


Mixed-effect model


Obtaining an overall effect in the presence of subgroups


Summary points


20 META-REGRESSION


Introduction


Fixed-effect model


Fixed or random effects for unexplained heterogeneity


Random-effects model


Statistical power for regression


Summary points


21 NOTES ON SUBGROUP ANALYSES AND META-REGRESSION


Introduction


Computational model


Multiple comparisons


Software


Analysis of subgroups and regression are observational


Statistical power for subgroup analyses and meta-regression


Summary points


PART 5: COMPLEX DATA STRUCTURES


22 OVERVIEW


23 INDEPENDENT SUBGROUPS WITHIN A STUDY


Introduction


Combining across subgroups


Comparing subgroups


Summary points


24 MULTIPLE OUTCOMES OR TIME POINTS WITHIN A STUDY


Introduction


Combining across outcomes or time-points


Comparing outcomes or time-points within a study


Summary points


25 MULTIPLE COMPARISONS WITHIN A STUDY


Introduction


Combining across multiple comparisons within a study


Differences between treatments


Summary points


26 NOTES ON COMPLEX DATA STRUCTURES


Introduction


Combined effect


Differences in effect


PART 6: OTHER ISSUES


27 OVERVIEW


28 VOTE COUNTING ? A NEW NAME FOR AN OLD PROBLEM


Introduction


Why vote counting is wrong


Vote-counting is a pervasive problem


Summary points


29 POWER ANALYSIS FOR META-ANALYSIS


Introduction


A conceptual approach


In context


When to use power analysis


Planning for precision rather than for power


Power analysis in primary studies


Power analysis for meta-analysis


Power analysis for a test of homogeneity


Summary points


30 PUBLICATION BIAS


Introduction


The problem of missing studies


Methods for addressing bias


Illustrative example


The model


Getting a sense of the data


Is the entire effect an artifact of bias


How much of an impact might the bias have?


Summary of the findings for the illustrative example


Small study effects


Concluding remarks


Summary points


PART 7: ISSUES RELATED TO EFFECT SIZE


31 OVERVIEW


32 EFFECT SIZES RATHER THAN P -VALUES


Introduction


Relationship between p-values and effect sizes


The distinction is important


The p-value is often misinterpreted


Narrative reviews vs. meta-analyses


Summary points


33 SIMPSON?S PARADOX


Introduction


Circumcision and risk of HIV infection


An example of the paradox


Summary points


34 GENERALITY OF THE BASIC INVERSE-VARIANCE METHOD


Introduction


Other effect sizes


Other methods for estimating effect sizes


Individual participant data meta-analyses


Bayesian approaches


Summary points


PART 8: FURTHER METHODS


35 OVERVIEW


36 META-ANALYSIS METHODS BASED ON DIRECTION AND P -VALUES


Introduction


Vote counting


The sign test


Combining p-values


Summary points


37 FURTHER METHODS FOR DICHOTOMOUS DATA


Introduction


Mantel-Haenszel method


One-step (Peto) formula for odds ratio


Summary points


38 PSYCHOMETRIC META-ANALYSIS


Introduction


The attenuating effects of artifacts


Meta-analysis methods


Example of psychometric meta-analysis


Comparison of artifact correction with meta-regression


Sources of information about artifact values


How heterogeneity is assessed


Reporting in psychometric meta-analysis


Concluding remarks


Summary points


PART 9: META-ANALYSIS IN CONTEXT


39 OVERVIEW


40 WHEN DOES IT MAKE SENSE TO PERFORM A META-ANALYSIS?


Introduction


Are the studies similar enough to combine?


Can I combine studies with different designs?


How many studies are enough to carry out a meta-analysis?


Summary points


41 REPORTING THE RESULTS OF A META-ANALYSIS


Introduction


The computational model


Forest plots


Sensitivity analysis


Summary points


42 CUMULATIVE META-ANALYSIS


Introduction


Why perform a cumulative meta-analysis?


Summary points


43 CRITICISMS OF META-ANALYSIS


Introduction


One number cannot summarize a research field


The file drawer problem invalidates meta-analysis


Mixing apples and oranges


Garbage in, garbage out


Important studies are ignored


Meta-analysis can disagree with randomized trials


Meta-analyses are performed poorly


Is a narrative review better?


Concluding remarks


Summary points


PART 10: RESOURCES AND SOFTWARE


44 SOFTWARE


Introduction


Three examples of meta-analysis software


The software


Comprehensive meta-analysis (CMA) 2.0


Revman 5.0


StataTM macros with Stata 10.0


Summary points


45 BOOKS, WEB SITES AND PROFESSIONAL ORGANIZATIONS


Books on systematic review methods


Books on meta-analysis


Web sites


INDEX

About the Author

Michael Borenstein, Director of Biostatistical Programming Associates
Professor Borenstein is the co-editor of the recently published Wiley book Publication Bias in Meta-Analysis, and has taught dozens of workshops on meta-analysis. He also helped to develop the best-selling software programs for statistical power analysis.


Hannah Rothstein, Zicklin School of Business, Baruch College
Professor Rothstein teaches regular seminars on meta-analysis and systematic reviews, and has 20 years of active research in the area of meta-analysis. She has authored several meta-analyses as well as articles on methodological issues in the area, and made numerous presentations on the topic. Having contributed chapters to two books on meta-analysis, she co-edited Publication Bias in Meta-Analysis.


Larry Hedges, University of Chicago
A pioneer in meta-analysis, Professor Hedges has published over 80 papers in the area (many describing techniques he himself developed, that are now used as standard), co-edited the Handbook for Synthesis Research, and co-authored three books on the topic including the seminal Statistical Methods for Meta-Analysis. He has also taught numerous short courses on meta-analysis sponsored by various international organizations such as the ASA.


Julian Higgins, MRC Biostatistics Unit, Cambridge
Dr Higgins has published many methodological papers in meta-analysis. He works closely with the Cochrane Collaboration and is an editor of the Cochrane Handbook. He has much experience of teaching meta-analysis, both at Cambridge University and, by invitation, around the world.

Reviews

?Both books can be recommended
for graduate training and are useful additions to the library of
those interested in the meta-analytic accumulation of literatures
on training, vocational learning, and education in the
professions.?  (Vocations and Learning, 15 December
2010)