Lecture Meta-analysis

Meta-analysis
The EBM workshop
A.A.Haghdoost, MD; PhD of Epidemiology



Definition
Meta-analysis: a type of systemic review
that uses statistical techniques to
quantitatively combine and summarize
results of previous research
A review of literature is a meta-analytic
review only if it includes quantitative
estimation of the magnitude of the effect
and its uncertainty (confidence limits).
EBM workshop

Meta analysis

A.A.Haghdoost


Function of Meta-Analysis(1)
1-Identify heterogeneity in effects among
multiple studies and, where appropriate,
provide summary measure
2-Increase statistical power and precision
to detect an effect
3-Develop ,refine, and test hypothesis
continued
EBM workshop

Meta analysis

A.A.Haghdoost


Function of Meta-Analysis(2)
continuation

4-Reduce the subjectivity of study
comparisons by using systematic and
explicit comparison procedure
5-Identify data gap in the knowledge base
and suggest direction for future research
6-Calculate sample size for future studies
EBM workshop

Meta analysis

A.A.Haghdoost


Historical background
Ideas behind meta-analysis predate Glass’ work
by several decades
– R. A. Fisher (1944)
• “When a number of quite independent tests of significance have
been made, it sometimes happens that although few or none can
be claimed individually as significant, yet the aggregate gives an
impression that the probabilities are on the whole lower than

would often have been obtained by chance” (p. 99).
• Source of the idea of cumulating probability values

– W. G. Cochran (1953)
• Discusses a method of averaging means across independent
studies
• Laid-out much of the statistical foundation that modern metaanalysis is built upon (e.g., inverse variance weighting and
homogeneity testing)
EBM workshop

Meta analysis

A.A.Haghdoost


Basic concepts
The main outcome is the overall magnitude
of the effect.
It's not a simple average of the magnitude
in all the studies.
Meta-analysis gives more weight to studies
with more precise estimates.
– The weighting factor is 1/(standard error)2.
EBM workshop

Meta analysis

A.A.Haghdoost

Main magnitude of effects
Descriptive
Mean
Prevalence

Analytical
Additive
Mean difference
Standardized mean difference
Risk, rate or hazard difference
Correlation coefficient
Multiplicative
Odds ratio, Risk, Rate or Hazard Ratio
EBM workshop

Meta analysis

A.A.Haghdoost


Statistical concepts(1)
You can combine effects from different
studies only when they are expressed in
the same units.
Meta-analysis uses the magnitude of the
effect and its precision from each study
to produce a weighted mean.

EBM workshop

Meta analysis

A.A.Haghdoost


Statistical concepts(2)
The impact of fish oil consumption on Cardio-vascular diseases

EBM workshop

Meta analysis

A.A.Haghdoost


Forest plot
the graphical display of results from
individual studies on a common scale is a
“Forest plot”.
In the forest plot each study is represented
by a black square and a horizontal line
(CI:95%).The area of the black square
reflects the weight of the study in the
meta-analysis.
A logarithmic scale should be used for
plotting the Relative Risk.
EBM workshop

Meta analysis

A.A.Haghdoost


Forest plot

EBM workshop

Meta analysis

A.A.Haghdoost


Statistical concepts(3)
There are two basic approach to
Quantitative meta –analysis:
Weighted-sum
Fixed effect model
Random effect model
Meta-regression model
EBM workshop

Meta analysis

A.A.Haghdoost


Fixed effect model
General Fixed effect model- the inverse
variance – weighted method
Specific methods for combining odds ratio

Mantel- Haenszel method
Peto’s method
Maximum-Likelihood techniques
Exact methods of interval estimation
EBM workshop

Meta analysis

A.A.Haghdoost


Fixed effect model
In this model, all of the observed difference
between the studies is due to chance
Observed study effect=Fixed effect+ error
Xi= θ + ei

ei is N (0,δ2 )

Xi = Observed study effect
θ = Fixed effect common to all studies

EBM workshop

Meta analysis

A.A.Haghdoost


General Fixed effect model

Ť=∑ wiTi/ ∑ wi
The weights that minimize the variance of Ť
are inversely proportional to the
conditional variance in each study
Wi=1/vi
Var(Ť)=1/ ∑ wi

EBM workshop

Meta analysis

A.A.Haghdoost


Mantel- Haenszel method
Each study is considered a strata.

Ť=∑ai di / ni / ∑bi ci /ni

EBM workshop

Meta analysis

A.A.Haghdoost


Random effect model
The “random effect” model, assumes a
different underlying effect for each study.
This model leads to relatively more weight

being given to smaller studies and to
wider confidence intervals than the fixed
effects models.
The use of this model has been advocated
if there is heterogeneity between study
results.
EBM workshop

Meta analysis

A.A.Haghdoost


Source of heterogeneity
Results of studies of similar interventions usually
differ to some degree.
Differences may be due to:
- inadequate sample size
- different study design
- different treatment protocols
- different patient follow-up
- different statistical analysis
- different reporting
- different patient response
EBM workshop

Meta analysis

A.A.Haghdoost

An important controversy has arisen over
whether the primary objective a metaanalysis should be the estimation of an
overall summary or average effect across
studies (a synthetic goal)
or the identification and estimation of
differences among study-specific effects
(analytic goal)

EBM workshop

Meta analysis

A.A.Haghdoost


Test of Homogeneity
This is a test that observed scatter of study
outcomes is consistent with all of them
estimating the same underlying effect.
Q= X2homo=∑i=1nwi (mi -M)2
df=n-1
wi =weight
M=meta analytic estimate of effect
mi =effect measure of each study
EBM workshop

Meta analysis

A.A.Haghdoost

Dealing with statistical heterogeneity
The studies must be examined closely to
see if the reason for their wide variation
in effect. If it’s found the analysis can be
stratified by that factor.
Subgroup analysis
Exclusion of study
Choose another scale
Random effect model
Meta-regression
EBM workshop

Meta analysis

A.A.Haghdoost


Random effect model
Assume there are two component of
variability:
1)Due to inherent differences of the effect
being sought in the studies (e.g. different
design, different populations, different
treatments, different adjustments ,etc.)
(Between study)
2)Due to sampling error (Within study)

EBM workshop

Meta analysis

A.A.Haghdoost


Random effect model
There are two separable effects that can be
measured
The effect that each study is estimating
The common effect that all studies are
estimating
Observed study effect=study specific (random )effect +
error

EBM workshop

Meta analysis

A.A.Haghdoost


Random effect model
This model assumes that the study specific
effect sizes come from a random distribution
of effect sizes with a fixed mean and
variance.
There are five approach for this model:
Weighted least squares
Un-weighted least squares

Maximum likelihood
Restricted Maximum likelihood
Exact approach to random effects of binary data.
EBM workshop

Meta analysis

A.A.Haghdoost


Random effect
Xi= θi + ei

ei is N (0,δ2 )

Xi = Observed study effect
θi = Random effect specific to each study
θi =U+di
U=Grand mean (common effect)
di is N (0, 2‫ – ) ד‬Random term

EBM workshop

Meta analysis

A.A.Haghdoost