Meta-analysis

Editorial approval scope: The team checked current source fidelity, scope boundaries, dates, arithmetic, reader usefulness, practical workflow, and general-information limitations.

A meta-analysis is usually part of a review

A systematic review identifies and appraises eligible evidence using explicit methods. A meta-analysis is the quantitative step used when selected results can sensibly be combined. A review may contain several meta-analyses or none.

Skipping a pooled estimate can be the stronger decision when participants, interventions, comparisons, designs, outcomes, or follow-up periods differ too much.

Pool only compatible questions

The AHRQ quantitative-synthesis guide says clinical, methodological, and statistical factors all matter when deciding whether to combine studies. Researchers first examine whether studies are similar enough for an average effect to be meaningful.

Compatibility includes the population, intervention, comparator, outcome, timing, design, measurement, and effect metric. A broad topic label such as “communication intervention” does not establish compatibility.

Put effects on a common direction and scale

Researchers may combine raw mean differences when studies use the same scale, standardized mean differences when scales differ but represent a common construct, or ratios and risk differences for binary outcomes.

Every conversion needs a stated direction. Higher scores might mean improvement on one measure and harm on another. Reverse coding, imputed standard deviations, and translated outcomes should be documented and tested in sensitivity analyses.

Weighting does not equal quality voting

Many models give more weight to estimates with greater precision. A large study can therefore contribute more than a small study. Weight reflects the model and variance estimate rather than a complete judgment of relevance or risk of bias.

Poorly measured or inapplicable evidence can remain influential. Reviewers should assess study limitations and explore whether conclusions change when high-risk studies are excluded.

A fictional weighted estimate

Suppose three fictional compatible studies estimate standardized effects of 0.20, 0.50, and 0.80. Their prespecified inverse-variance weights are 0.50, 0.30, and 0.20.

The weighted estimate is (0.20 × 0.50) + (0.50 × 0.30) + (0.80 × 0.20) = 0.41. The simple unweighted mean would be 0.50.

The difference shows why weights matter. Neither number gives the confidence interval, heterogeneity, study quality, client relevance, or risk of missing studies. The published synthesis needs those elements.

Heterogeneity asks how effects vary

Variation can arise from real differences in people, settings, procedures, comparisons, outcomes, follow-up, or study methods, as well as sampling error. Report study estimates and a forest plot rather than relying only on a pooled value.

A heterogeneity statistic does not explain the source. Subgroup analysis or meta-regression needs enough studies, a prespecified rationale, and cautious interpretation. Many exploratory comparisons can create chance findings.

Dependence needs explicit handling

One study may report several outcomes, follow-up points, treatment arms, participants, or single-case effects. Treating every estimate as independent can produce confidence intervals that are too narrow and give one study excessive weight.

Use a method suited to the dependence structure, select one estimate under a declared rule, or combine outcomes appropriately. State the unit of analysis and any assumptions.

Missing results can shift the answer

Published studies and reported outcomes may differ from all studies and outcomes that were conducted. Search registries, dissertations, reports, reference lists, and other sources appropriate to the question. Compare protocols with reports when available.

Funnel plots and statistical tests can sometimes flag small-study patterns, yet they do not prove publication bias and often have low power with few studies. Discuss the likely direction and importance of missing evidence.

Read beyond the headline

Check the protocol, search date, databases, eligibility, screening, data extraction, risk-of-bias method, effect metric, model, heterogeneity, dependence, sensitivity analyses, certainty assessment, and conflicts. Look at the range of study results and prediction interval when provided.

A pooled group estimate cannot prescribe care for one person. Combine it with individual priorities, assessment, alternatives, risks, burden, clinical expertise, and ongoing data.

Report enough to reproduce the synthesis

A useful report provides the protocol, search, included-study table, extracted effects, code or formulas, direction rules, model, variance estimates, dependence method, excluded effects, and sensitivity analyses. Forest plots should identify studies and confidence intervals. Data corrections and author queries need an audit trail.

Readers should be able to reconstruct why each study entered a particular pool. When proprietary measures or unavailable data prevent full reproduction, state the limitation and show the remaining evidence as transparently as possible.

Archive the analysis date and software version. A later correction, newly available study, or revised coding decision can change the pooled estimate and should produce a traceable new version.

Before carrying a pooled result into a family decision, examine the study range, prediction interval when available, risk of bias, and populations missing from the evidence. Explain why the estimate is or is not applicable to the person's goal and setting.