Systematic review

Editorial approval scope: The team checked current source fidelity, scope boundaries, dates, arithmetic, reader usefulness, practical workflow, and general-information limitations.

The question sets the review’s boundaries

Specify the population, intervention or exposure, comparison, outcomes, design, setting, and time frame that matter. Qualitative, diagnostic, prevalence, single-case, and group-intervention questions need different eligibility and appraisal methods.

Involve people who understand the methods, topic, lived experience, information retrieval, and statistics. Disclose financial and intellectual conflicts.

A protocol comes before the results

A protocol records the question, eligibility, search, screening, extraction, bias assessment, synthesis, subgroup, and amendment plans. Public registration or publication helps readers compare planned and completed methods.

Changes may be justified, but authors should date and explain them. Quietly changing the outcome or eligibility after seeing results can bias the conclusion.

Search broadly enough for the question

Search multiple appropriate databases with documented dates and strategies. Consider trial registries, dissertations, conference records, government reports, reference lists, citation searches, and expert contact when relevant.

Language, date, publication-status, and database limits can exclude evidence. Report each limit and its rationale. Searching one convenient database rarely supports a comprehensive claim.

Apply eligibility consistently

Screen titles and abstracts, then full texts, against prespecified criteria. Independent duplicate screening or a validated alternative can reduce errors. Resolve disagreements through a documented process.

List excluded full texts with reasons. Avoid vague categories such as “poor fit” when the actual reason is wrong design, population, outcome, or intervention.

Extract enough detail to judge evidence

Record participants, setting, design, intervention, comparison, outcomes, measures, timing, attrition, missing data, fidelity, adverse effects, funding, and author conflicts. For single-case studies, preserve participant-level and phase-level structure.

Use piloted forms and verification. A transcription error in sample size, direction, variance, or phase value can change the synthesis.

Appraise risk of bias and relevance

The National Academies systematic-review standards cover team organization, stakeholder input, protocol, search, screening, extraction, critical appraisal, synthesis, and reporting. The source describes systematic reviews as identifying, selecting, assessing, and synthesizing related studies.

Choose appraisal tools suited to each design. A quality score that adds unrelated items can hide a fatal bias. Explain how limitations affect confidence in each outcome and the body of evidence.

A fictional review flow

A fictional review of caregiver-coaching procedures identifies 620 database records and 35 additional records. After removing 105 duplicates, 550 records enter title and abstract screening.

Reviewers exclude 480 and assess 70 full texts. They include 18 reports representing 14 studies. The flow records 52 full-text exclusions by reason and links multiple reports from the same study.

The arithmetic reconciles: 620 + 35 − 105 = 550, and 550 − 480 = 70. Clean flow does not establish that the included studies are unbiased or compatible.

Synthesis can remain narrative

Organize studies by question, design, population, intervention, and outcome. Show tables and study-level results. Explain patterns, inconsistency, limitations, and missing evidence.

Meta-analysis is appropriate only for sufficiently compatible estimates with dependence handled correctly. A narrative synthesis still needs transparent methods and should avoid vote counting based only on statistical significance.

Certainty differs by outcome

Confidence can vary across outcomes because design, bias, consistency, precision, directness, and publication bias differ. Avoid one blanket grade for an entire intervention.

State what is known, uncertain, and missing. Distinguish absence of evidence from evidence of no effect.

Check freshness and applicability

Find the last search date and whether important new studies have appeared. An excellent review can become outdated. Living reviews require clear update triggers and version history.

For practice, compare the reviewed population, goals, supports, risks, and settings with the person at hand. Integrate the review with clinical expertise, client values, assessment, alternatives, and ongoing data.

Common shortcuts weaken confidence

A review is less trustworthy when authors search one database, omit search dates, screen alone without verification, change eligibility after seeing results, exclude unpublished evidence without explanation, or treat several reports from one study as separate studies. Counting positive and negative papers ignores sample size, precision, design, and bias.

Watch for conclusions that outrun the eligible question. A review of short clinic studies cannot establish long-term community benefit. A review limited to autistic preschoolers cannot automatically support the same claim for adults or people with other conditions. The abstract should reflect the review’s actual population, outcomes, evidence quality, and uncertainty.

Check whether two papers report the same participants. Duplicate samples can make the evidence base look larger and can improperly give one study extra weight in a synthesis. Document suspected overlap.

Before relying on a review, compare its last search date with current databases and check for corrections, retractions, or important new studies. Record the update search and explain whether the newer evidence changes the conclusion or only its uncertainty.