Publication bias

Editorial approval scope: The team checked current source fidelity, scope boundaries, dates, arithmetic, reader usefulness, practical workflow, and general-information limitations.

The missing studies are systematic

Publication bias is more than an incomplete search. The missingness relates to findings, so the available studies can be a distorted sample of the research that occurred.

An NCBI methods guide discusses study-level nonpublication and delayed publication as forms of reporting bias. Incentives from researchers, sponsors, editors, and journals can all shape visibility.

Outcome reporting is related and distinct

A study can be published while unfavorable outcomes, time points, subgroups, adverse events, or analyses remain unreported. That is selective outcome or analysis reporting rather than whole-study publication bias.

Compare publications with protocols, registrations, methods, statistical plans, abstracts, and regulatory reports. Record which prespecified outcomes are missing or changed.

Effect estimates can look too favorable

If smaller null or unfavorable studies disappear, a pooled estimate may become larger and more certain. Apparent consistency can improve because contradictory findings are absent.

Bias can also affect safety estimates, feasibility, acceptability, maintenance, or generalization. A large number of published articles does not prove a complete evidence base.

Search beyond journal articles

A systematic search may include ClinicalTrials.gov, other registries, conference records, dissertations, government reports, funder databases, protocols, preprints, and contact with investigators when appropriate.

Define eligible sources in advance and document dates, search strings, screening, duplicate records, and unavailable results. Grey literature has its own quality and retrieval limits.

Funnel plots require caution

Funnel-plot asymmetry or statistical tests can be consistent with publication bias, but heterogeneity, study size, design quality, outcome choice, chance, and other small-study effects can produce similar patterns.

The NCBI systematic-review methods resource should be applied with method-specific guidance. A symmetrical plot cannot guarantee that nothing is missing.

A fictional evidence map

An evidence team finds 18 published studies and five additional registry records that meet the same initial topic criteria. Three registry records report completion without linked results; two were terminated.

The team reports 18 published plus 5 registry records, not “18 of 23 positive studies.” Study eligibility, completion, and outcome availability remain separate. Missing results are a risk signal, not proof of their direction.

Preregistration improves traceability

Prospective registration and accessible protocols can reveal intended outcomes, analyses, sample size, and completion status. They reduce ambiguity when the final article differs from the original plan.

Registration alone does not prevent selective reporting. Check timing, completeness, amendments, and correspondence between registered and reported methods.

Selective citation can compound bias

Reviews, guidelines, and clinical discussions may repeatedly cite positive studies while overlooking null, conflicting, or methodologically stronger evidence. Citation count is not a quality rating.

Use transparent inclusion criteria and extract risk of bias, population, comparator, dosage, outcomes, follow-up, and funding for every eligible study.

Sensitivity analysis shows dependence

Reviewers can examine how conclusions change under alternative assumptions, inclusion sets, statistical models, or plausible missing results. Selection models and other methods require expertise and assumptions.

Report the main analysis and sensitivity results together. Avoid converting one diagnostic into a precise correction for unknown missing studies.

Clinical decisions need the full evidence picture

Evidence-based practice integrates research evidence with clinical expertise and client values and context. Publication bias lowers confidence in the visible literature but does not dictate one care choice.

State uncertainty plainly. Monitor individualized benefit, burden, assent, harms, feasibility, and change over time.

A practical reading checklist

• Was the protocol registered before results were known?
• Were unpublished and ongoing studies sought?
• Do reported outcomes match the plan?
• Are null findings and adverse events visible?
• Could small-study effects have several explanations?
• Do sensitivity analyses change the conclusion?
• How would missing evidence affect this decision?

Audit the evidence-search denominator

Create a record for every potentially eligible study or registry entry, including unpublished, terminated, ongoing, duplicate, abstract-only, and inaccessible records. Give each one a stable ID, source, status, eligibility decision, outcome availability, sponsor, and last contact date.

Then report separate counts:

• records identified and deduplicated
• studies meeting eligibility criteria
• completed studies with public results
• completed studies without located results
• ongoing, terminated, withdrawn, or unknown-status studies
• studies included in each outcome analysis

This denominator prevents published articles from silently becoming the entire evidence base. It also shows whether a meta-analysis excludes studies because the outcome is unavailable rather than because the study was ineligible.

Predefine when investigators will be contacted, how long the team will wait, and how unavailable data affect certainty. Preserve correspondence and sensitivity assumptions. Update the search before publication because a delayed study can appear after the original cutoff.

End the review with a bias-impact statement. Identify which outcomes, populations, sponsors, and study sizes appear most likely to be missing, then show whether plausible unavailable results would change the conclusion. Lower certainty when the decision depends on an optimistic evidence subset, even if the published studies are internally sound.