Effect size

Editorial approval scope: The team checked current source fidelity, scope boundaries, dates, arithmetic, reader usefulness, practical workflow, and general-information limitations.

Magnitude needs a reference

A difference of five points can be large on one scale and trivial on another. State the outcome, units, comparison, direction, period, and population before naming a magnitude.

Some effect sizes remain in original units, such as a five-minute reduction. Others standardize the difference, express a ratio, or summarize overlap. Each answers a different question.

Match the metric to the design

Group designs may use mean differences, standardized mean differences, risk differences, risk ratios, odds ratios, or correlations. Clustered, repeated, or nested observations require methods that respect dependence.

Single-case designs use repeated measurement within participants and conditions. Metrics may summarize level, trend, nonoverlap, or model-based change. A group standardized mean difference cannot simply be dropped onto a single-case graph, and two single-case metrics can rank the same graph differently.

Keep the graph and raw scale visible

For single-case research, examine level, trend, variability, immediacy, overlap, consistency across replications, and possible outliers as the design permits. Report the raw series alongside any summary.

The WWC Single-Case Design Technical Documentation separates design standards from evidence standards, uses trained visual analysis, and includes recommendations concerning effect-size estimates. It does not identify one universal effect-size metric for every ABA study.

A fictional group calculation

Suppose a fictional randomized study reports a mean posttest score of 18 for one condition and 14 for the comparison, with a pooled standard deviation of 5. The raw mean difference is 18 − 14 = 4 points. A simple standardized mean difference is 4 ÷ 5 = 0.80 before any small-sample or design adjustment.

That value describes the study under the selected formula. It does not establish that every participant improved by 0.80 standard deviations, that the outcome is important to clients, or that the estimate transfers to another measure or setting.

A single-case example needs another lens

Fictional participant Nia records the number of independent route-checking steps during six baseline trips and eight intervention trips. The intervention phase has a higher median, yet both phases show trend and overlap.

Reporting only a nonoverlap percentage would hide those features. The review shows the graph, phase values, intervention changes, replication opportunities, and a design-compatible effect summary selected in advance. One phase contrast does not prove a functional relation.

Sampling uncertainty still matters

An observed effect is an estimate. Confidence intervals, standard errors, randomization tests, sensitivity analyses, or other uncertainty tools may apply depending on the design and metric.

A large point estimate with a wide interval can be less informative than a smaller, precise estimate. In single-case work, few observations, autocorrelation, trend, phase length, measurement error, and modeling choices can affect uncertainty.

Statistical size differs from practical importance

Ask whether the outcome matters to the person, whether the change is observable in daily life, what burden or risk accompanied it, and whether benefit persisted or generalized. A standardized value has no automatic threshold for clinical significance.

Report baseline severity, raw-unit change, client priorities, social validity, adverse effects, and costs where relevant. A tiny effect can matter for a critical outcome; a large effect can target something the person never wanted changed.

Compare only compatible effects

Before combining or comparing values, check outcome definitions, direction, scales, design, phase, follow-up, population, intervention, and adjustment. Reversing scales or mixing ratios with standardized differences can produce false conclusions.

When a review converts metrics, document every formula and assumption. Keep multiple effects from one participant or study from being treated as independent unless the model handles their dependence.

Read an effect-size claim critically

Ask which formula was used, why it fits the design, whether it was prespecified, what data were excluded, whether uncertainty is shown, and how sensitive the result is to another reasonable metric. Look for raw data, a graph, replication, attrition, missing outcomes, and conflicts of interest.

An effect size can organize evidence. It cannot repair a weak design, biased measurement, missing cases, an inaccessible outcome, or a poorly fitted clinical goal.

Report the minimum context

An effect-size table should name the design, sample or participant, outcome, direction, comparison, time point, raw values, formula, adjustment, estimate, uncertainty, and missing-data treatment. Add risk-of-bias and relevance notes. When several outcomes or phases come from one study, identify their dependence and the rule used to avoid selective reporting.

State whether larger values represent benefit, harm, or an arbitrary scoring direction.

Before using an estimate in practice, translate it back to the original outcome and ask whether the observed range matters to this person under comparable conditions. Preserve uncertainty, adverse effects, and study limitations beside the headline number.