Randomization Tests With Few Possible Assignments

A randomization test with few possible assignments produces coarse exact p values. With R equally likely assignments, the smallest attainable positive one-sided p value under the inclusive rule is 1/R. If R=14, the minimum is 1/14, or about 0.071. A conventional 0.05 threshold cannot be reached. Report that limitation before interpreting data and preserve effect magnitude, visual evidence, client relevance, and design improvements.

Count permissible assignments first

Resolution is a design property. Calculate R from the prospective randomization scheme before choosing alpha or promising a confirmatory test.

List attainable p values

Show the fractions k/R that the exact test can produce. This makes threshold feasibility and coarse steps obvious.

Distinguish low resolution from no change

A coarse test can be unable to reject while the graph shows a potentially important pattern. Report both facts without converting either into a universal conclusion.

Avoid post hoc assignment expansion

Extra schedules added after the draw were never possible under the experiment. They cannot validly improve the denominator.

Plan future designs around welfare and information

Increase randomization possibilities only when the added observations or cases are ethical, feasible, accessible, and capable of answering the research question.

Build Gio's attainable-p-value table

Start Gio's table with a versioned record for the design with few randomization possibilities. For the few possible assignments randomization test question, record the protocol date, design, randomization unit, complete permissible assignment set, assignment probabilities, actual draw, test statistic, direction, tail, tie rule, missing-data rule, analysis mode, software, and reviewer. Keep Gio's raw time series in chronological order and link every derived value to its source. The packet should let an independent analyst reproduce both the observed statistic and its reference distribution.

Work the example for Gio

Gio's alternating design permits 14 condition sequences. Even if the observed statistic is more extreme than every other sequence, its exact inclusive p value is 1/14, or 0.0714. The study therefore cannot produce p below 0.05 under this test. A result of 2/14 would be 0.1429 rather than nearly significant. Preserve every intermediate assignment count, statistic, fraction, and rounding step. The example is fictional and teaches the stated method; it does not create a universal alpha, minimum design, treatment recommendation, or effect category for Gio's clinic self-care study.

Audit Gio's randomization evidence

Gio's table lists all attainable fractions from 1/14 through 14/14, the planned alpha, minimum 0.0714, actual numerator, and design constraints that created 14 sequences. The limitation is documented before outcomes and appears in the report's abstract and decision note. The audit also checks protocol amendments, assignment deviations, duplicate or missing observations, phase-label changes, output precision, code and package versions, and whether the reported test matches the actual randomization. Unresolved discrepancies remain on hold with an owner and due date.

Prevent the main inference error in Gio's review

A team may round 0.0714 to 0.07 and call it significant, or interpret failure to cross 0.05 as evidence of no effect. Gio's report states that the design lacks the p-value resolution for that threshold and examines the observed pattern without dichotomous language. A randomization p value depends on the assignment mechanism and predeclared statistic. It does not turn nonrandom treatment changes into randomized experiments, quantify the effect's size, establish clinical importance, or estimate a population effect.

Integrate design evidence for Gio

Future research can consider more observations, cases, behaviors, settings, or permissible assignments when ethical and feasible. The redesign must retain meaningful clinical constraints and prospective randomization rather than creating burdensome measurements solely to chase a threshold. The WWC Version 5.0 handbook is a research-review standard rather than a universal care protocol. The responsible team still evaluates the design's prediction, verification, replication, phase patterns, attrition, assignment fidelity, measurement quality, and fit to the question.

Protect participation and access for Gio

Research design never outranks Gio's welfare. Keep augmentative and alternative communication, interpreters, mobility, food, water, bathroom use, prescribed care, health support, rest, relationships, and emergency help available. Use accessible communication for consent and assent processes, withdrawal, discomfort, and feedback. If the selected schedule becomes unwanted, unsafe, medically inappropriate, or infeasible, qualified people act for safety and document the design consequence.

Use randomized single-case sources within scope for Gio

For Gio's small assignment-set resolution, the BACB ethics hub and CASP public summary provide professional context, and the BCBA Test Content Outline identifies examination content on measurement and single-case design. A randomized SCED methods paper explains prospective assignment sets, randomization tests, and attainable p-value limits. An effect-measure paper describes a priori statistic choice and the boundary between tentative causal inference and effect magnitude. Research on rapidly alternating designs, changing-criterion designs, and randomized case series shows that assignment schemes and test behavior are design-specific. ASHA supports continuous AAC access.

Rehearse Gio's analysis before outcomes exist

Before the clinic self-care study begins, Gio's team runs the attainable-p-value table with a small fictional sequence. The dry run confirms that the assignment generator returns exactly the permitted schedules, the selected schedule can be traced to the random draw, and the small assignment-set resolution statistic matches a hand calculation. Reviewers deliberately include a tie, a missing observation, and an assignment at each boundary so default behavior becomes visible. They save the test inputs, expected outputs, code version, and correction log. This rehearsal checks the machinery for a design with few randomization possibilities; it never uses Gio's future outcomes or changes the predeclared analysis after data collection.

Few Possible Assignments Randomization Test Review

Review the attainable-p-value table with Gio, the responsible clinician, and a statistician or methodologist experienced in randomization inference. Preserve protocol, assignments, raw observations, graph, code, outputs, deviations, inference limits, client input, decisions, and future design changes. Keep the page draft and noindex until the named external reviews confirm that the research description, accessible participation, and clinical boundary are accurate.

How to Interpret a Randomization Test With Few Possible Assignments