To report IOA results without overclaiming, state the response definition, observation system, sampling frame, observer independence, calculation, raw agreements and disagreements, missingness, prevalence, range, and context. Link the result to the exact data sampled. IOA describes observer correspondence under those conditions. It does not establish measurement validity, treatment integrity, clinical benefit, or causation.

Name the sample

State clients or cohort, dates, settings, observers, procedure versions, sessions due, observations completed, and exclusions.

Show the calculation

Provide method, raw numerator, denominator, percentage, rounding, and any zero-zero or missing-state rule.

Display variation

Report session or condition values, range, prevalence, and disagreement type when they change interpretation.

Bound the conclusion

State that agreement supports confidence in observer correspondence for the sample. Review validity, integrity, outcomes, and client experience separately.

Keep sampling coverage separate from agreement

Define the sampling plan before reporting. Suki's practice preselects six of twenty sessions for dual observation. Five produce valid matched pairs and one planned pair is missed. Pair-completion coverage is 5/6 = 83.3%, while completed IOA coverage across all sessions is 5/20 = 25%.

These denominators answer different questions. The first shows execution of the sampling plan. The second shows how much of the full service cohort received a valid dual observation. Neither can be replaced by the 80% agreement among matched trials.

Show variation before the pooled percentage

Report session-level numerator, denominator, method, percentage, prevalence, and disagreement type. Identify the two sessions below 70% without turning 70% into a universal acceptability rule. Describe whether disagreement clusters by observer, setting, response form, trial type, access condition, or procedure version.

Pooled agreement can hide a weak session or give larger sessions more weight. A simple average of session percentages has a different weighting rule. State which summary is used and retain the session rows so another reviewer can reproduce it.

Write only the claim the sample supports

A defensible sentence identifies the method, sample, result, and limits: “Across five valid dual-observation sessions, observers agreed on 24 of 30 matched trials, or 80%; completed dual-observation coverage was 5 of 20 sessions, and session results varied.” It does not establish validity of the target definition, treatment integrity, treatment effect, or general reliability outside the observed sample.

Name any missing pair and corrective action. Preserve later corrections and updated reports without rewriting the evidence available at the original decision time.

Use the formula and units for Suki

For Suki's bounded IOA results report: a complete IOA result includes method, numerator, denominator, percentage, observation coverage, and disagreement pattern. Keep observer records independent until both are locked. Preserve the matched exposure, raw observations, numerator, denominator, unrounded calculation, and final rounding.

Work the arithmetic for Suki

Suki reports 24 agreements in 30 matched trials, or 80% trial-by-trial IOA, across five completed dual-observation sessions. The practice had preselected six of twenty sessions, so one planned pair is missing. Two completed sessions fall below 70%. The report shows session values, 5/6 pair completion, 5/20 full-cohort coverage, disagreement types, and follow-up alongside the pooled 80%.

Apply the bounded reporting rule prospectively

Suki's qualified clinician identifies the decision before choosing an agreement method or threshold. Compare a stricter or complementary method when prevalence, totals, or aggregation could hide relevant disagreement. Repair definitions, visibility, training, sampling, or matching before judging the implementer or client data.

Audit observer and sample conditions for Suki

Suki's record identifies observers, independence, training, recording mode, shared exposure, response definition, procedure version, observation coverage, missing pairs, reactivity, and disagreements. Consent, privacy, recording, employment, supervision, and payer rules remain separate requirements. Keep every open pair visible with an owner and next action.

Protect client access during Suki's observation

Suki's observation keeps AAC, interpreters, mobility, food, water, bathroom use, prescribed care, health support, rest, relationships, and emergency help available. Do not create extra distress or remove ordinary supports to obtain an agreement sample. Record when access, health, withdrawal, or observer presence changes the condition.

Ask five IOA questions for Suki

Use these questions in the bounded IOA results report:

  • Did both observers independently score the same exposure, unit, definition, and condition?
  • Which calculation, numerator, denominator, prevalence, and rounding reproduce the result?
  • Which disagreements, missing pairs, invalid units, reactivity, or matching problems remain?
  • Which validity, treatment-integrity, client-outcome, experience, and access questions stay separate?
  • Which repair, complementary method, next sample, owner, and review date follow?

Keep raw observer records and disagreement types available.

Use current sources within their scope for Suki

For Suki's bounded IOA results report, the BACB ethics hub points to current professional ethics materials. The CASP public summary provides high-level autism-treatment scope without licensed detailed procedures. The BACB Ethics Code addresses competence, client involvement, consent and assent when applicable, documentation, risk, supervision, and evaluation for covered behavior analysts. The BCBA Test Content Outline includes measurement, reliability, sampling, and data-based decisions as examination content rather than a case protocol.

For Suki, Vollmer, Sloman, and St. Peter Pipkin discuss practical implications of data reliability and treatment-integrity monitoring. Essig, Rotta, and Poling review IOA and fidelity reporting in an identified ABA research corpus. Reed and Azulay describe a calculation tool and agreement methods. ASHA says AAC users should always have access to their tools or devices. These sources do not create one universal IOA method or threshold.

Close Suki's agreement review

Ask Suki and the responsible clinician to review the bounded IOA results report through accessible communication. Record the method, arithmetic, sample, disagreements, limits, measurement repair, selected decision, next observation, and review date. Reopen the analysis when the definition, observer, system, exposure, prevalence, client access, procedure, or decision changes.

Clinician release checklist

  • Are 24 agreements, 30 matched trials, and 80% reproducible from session rows?
  • Are five completed pairs, six planned pairs, and twenty total sessions kept as distinct cohorts?
  • Does the report show 5/6 = 83.3% sampling-plan completion and 5/20 = 25% full-cohort coverage?
  • Are low sessions, the missing pair, disagreement types, and follow-up visible?
  • Is the conclusion limited to observer correspondence in the observed sample?

Related resources

Sources