To analyze IOA by observer pair, preserve pair identifiers, method, condition, numerator, denominator, and disagreement type for every sample. Compare pairs only when their units and exposure are compatible. Use patterns to inspect definitions, training, visibility, or recording systems. Thin pair-specific samples cannot support staff rankings, competence conclusions, or punitive decisions.

Keep pair identity

Store pseudonymous observer IDs, role, training state, and pair combination under role-limited access.

Compare compatible samples

Match calculation, response definition, setting, procedure version, exposure, and decision.

Inspect disagreement type

Separate occurrence, prompt, duration, category, visibility, event-matching, and missing-record discrepancies.

Use supportive follow-up

Calibrate definitions and systems, observe again, and protect employment and supervision fairness.

Design pair coverage before reviewing results

List the observer pairs needed for the decision and the conditions each pair should sample. Rotate pairs across comparable routines when feasible. If one pair observes only complex community activity while another sees structured clinic trials, pair identity and condition are confounded. The table should show that limitation before anyone compares percentages.

Use pseudonymous IDs in the analysis file and restrict the identity key to the roles that need it. Record training version, observer role, client and setting exposure, and any supervisory relationship. Pair analysis can guide calibration and system repair. Employment action requires its own fair process, adequate evidence, and applicable policy or law.

Reconcile Farid's weighted result

For Farid, the three pair numerators sum to 18 + 8 + 6 = 32 agreements, and the denominators sum to 20 + 10 + 10 = 40 matched trials. The weighted pooled result is therefore 32 of 40, or 80%. Averaging the three percentages equally would produce 76.7% and would give each ten-trial sample the same weight as the twenty-trial sample. Preserve the raw counts so the aggregation method remains visible.

The BC pair has 6 of 10 agreements. That small result identifies a follow-up priority. It cannot show whether B, C, their shared training, the sampled condition, or the recording system produced the pattern. Review disagreement categories and collect comparable samples before drawing a personnel conclusion.

Compare disagreement signatures

Count the kinds of disagreements within each pair. A concentration of onset-timing differences suggests a different repair than mismatched response categories or missing records. Check whether the same observer shows the same discrepancy across more than one pair and comparable conditions. Also inspect whether a definition, device, visibility problem, or access change affected everyone.

Report the pair-specific numerator, denominator, date range, conditions, and unresolved units. A pair with ten trials has wide practical uncertainty, so a single new disagreement changes the result by ten percentage points.

Apply the field control to one defined question

The observer-pair analysis for Farid begins with the exact decision, the responsible qualified role, and the evidence needed. Preserve original observer records, matched exposure, the selected calculation, raw numerator and denominator, unrounded result, and every excluded or unresolved unit.

Work the example for Farid

Farid's pairs score 40 matched trials: AB agrees on 18 of 20, AC on 8 of 10, and BC on 6 of 10. Pair results are 90%, 80%, and 60%; pooled agreement is 32 of 40, or 80%. The pooled figure hides the BC pattern, while ten trials still call for cautious follow-up.

Audit the denominator for Farid

The denominator for Farid is the eligible cohort defined before review. Report planned, due, observed, matched, invalid, missing, and open units separately. A percentage gains meaning from the unit, coverage, dates, conditions, and disagreement pattern.

Protect access and safety during the sample for Farid

The observation for Farid keeps AAC, interpreters, mobility, food, water, bathroom use, prescribed care, health supports, rest, relationships, and emergency help available. Observer procedure never delays immediate safety or mandated action. Record changes in access, health, assent, distress, visibility, staff behavior, or routine.

Use current sources within their scope for Farid

For Farid, the BACB ethics hub points to current ethics materials. The CASP public summary supplies high-level autism-treatment scope. The BACB Ethics Code addresses competence, client involvement, consent and assent when applicable, documentation, supervision, risk, and evaluation for covered behavior analysts. The BCBA Test Content Outline includes measurement, reliability, sampling, and data-based decisions as examination content.

For Farid and the pair-specific IOA patterns question, Vollmer, Sloman, and St. Peter Pipkin discuss practical implications of data reliability and treatment-integrity monitoring. Essig, Rotta, and Poling review IOA and fidelity reporting in an identified research corpus. Reed and Azulay describe agreement calculations and a calculation tool. ASHA says AAC users should always have access to their tools or devices. The responsible clinician still selects a case-specific measurement and review plan.

Close the IOA review for Farid

Review the observer-pair analysis with Farid through accessible communication and with the responsible clinician. Record the question, sample, method, arithmetic, disagreement types, limits, repair, owner, next observation, and review date. Reopen the review when the definition, observer, setting, system, exposure, access, prevalence, procedure, or decision changes.

Before release, confirm that every sample appears under one pair, pair conditions are comparable or clearly labeled, and pooled arithmetic uses the underlying counts. Check that follow-up addresses definitions, training, visibility, and systems before assigning responsibility to an individual.

Related resources

Sources