Compare Treatment Integrity Across Implementers

To compare treatment integrity across implementers, first align the procedure version, eligible opportunities, clients, settings, risk, supports, observation method, sample size, observer agreement, and missingness. Report raw counts and uncertainty. Differences can reflect assignments or systems as well as skill. Use comparisons to locate learning and design questions, not to rank people from unmatched conditions.

Create comparable cells

Group observations by active version, client, routine, setting, opportunity type, risk, and ordinary support. Do not pool conditions solely to increase sample size.

Add observation mode, announcement status, supervisor presence, and stage of training when those conditions could change performance. Define the comparison before viewing names or results when feasible. If staff served different clients or routines, report separate cells or matched subsets rather than implying an individual difference. Preserve unmatched work because it may reveal an assignment or system problem.

Create a comparability table that states which cells can be compared, which need adjustment or stratification, and which remain descriptive only. Include valid adaptations and whether each implementer had the same access to instructions and materials. When there is no fair matched cell, report that limitation plainly rather than manufacturing a rank from incomparable work.

Show exposure and sample size

Report opportunities served, opportunities observed, components due, and missing observations for each implementer. Small samples remain visibly small.

Distinguish procedure exposure from observation exposure. A person may have implemented many sessions but been observed only once, or may have received the new version late. Report raw numerators and denominators, settings, dates, and agreement coverage beside each rate. Avoid ranking people whose samples differ materially in size, difficulty, visibility, or maturity.

Display confidence in words the audience can use, such as limited evidence from one routine or repeated evidence across three settings, instead of implying precision from a percentage alone. Keep missed observations and reasons visible. If sampling is systematically heavier for one person, correct the observation plan before using the comparison to target individual coaching.

Inspect assignment and system effects

Review schedules, materials, staffing, supervision, training access, technology, and client complexity. These factors can produce apparent implementer differences.

Ask whether one person received higher-risk routines, fewer supplies, less overlap, more urgent substitutions, or only unannounced observations. Examine procedure clarity and whether the client had the same communication and access supports. Treat missing infrastructure as an accountable system condition. Do not convert a staffing, version, or observation-design difference into an unsupported conclusion about competence.

Use the comparison for action

Select shared training, individual coaching, environmental repair, procedure redesign, or more balanced sampling. Keep employment decisions in their authorized process.

Choose an intervention based on the supported cause and state what follow-up will test. A widespread error may call for procedure clarification or system redesign, while a role-specific skill gap may support coaching. Include client protection, access, and feedback. If employment review is warranted, use its own evidence, notice, and authority rather than treating the clinical comparison as an automatic disciplinary scorecard.

Put the cross-implementer comparison into practice

Nabil's comparison separates three staff by the procedure version and client opportunities they actually encountered. One staff member served higher-risk routines with fewer materials, while another had announced observations only. The review matches comparable cells, reports unmatched work, and routes material failures to operations before coaching conclusions.

Keep supervision, quality, and employment roles separate for Nabil

Nabil's cross-implementer integrity comparison identifies the clinical decision owner, observer, supervisor, quality reviewer, operations owner, privacy contact, and employment owner. State which process is active, what notice and evidence it uses, who may decide, and where the record belongs. One integrity observation should not silently change purpose after collection.

Protect client access during Nabil's review

Nabil's process keeps AAC, interpreters, mobility, food, water, bathroom use, prescribed care, health support, rest, relationships, and emergency help available. Obtain required consent and assent when applicable and follow the governing response to withdrawal or distress. Client protection can require immediate action while broader measurement and personnel conclusions remain pending.

Use raw evidence and matched denominators for Nabil

Report Nabil's correctly implemented components divided by eligible components due, observations completed divided by observations due, agreement pairs divided by pairs due, and repairs verified divided by repairs due. Keep procedure versions, critical steps, exposure, client outcomes, direct experience, access, missingness, adaptations, drift, burden, and adverse effects in separate series.

Ask six supervision and review questions for Nabil

Use these questions in the cross-implementer integrity comparison:

Which client decision, procedure version, component, and eligible opportunity apply?
Which people, settings, risks, supports, observation modes, and sample limits matter?
Which integrity, agreement, exposure, outcome, experience, access, and adverse-effect series stay separate?
Which skill, clarity, resource, workflow, safety, burden, supervision, or plan-fit barrier is supported?
Which clinical, quality, operations, privacy, employment, certification, or payer owner may act?
Which repair, stop rule, follow-up evidence, and review date close the loop?

Keep unresolved evidence visible with a state, owner, age, and next action.

A fictional supervision example for Nabil

Nabil is fictional and involved in a community recreation program with three staff and different client opportunities. Reviewers freeze 34 version, opportunity, client, setting, risk, support, sample, observer, agreement, and comparison fields and complete 24 of 34 by the checkpoint. Missing, unobservable, inapplicable, adapted, drifted, disputed, failed, and pending fields keep their defined states.

The cross-implementer integrity comparison measures evidence completeness. It does not establish effectiveness, safety, acceptability, competence, employment performance, certification compliance, medical necessity, authorization, payment, generalization, maintenance, or causation.

Use current sources within their scope for Nabil

For Nabil's cross-implementer integrity comparison, the CASP public summary supplies high-level individualized assessment, implementation, and evaluation scope for ABA treatment of autistic people. The BACB Ethics Code addresses competence, client involvement, consent and assent when applicable, supervision, risk, documentation, and evaluation for covered behavior analysts. The BACB supervision page routes readers to role-specific supervision and training resources; those requirements do not replace licensure, payer, employment, or case authority.

For Nabil, the BCBA Test Content Outline addresses procedural fidelity as examination content; case protocols require qualified case-specific authority. The Ferguson practitioner guide supports observable components and eligible opportunities. The Essig, Rotta, and Poling review supports caution in fidelity-report interpretation. ASHA says AAC users should always have access to their tools or devices. Set coaching triggers, comparison rules, and performance thresholds from the current case and setting.

Close Nabil's review

Ask Nabil, the implementer, and the responsible clinician to review the cross-implementer integrity comparison through accessible communication. Record the evidence, perspectives, limitations, selected repair, responsible system, clinical decision, follow-up observation, client outcome and experience, and review date. Reopen the analysis when the procedure, client priority, access, health, setting, implementer, supervisor, risk, or outcome changes.

How to Compare Treatment Integrity Across Implementers