How Should ABA Treatment Fidelity Be Compared Across Settings?

ABA fidelity across settings

Create one row per setting with plan version, dates, implementer role, observer role, eligible opportunities, completed critical steps, prompts, missing supports, client feedback, and outcome. Keep a setting out of a pooled percentage when the procedure, denominator, or observation method differs materially.

A cross-setting comparison can help a team locate a support that works differently at home, in a center, at school, or in the community. It can also create a misleading ranking when one setting has more observations, an easier opportunity definition, different plan instructions, or fewer barriers. Begin with the conditions in each setting before comparing percentages.

Match the measurement before comparing results

Check these fields side by side:

FieldQuestion to resolvePlan versionWere staff following the same dated instructions?Scored componentDid “correct” mean the same observable action?OpportunityWhat event made a trial eligible in each setting?Response windowDid observers allow the same amount of time?SupportsWere AAC, materials, health supports, and ordinary accommodations available?ObserverWere observers trained to apply the same scoring rule?CoverageHow many staff, visits, times, and opportunities were sampled?Client experienceDid the client want the same activity and report similar fit?

If any field differs, describe the difference. A clinical team may still learn from the data, while a direct percentage comparison may no longer answer the intended question.

Define the scored components

Differences can locate a support need. They can also reflect unequal observation coverage. Ten home observations and one center observation provide different certainty. Check who was present, whether ordinary supports were available, and whether the client wanted the same goal and procedure in both places.

Preserve setting-specific context

Home may have familiar partners and materials. A center may have more controlled space and immediate supervision. School and community settings can bring different schedules, noise, peer presence, transportation, privacy, and rules. Those conditions are part of implementation, not background details to discard.

Ask whether a procedure should be identical across settings. Some goals need the same core response and safety boundaries with setting-specific materials or partner actions. If the qualified clinician approves different instructions, each version needs its own fidelity definition. Staff should be scored against the version that actually governs that place and time.

Compare observation coverage as its own measure

Suppose the plan calls for four observations in each setting. Home completes four and center completes one. The fidelity rates describe observed opportunities, while coverage is 4 of 4 at home and 1 of 4 at the center. A high center score from one observation provides less evidence about consistency than four observations spread across staff and days.

Report missing and overdue observations by setting. Avoid replacing missing data with zero or success. The practice can assign an owner and new due date without pretending the sample is complete.

Use clinical and measurement sources carefully

The CASP public summary places assessment, planning, implementation, and evaluation within its autism-treatment scope. The BACB Ethics Code addresses competence, client involvement, consent and assent when applicable, risk, documentation, and evaluation for covered behavior analysts.

The BCBA Test Content Outline includes procedural fidelity as examination content. It supplies a professional knowledge map rather than a case protocol, legal standard, or universal percentage threshold.

Read the evidence with its limits

A 2024 practitioner guide recommends defining observable components and valid opportunities. A reporting review found fidelity reporting was less consistent than observer-agreement reporting in the literature it examined.

Keep communication and access visible

The ASHA AAC portal supports continuous access to communication tools. Fidelity records should show whether communication, health, mobility, sensory, and other required supports were available during the scored opportunity.

Do not treat an access need as evidence that the client or family caused lower fidelity. If the center has a tested AAC backup and the community visit does not, the comparison may reveal an operational gap. The repair belongs with the system responsible for making the support available. The client's communication method remains valid in every setting.

A practical example

Luis's plan has six scored steps per eligible activity. At home, staff complete 32 of 36 due steps across six activities, or 88.9%. At the center, staff complete 15 of 18 across three activities, or 83.3%. The team reports the settings separately.

The component record shows that all three missed center steps involve giving Luis access to the quiet exit before the activity begins. Home has a permanent quiet space, while the center relies on a room that is sometimes occupied. The difference points to a facility and scheduling barrier. It does not show that center staff are generally less skilled or that Luis behaves better at home.

Operations reserves an accessible space, and the clinician confirms that the same exit rule applies. The next center sample covers four activities with two staff members. The exit is available in 4 of 4, and all due critical components occur in 23 of 24 steps. Luis's comfort rating and participation are reported separately. The team documents that multiple conditions changed, so the follow-up supports the repair without proving a single cause.

Decide whether pooling adds value

A combined rate can be useful for a narrow operational question when the plan, definitions, periods, and observation methods match. It can hide the very pattern the team needs when settings differ. Show setting counts first. If a pooled result is reported, name its numerator and denominator and explain why combining them answers a meaningful question.

Families can ask for action by setting: what needs repair, who owns it, when it will be checked, and whether clients already receiving care need a continuity plan while a required support is unavailable.

When different settings need different plans

Consistency does not always mean identical materials or partner behavior. A community setting may need a travel and emergency route that is irrelevant at home. School may use another communication partner or schedule. The clinician can preserve the goal and core safety or communication boundaries while authorizing setting-specific instructions. Each version should identify its setting, effective date, components, and training requirements. Compare results within each version first. A later cross-setting review can ask whether the person reaches the valued outcome with ordinary supports, without forcing one environment to imitate another.

Questions families can use

Ask whether the same plan and definitions applied, how many opportunities were observed, which supports differed, what the client preferred, which components varied, and whether pooling would hide a setting-specific need.