What if ABA Assessment Results Differ Across Settings?

ABA Results Across Settings

Build a comparison table with setting, date, people, task, definition, opportunity, support, prompt, AAC, health context, raw count, missing data, and observer. Preserve each source before creating a combined measure.

Differences are information, not proof of error

A person may communicate, participate, tolerate demands, or use a skill differently at home, school, clinic, work, or in the community. The settings can have different people, expectations, sensory conditions, communication partners, materials, histories, and consequences. A difference invites a closer comparison before anyone decides that one observer is wrong.

Ask the clinician to preserve setting-specific results. Combining them into one percentage can hide the condition in which support was effective or access was missing.

Make definitions and opportunities match

Two teams may use the same label for different responses. “Asked for a break” might mean speech only in one setting and speech, sign, or AAC in another. “Independent” may allow a visual cue in one record and exclude it in another.

Define the response, opportunity, prompt, scoring window, and exclusion before comparing. Show raw counts beside percentages. A rate of 2/3 across a ten-minute activity should not be treated like 8/12 across a full day without explaining the exposure.

Compare ordinary supports

List whether the person had AAC, glasses, hearing supports, mobility access, visual schedule, familiar vocabulary, medication, pain care, sensory accommodations, breaks, wait time, and a trained communication partner. The absence of an ordinary support can change what the assessment observed.

Access failures should not be scored as client skill failures. If AAC was unavailable, record that system condition and repeat the observation with the person’s usual communication when appropriate. Do not remove a useful support merely to create a “pure” test unless a qualified professional has a safe, justified assessment purpose and applicable consent and assent.

Ask the client what feels different

The person may identify noise, pace, people, uncertainty, pain, task difficulty, fear, privacy, or another feature that observers missed. Offer speech, sign, writing, AAC, gesture, or another reliable form. A caregiver or teacher can add observations without replacing the client’s response.

Record the setting and conditions of the client’s report. One comment may generate a useful assessment question without proving a cause.

Use matched observations carefully

A matched observation can hold definitions and key supports more consistent while examining performance in two settings. Predefine the comparison, keep the measurement window clear, and document concurrent changes. Avoid staging distress or removing essential supports to make settings look alike.

Even a better-matched comparison has limits. Time, history, relationship, novelty, and unmeasured variables can remain. The clinician should describe what the pattern supports and what it cannot establish.

Let differences shape the next step

A setting difference may lead to partner training, communication-access repair, environmental modification, a medical or interdisciplinary referral, more observation, or a setting-specific plan. It may also show that a goal is not useful everywhere.

Ask whether the recommendation aims to change the person, improve the setting, teach partners, preserve an effective support, or gather more information. The client’s priorities and the burden of generalization efforts should be part of that decision.

Avoid ranking settings by one result

A higher percentage in one place does not prove that the setting, staff, or caregiver is better. The person may encounter easier tasks, fewer opportunities, stronger preferences, more familiar partners, or different measurement. A lower value can appear in a setting that offers more meaningful challenges and opportunities.

Describe the conditions and result before making a quality judgment. If the concern is implementation, gather direct evidence about whether the agreed supports and procedures were present. Do not infer poor effort from the outcome alone.

Build a matched comparison table

Use one row per setting and columns for response definition, opportunity definition, dates, number of opportunities, people, activity, AAC and other supports, prompt level, wait time, health context, raw result, and missing data. Add the client’s report in its own attributed field.

Mark which columns truly match. The table may reveal that the settings are too different for a direct comparison, which is itself useful. It can guide the next observation without forcing the data into one combined rate.

Recheck after changing a system condition

If the team restores AAC, trains a partner, changes materials, or adjusts the environment, predefine the next measurement and retain the prior phase. Ask whether the change is an access repair, assessment condition, or intervention. Those labels have different implications.

Report the new raw counts and concurrent changes. A better result after several repairs can support continued use of the improved setup while remaining insufficient to isolate one cause.

Preserve the original setting-specific results so the comparison remains auditable and future teams can see which access conditions changed.

Keep assessment authority and evidence clear

The CASP public summary places assessment and planning within its autism-treatment scope. The BACB Ethics Code addresses competence, understandable communication, client involvement, consent and assent when applicable, assessment, documentation, and evaluation for covered behavior analysts.

Make feedback accessible

The ASHA AAC portal supports continuous AAC access. For covered private practices, DOJ Title III guidance addresses effective communication and reasonable modifications, subject to scope and defenses.

Use the correct record route

For a HIPAA covered entity, HHS access guidance describes access to protected health information in a designated record set, subject to the rule's scope and procedures.

A feedback request, draft review, final report, and formal access request are different events.

A practical example

Kai independently asks for a break in 8/10 clinic opportunities and 2/8 community opportunities. The clinic accepts speech or AAC within ten seconds, while the community partner uses unfamiliar vocabulary and waits only five seconds. The device also opens to a different page in the community setting.

The team keeps the first results as setting-specific observations rather than calling the community score a loss of skill. It trains the partner, restores the familiar vocabulary, and uses the same ten-second window. Across a later eight community opportunities, Kai requests a break in 6/8. The change supports another review while several conditions changed together, so the team avoids claiming that partner training alone caused it.

Questions families can use

Ask which definitions, opportunities, and scoring windows match; who observed; whether the client had the same communication, supports, and wait time; what changed in health or routine; what the client reports; whether a matched observation is safe and useful; and how the setting difference changes the next decision.