Caregiver training observer agreement checks whether two observers applying the same operational definitions score the same events. It is separate from caregiver fidelity. Families can ask how observers were trained, whether scoring was independent, which conditions were sampled, how agreement was calculated, and whether disagreements changed the definition. High agreement cannot repair a vague skill, biased opportunities, or a goal that lacks value.
Check caregiver training observer agreement
Define each step and response with examples and nonexamples. Calibrate observers on the same practice records, then sample agreement across relevant people, routines, difficulty levels, and stages. Report observer agreement beside caregiver implementation, not inside the same numerator.
Interpret performance in context
Agreement based only on easy role-plays may miss ambiguity in the actual routine. Observers can also share the same mistaken interpretation. Review disagreements by step and context. If scores influence mastery, safety, or plan changes, the team needs enough representative checks to trust the measure.
Use current ethics and caregiver-training evidence
The systematic review documents weaknesses in the caregiver BST literature and supports transparent method reporting. Hassan and colleagues distinguish ex-situ mastery from natural-environment performance. The BACB ethics hub directs readers to current competence, documentation, and evaluation requirements.
Preserve client communication and family choice
The ASHA AAC portal supports continuous communication access. For this training decision, preserve the client's established communication, assent and dissent, and ordinary supports while allowing the caregiver to ask questions, disagree, pause, or request a different method.
A practical example
Two observers independently score eight routine opportunities. They agree on seven and disagree on whether a delayed response counts. The team clarifies the timing rule, recalibrates, and reports seven of eight before the definition change rather than overwriting it.
Questions families can use
What was scored? Were observers independent? Which settings were sampled? Where did they disagree? Did the definition change? Is agreement reported separately from caregiver performance?
Sources
- Behavior Analyst Certification Board, Ethics Codes
- Council of Autism Service Providers, ABA Practice Guidelines Version 3.0 public summary
- Hsieh and colleagues, Caregiver Implementation of Incidental Teaching
- Preas and Mathews, Caregiver Training for Daily Living Skills
- Hassan and colleagues, Behavioral Skills Training for Caregivers
- Behavior Skills Training for Family Caregivers: A Systematic Review
- American Speech-Language-Hearing Association, Augmentative and Alternative Communication
Finni resources
