Can ABA Observer Agreement Be Checked From Video?

ABA observer agreement video

Record the approved purpose, people and setting captured, device and system, start and end, access list, retention date, deletion owner, observer independence, viewing conditions, plan and definition version, and camera limitations. Consent to services does not automatically answer every recording or secondary-use question.

Video can be useful when events are brief, two live observers would be disruptive, or the same exact period must be scored independently. It also creates a new sensitive record. The decision should weigh measurement value against privacy, security, client comfort, setting rules, and the possibility that the camera changes ordinary behavior.

Decide whether recording is necessary

Ask which question requires a recording. A live paired observation, existing authorized system record, fictional calibration clip, or another measure may answer it with less collection. If video is selected, limit the view and duration to what the definition requires.

The plan should identify:

• the specific measurement and decision
• who is recorded and which setting is included
• the person authorized to approve the recording and access
• how the client can communicate assent, dissent, discomfort, or a request to stop when applicable
• the approved device, account, storage, transmission, and access route
• whether replay, still images, audio, downloads, or secondary uses are allowed
• the retention event and person responsible for deletion

Other laws, payer terms, school rules, employment requirements, or contracts may apply. Route uncertainty to the responsible privacy, legal, clinical, or operational role before recording.

Check what the result can support

Video creates a stable observation window, yet it may omit off-camera communication, sound, materials, or partner actions. Observers should mark unscorable periods rather than guessing. A recording selected because it looked clear or successful may also overstate routine measurement quality.

Check whether the camera captures the definition

A recording can preserve time while losing context. Camera angle, resolution, microphone position, lighting, screen glare, network delay, and compression can affect what observers see or hear. If the measure depends on a partner response, device output, physical contact, or event elsewhere in the room, the recording must capture that element or label it unscorable.

Do not change the environment solely to make scoring easier when the change alters the behavior or support being measured. Record the camera's location and any difference from ordinary care. A high agreement score from a staged view may say little about routine observation quality.

Keep the two scorings independent

Give both observers the same approved file, observation window, playback rules, definition, and time source. Decide in advance whether pausing, slow motion, or repeated viewing is allowed. Those tools can improve event detection while producing a measure that differs from live observation.

Each observer should lock the record before comparison. If one person supplies timestamps or clips only the events another person should score, the second observer may be cued. That process can support training or accuracy review, while it is weaker evidence of independent agreement.

Protect the recording through deletion

Use role-based access and an approved system. Avoid personal devices, consumer file sharing, or local downloads unless the responsible policy expressly permits them. Record who accessed the file and investigate any suspected unauthorized access through the applicable incident process.

Deletion should cover authorized copies and exports according to the retention decision. A scheduled deletion date means little without an owner and completion evidence. If a hold or other requirement changes the date, record the reason and new disposition.

Use current clinical and measurement sources

The CASP public summary places assessment, planning, implementation, and evaluation within its autism-treatment scope. The BACB Ethics Code addresses competence, client involvement, consent and assent when applicable, documentation, and data-based evaluation for covered behavior analysts.

The BCBA Test Content Outline includes measurement, data integrity, observer agreement, and visual analysis as examination content. It does not set one calculation, sampling percentage, or action threshold for every case.

Choose the calculation for the measure

Vollmer and colleagues discuss practical consequences of data reliability and treatment-integrity monitoring. Reed and Azulay describe several agreement calculations. Method choice depends on the measurement system and the question being asked.

Keep communication and conditions visible

The ASHA AAC portal supports continuous communication-tool access. Observer agreement is meaningful only when the record also identifies communication access, ordinary supports, opportunity availability, and other conditions that affect what could be observed.

Positioning the camera should never block the person's AAC, movement, privacy, or view of communication partners. If the screen itself contains private information, capture only what the approved measurement requires. When the device output is necessary for scoring, explain that collection and protect it as part of the recording.

A practical example

A ten-minute authorized recording contains eight defined communication opportunities. The plan allows pausing once per opportunity and requires both observers to use the same time-stamped file. Two responses occur outside the camera view, leaving six scorable opportunities.

The observers independently score the six and agree on five, so agreement is 5 of 6, or 83.3%. The report also states that 2 of 8 opportunities, or 25%, were unscorable because of framing. It does not convert the two missing observations into disagreements or remove them from the coverage record.

The practice repairs the camera position before another sample. Access logs show only the two approved observers and supervisor viewed the file. The deletion owner confirms deletion on the approved date and records completion. The family-facing summary includes the result, camera limitation, repair, access group, and file disposition without exposing unrelated workforce information.

Questions before agreeing to video

Ask why video is needed, what alternatives were considered, what the frame and audio collect, how the client can stop participation, who can view or download it, which playback rules apply, how unscorable periods are handled, when the file is deleted, and how the practice proves that disposition.

Avoid choosing only the clearest clips

Define the selection rule before reviewing recordings when possible. A convenience sample of successful, well-framed visits may be useful for calibration while overstating routine measurement quality. Report how many recordings were eligible, selected, excluded, and unscorable, with reasons. If six of ten eligible files are scored, coverage is 6 of 10 even when agreement within those six is high. Keep the four omitted files visible and decide whether their camera or workflow failures need their own repair.

Questions families can use

Ask why recording is needed, who authorized it, what the camera captures, who can view it, how observers stay independent, which periods are unscorable, and when the file will be deleted.