How Accurate Must ABA Self-Monitoring Be?

Set the self-monitoring accuracy ABA purpose requires

Name the decision the record supports. A rough reflection may guide a conversation. A safety or treatment decision needs stronger definitions, observation, and calibration. Decide what happens when accuracy is below the threshold, including simplifying the target or stopping use for that decision.

Accuracy is useful only in relation to a purpose. A personal mood note may help someone notice patterns even when entries are incomplete. A record used to adjust treatment intensity, document workplace performance, or decide whether a safety routine is ready needs a more controlled comparison. The team should state the stakes, required precision, review owner, and consequence of an uncertain result before asking the person to collect data.

Avoid one universal cutoff. A 90% agreement rule may sound objective, yet its meaning depends on the event rate and error type. Missing a rare safety event can matter more than adding an extra low-stakes entry. A large number of intervals where both record “nothing happened” can raise total agreement while occurrence scoring remains weak.

Define the error types that matter

Use matched categories:

• agreement on occurrence: both records show the event
• self-record only: the person records the event and the observer does not
• observer-record only: the observer records the event and the person does not
• agreement on absence: both records show no event in an eligible opportunity
• invalid comparison: the opportunity, timing, tool, or observer view does not support a fair match

Labels such as false positive and miss require a reference standard. When an observer is used as the comparison, call the categories self-only and observer-only until the team has reason to treat another record as authoritative. Video, device logs, physical products, or a second observer may help resolve selected questions, subject to consent, privacy, and setting rules.

For common events, the team may report occurrence agreement as agreements on occurrence divided by opportunities where either record shows occurrence. It can also report absence agreement separately. State the formula beside the result. Different agreement formulas answer different questions.

Compare matched events

The person and observer should score the same defined opportunities or intervals independently when feasible. Report agreements, self-record-only events, observer-record-only events, and both-record-absent events. A percent agreement can hide systematic misses when the event is rare.

Match the start and end of every comparison. If the person records after an activity and the observer stops earlier, disagreement may come from the window rather than the skill. Confirm that both can detect the event, access the cue, and use the same examples and nonexamples. Sampling should include relevant settings, times, task difficulties, and support levels rather than only convenient sessions.

Keep independence during the check when feasible. An observer who reminds the person to record, looks at the person’s entry first, or supplies feedback during the event changes what is being measured. Teaching trials can still be useful, but label them separately from independent accuracy checks.

Check the observer too

Train the observer on examples and nonexamples, sample across relevant conditions, and check observer agreement or drift when stakes warrant it. Disagreement opens a definition and access review. It should not become an automatic correction of the person's report.

The observer may miss quiet, private, internal, or client-authored events. The person may have better access to pain, effort, anxiety, intention, or an action outside the observer’s view. Conversely, device failures, memory demands, unclear cues, or complicated response options may affect self-recording. Review both measurement systems.

When the decision is high impact, a second trained observer or another independent source can test whether the comparison record is stable. Record who observed, their training, the sample, and any conflicts of interest. An employer, school, payer, or clinician may also have separate requirements for records used in its decisions.

Use measurement and ethics anchors

The BACB outline covers operational definitions, measurement validity, agreement, representative sampling, and self-management. The Ethics Code addresses client involvement, documentation, risk, and evaluation. The technology review supplies examples, not an accuracy threshold.

A practical example

Theo records whether a tool check occurred during ten eligible workshop transitions. The definition requires touching three named controls after the machine stops and before the next setup begins. Theo and a trained observer record independently.

They agree that the check occurred in six transitions and agree that it was absent in two. Theo alone records one, and the observer alone records one. Total agreement is 8 of 10, while occurrence agreement is 6 divided by 8 opportunities where either record shows occurrence, or 75%. Both numbers are reported because they answer different questions.

Review shows that one observer-only event involved a control blocked from Theo’s view, while the self-only event occurred after the observer ended the window. The team repairs the workstation cue and window definition before another check. It does not use this small sample to make a safety release decision. Theo’s report about effort and visibility remains part of the redesign.

A family checklist for accuracy review

• Which real decision will use the self-monitoring record?
• What are the consequences of a self-only or observer-only event?
• Are the event, opportunity, start, and end defined?
• Can both recorders detect the same event independently?
• Which agreement formula fits the event rate and decision?
• Are invalid comparisons and missing records visible?
• Has observer accuracy or drift been checked where stakes warrant it?
• What happens when the accuracy threshold is missed?
• Can the target, reminder, or response method be simplified?

Turn the result into a proportionate decision

An accuracy review should end with a named action. Low agreement may lead to a clearer definition, a more visible cue, fewer targets, a simpler response, better observer training, a repaired device, or another comparison sample. It may also show that the record is unsuitable for the intended high-impact decision.

High agreement supports only the conditions that were sampled. Report the settings, activities, supports, and dates. Continue ordinary checks when the governing decision warrants them, while reducing observation that adds little value. The person and family should receive the result in accessible language and know how to correct the record or raise concerns.

Questions families can use

Which decision uses the data? What accuracy matters for that decision? Was the observer calibrated? Are self-only, observer-only, absence, and invalid events separate? What happens after disagreement? Can the target or tool be simplified?