Audit an AI-Assisted ABA Clinical Documentation System

Define Tomas's AI documentation system audit

Tomas audits the full sociotechnical workflow. Model performance alone cannot show whether sources were complete, reviewers had time, clients could communicate, vendors followed terms, or released records reached the right downstream systems. The unit identifies approved use, client and event, source records, model and version, vendor, user and accountable author, output state, reviewer, downstream use, exception, and evidence needed for release or closure.

Build Tomas's page-specific control record

Tomas records audit period, approved use and prohibited uses, model and deployment, vendor and subprocessors, prompt and retrieval versions, source records, users and roles, client and setting, communication and AAC, consent or notice when applicable, privacy and security controls, training and secondary-use settings, generated draft, edits, review time and decision, clinical and coding boundaries, released record, unsupported and omitted claims, accessibility and subgroup results, incident and complaint, downstream use, monitoring threshold, corrective action, owner, due date, retest, recurrence, decommissioning, and closure. Design, operation, and outcome findings remain separate.

Put Tomas's human-review boundary into practice

Tomas selects random released records, held drafts, near misses, complaints, corrections, different client communication methods, low-frequency settings, and recent model or prompt changes. Reviewers compare claims with locked sources and inspect actual user behavior, access logs, vendor settings, and downstream records. They measure invention, omission, distortion, misattribution, calculation, accessibility, privacy, and release-gate defects separately. Subgroup results are interpreted cautiously when samples are small and never used to stigmatize clients or staff. Immediate safeguards pause risky uses, restore human workflows, correct records, and communicate with affected people. Corrective action can change source quality, interface design, staffing, review, prompt, retrieval, vendor configuration, or tool choice. Documentation that everyone completed training does not prove the control operates. Tomas retests the exact defect and adjacent cases under the new version, then checks whether decommissioned models, accounts, data, and integrations truly stopped operating.

Protect client communication and ordinary access for Tomas

Tomas preserves the client's direct communication, AAC, language and disability access, consent and assent when applicable, dissent, health, safety, privacy, priorities, and correction route. AI use never makes communication, food, water, bathroom access, mobility, prescribed care, rest, or emergency help conditional on tool participation or task performance.

Work through Tomas's fictional example

Tomas locks 60 trace units. Forty-seven pass. Thirteen contain sixteen findings across source completeness, wrong attribution, omission, privacy settings, reviewer bypass, accessibility, coding suggestions, and downstream correction. Ten validate after remediation; three remain open. The cohort teaches AI governance and denominator discipline. It does not establish treatment effect, model safety, legal compliance, coding correctness, payer acceptance, accessibility, or product performance.

Keep Tomas's denominator tied to the locked population

Initial trace-unit integrity is 47 of 60, or 78.3%. Final validation is 57 of 60, or 95.0%. Sixteen findings across thirteen units remain separate. Released records, held drafts, incidents, and client complaints each keep their own population.

Assign Tomas's decisions to accountable people

Auditors test evidence. Authors and qualified clinicians own records and care. Clients evaluate communication and experience. Privacy, security, legal, coding, billing, procurement, and technical leaders own scoped controls. Independent reviewers validate high-risk closure.

Address Tomas's main AI documentation risk

An audit of only released successes misses blocked, abandoned, silently edited, and never-detected failures. Reconcile all use events before sampling.

Test Tomas's workflow with difficult cases

Tomas tests approved and prohibited use, different models and prompts, source gaps, wrong client, accessibility, vendor settings, reviewer bypass, incident response, correction propagation, monitoring alert, and decommissioning.

Check Tomas's release evidence

Tomas confirms the exact source set and versions, client and encounter, model and configuration, approved data route, generated draft, material edits, author and reviewer decisions, accessible client communication, release destination, correction path, monitoring cohort, and known limitations. The AI documentation system audit retains unresolved work, owner, deadline, downstream trace, and the next revalidation trigger.

Use Tomas's ABA governance sources within their scope

Tomas uses the CASP public overview only for high-level organizational context. The BACB Ethics Code applies to BCBA and BCaBA certificants and applicants as defined by the Code; BACB has no separate jurisdiction over organizations or corporations. These sources support competence, documentation, confidentiality, client involvement, assessment, intervention, risk, supervision, and correction boundaries. They do not approve a tool or transfer clinical authority to software.

Keep Tomas's source record and medical-review boundary visible

Tomas uses current CMS Program Integrity Manual Chapter 3 as Medicare medical-review guidance. It currently says services are expected to be documented when rendered; delayed or corrected entries may occur; date and author should be identifiable; and changes or addenda clearly and permanently noted. AI output cannot supply facts that were not documented, and Medicare guidance does not become a universal payer, state, or AI rule.

Map Tomas's privacy, security, and vendor roles

Tomas uses the current HHS Security Rule overview, HHS cloud guidance, and HHS business-associate guidance to analyze actual covered-entity, business-associate, subcontractor, cloud, and security roles. A BAA or vendor certification does not complete purpose, permissible-use, minimum-data, configuration, risk analysis, access, incident, retention, and shared-responsibility work.

Use Tomas's AI frameworks as voluntary risk tools

Tomas treats the NIST AI Risk Management Framework and NIST Generative AI Profile as voluntary risk-management resources, not clinical, legal, coding, or payer authority. The profile helps teams examine generative-AI risks and actions. A framework, score, benchmark, or vendor evaluation does not prove safety, accuracy, fairness, accessibility, compliance, or fitness for this ABA use.

Protect Tomas's data-purpose and communication boundaries

Tomas uses HHS de-identification guidance for its two HIPAA methods and residual-risk boundary. Calling output synthetic or removing names is not itself a method. FTC staff guidance warns AI companies to honor privacy and confidentiality commitments. The OIG GCPG is voluntary and nonbinding. ASHA's AAC portal says AAC users should always have access to their tools or devices.

Choose Tomas's next review trigger

Tomas reopens the AI documentation system audit after a model, prompt, retrieval, source, template, vendor, subprocessor, setting, language, client communication method, access role, data term, payer rule, incident, complaint, correction, audit finding, or regulation changes. The review records affected people and records, immediate safeguard, owner, deadline, communication, correction, downstream propagation, and validation.

Close Tomas's workflow without hiding uncertainty

Review the AI documentation system audit with affected clients and authorized people, qualified clinicians, health-information, privacy, security, AI-governance, accessibility, language, payer, coding, and technical leaders, and the specialists named in the manifest. Confirm source support, authorship, authority, client access, model provenance, vendor terms, errors, downstream use, correction, and independent validation. Keep this page draft and noindex until every required external review is complete.

Related resources

• Govern Speech-to-Text Dictation in ABA Clinical Documentation.
• Respond to an AI Documentation Error or Hallucination in ABA Records.
• Govern AI-Assisted Drafting of ABA Clinical Notes and Reports.
• Control Vendor Model-Training and Secondary-Use Rights for ABA Clinical Data.

Sources

• Council of Autism Service Providers, Organizational Guidelines public overview.
• Behavior Analyst Certification Board, Ethics Code for Behavior Analysts.
• Centers for Medicare & Medicaid Services, Medicare Program Integrity Manual, Chapter 3.
• U.S. Department of Health and Human Services, HIPAA Security Rule.
• U.S. Department of Health and Human Services, Guidance on HIPAA and Cloud Computing.
• U.S. Department of Health and Human Services, Business Associates.
• U.S. Department of Health and Human Services, Guidance Regarding Methods for De-identification of Protected Health Information.
• National Institute of Standards and Technology, AI Risk Management Framework.
• National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile.
• Federal Trade Commission staff, AI Companies: Uphold Your Privacy and Confidentiality Commitments.
• Office of Inspector General, General Compliance Program Guidance.
• American Speech-Language-Hearing Association, Augmentative and Alternative Communication.