Artificial intelligence

AI is a broad category

The NIST glossary describes AI as a machine-based system that can make predictions, recommendations, or decisions influencing real or virtual environments for human-defined objectives. In practice, the label covers varied methods and risk levels.

A rules engine may apply explicit conditions. A machine-learning system may learn patterns from examples. A predictive model may estimate an outcome. Generative AI may produce text, images, audio, or structured content. Some products combine several methods behind one interface.

Evaluate the workflow around the model

Risk comes from the full socio-technical system: problem selection, data, labels, prompts, model, interface, users, time pressure, review, downstream action, and affected people. A highly accurate model can still cause harm when applied to the wrong population or tied to an unsuitable decision.

Define one use case at a time. State what the tool may receive, produce, and influence. Document unsupported uses and conditions that require abstention or escalation.

Preserve professional authority

AI can assist with drafting, retrieval, classification, or pattern detection within an approved workflow. Qualified people retain decisions that require clinical, coding, legal, privacy, payer, or employment judgment. A system’s recommendation does not create informed consent, medical necessity, authorization, scope of practice, or claim payment.

The CASP AI parameters page describes ABA-sector guidance on organizational oversight, selection, deployment, change management, monitoring, and auditing. It is guidance rather than a product endorsement.

Validate with relevant evidence

Use authorized test data that reflects the actual task, language, formats, edge cases, and users. Predeclare measures and acceptance criteria. Depending on the use, examine omissions, fabrications, classification errors, subgroup results, reviewer agreement, overrides, latency, abstentions, security, and accessibility.

A human reviewer needs source access, time, competence, and real authority to change the output. Record the final author or decision-maker and retain the evidence required for the workflow.

A fictional drafting pilot

Priya’s practice tests a documentation-drafting assistant on 20 purpose-built fictional cases. Eighteen drafts contain every required section: 18 of 20, or 90%. Qualified reviewers judge 16 of those 18 usable after review under the predeclared rubric: 16 of 18, or 88.9%.

Original-cohort usable yield is 16 of 20, or 80%. Two incomplete drafts and two rubric failures remain visible by error type. These results describe this version, prompt, test set, rubric, and reviewers. They do not establish clinical accuracy, time savings, safety, or performance with real clients.

Generative output deserves special care

The NIST Generative AI Profile is a cross-sector companion to AI RMF 1.0. It identifies risks that generative systems can introduce or amplify and proposes lifecycle actions.

Fluent wording can conceal fabricated facts, unsupported citations, missing context, sensitive information, or harmful stereotypes. Require source verification, provenance where relevant, clear AI-origin labeling for reviewers, and tests of foreseeable misuse. Avoid using client data for model training or vendor improvement without an authorized, documented basis.

Monitor change after release

NIST AI RMF organizes voluntary risk management through Govern, Map, Measure, and Manage. NIST currently states that version 1.0 is being revised.

Track model and prompt versions, production errors, overrides, complaints, incidents, access, latency, and drift in the use population. Reassess after material changes. A safe stop rule should route work to a tested manual process. Retirement includes data return or deletion, access removal, open-case reconciliation, retained evidence, and notice to users who depend on the workflow.

Choose a first use with recoverable consequences

A narrow administrative use with easy source checking and a proven manual alternative usually gives a practice clearer evidence than a broad autonomous workflow. Define one output and one user group. Avoid combining drafting, clinical recommendation, authorization checking, and message delivery into a single pilot because a failure becomes difficult to locate.

Write an AI use card before procurement:

• the specific task, excluded tasks, and intended benefit
• data allowed and prohibited, including vendor reuse
• users, affected people, and required qualifications
• source evidence and human approval action
• validation cohort, measures, and minimum results
• error, incident, appeal, downtime, and stop routes
• model, prompt, configuration, and review versions

The practice should be able to answer who approved the use, what evidence supported release, and which output reached a real workflow. If the answer depends on a vendor’s broad capability statement, the operating boundary remains incomplete. Revisit the card when the model or surrounding process changes, and tell users which assumptions no longer hold.

Before renewal or expansion, review actual errors, overrides, incidents, subgroup results, complaints, and manual fallback use against the release criteria. Continue only with a named owner, current version evidence, and a tested stop route.