Machine learning

Machine learning finds patterns from data

Traditional software often applies rules written by people. A machine-learning model estimates patterns from examples or feedback. It may predict a cancellation, group an incoming document, flag a claim, rank outreach, transcribe speech, or generate text.

The NIST glossary describes machine learning as techniques that improve performance by exposure to data. The label says little about reliability. Model type, data, target, threshold, workflow, and consequences determine the practical risk.

Start with one named use case

Record the decision the model supports and what happens when it is wrong. Sorting a public inbox differs from proposing clinical content or prioritizing access to care.

For each use, name:

• intended users, affected people, setting, and purpose
• input data, target or output, exclusions, and prohibited uses
• model and service version, vendor, owner, and change process
• human review point, final decision-maker, and appeal or correction route
• validation cohort, error measures, subgroups, thresholds, and abstention rule
• monitoring, incident, stop, fallback, retention, and retirement controls

Do not approve a brand name for every possible use. A product can contain several models and support uses with different stakes.

Training data shapes the result

Labels can reflect inconsistent documentation, historical access, payer policy, staff behavior, or missing observations. A model may learn these patterns while appearing mathematically precise. Variables can also act as proxies for disability, language, geography, income, or prior access.

NIST SP 1270 explains that bias can arise from data, institutions, human decisions, and the broader context. Review how examples were selected, labeled, excluded, and updated. Vendor assurances do not replace local testing.

Validate the deployed workflow

Lock a representative evaluation cohort before tuning thresholds. Compare output with a credible reference determined by the qualified role. Report false positives, false negatives, abstentions, missing cases, and performance for relevant groups and settings.

Measure the complete workflow, including data extraction, model output, presentation, human review, decision, and downstream action. A model can score well while a confusing interface sends staff toward the wrong choice.

Compare the model with a useful baseline

A complex model should earn its place against a simple rule, current workflow, or qualified human process. Predeclare the comparison, evaluation period, eligible cohort, outcome definition, and operational cost. Include review time, false-alarm work, delayed action, integration failures, and cases where the model abstains.

Choose measures that fit the consequence. Precision and recall may matter for a flagging task. Calibration may matter when a score is presented as probability. Time saved may matter for an administrative draft, provided correction burden and missed work are counted. Keep the decision threshold separate from model performance because changing the threshold changes the mix of errors. A higher score on one metric may still produce a worse workflow.

A fictional prediction review

DeShawn's practice evaluates a waitlist-outreach model on 80 mature fictional records. Seventy-two have complete inputs and a reviewable reference outcome: 72 of 80, or 90%. Eight missing records remain visible and are analyzed by cause.

The practice reports errors across all 72 reviewable records and by predeclared language, location, and referral-route groups. It does not publish one “accuracy” number or use the model to close referrals. The example describes an evaluation design, not benefit, fairness, or causal impact.

Keep people able to disagree

Human review works only when a reviewer can see source evidence, recognize model involvement, change or reject the output, understand the criteria, and retain authorship for the final decision. Time pressure, workload, and polished language can weaken scrutiny.

Provide a manual workflow and an accessible route for clients, families, and workers to correct data or question a decision. High-stakes clinical, access, payer, employment, or safety decisions need authority and safeguards from their governing sources.

Monitor drift and change

Performance can change when populations, documentation, payers, workflows, thresholds, integrations, or the model itself changes. Track input quality, error patterns, overrides, subgroup results, incidents, latency, and failed connections. Predefine the conditions that pause use.

The voluntary NIST AI Risk Management Framework organizes work around Govern, Map, Measure, and Manage. Use it as a structure while qualified owners apply legal, clinical, payer, privacy, and security requirements.

If a cloud vendor maintains ePHI for a covered entity or business associate, HHS cloud guidance explains that business-associate duties can apply even without the decryption key. Map prompts, files, outputs, logs, support access, training use, subprocessors, and deletion.

At every material change, repeat the relevant validation before broadening use. Preserve the prior model, threshold, test cohort, reviewer decisions, production errors, and rollback path so the new version is evaluated rather than assumed equivalent.