To route ABA AI work across models and providers safely, approve each route as a separate configuration. Match purpose, data class, capability, tool access, vendor terms, region, quality evidence, latency, cost, and human review before selection. Apply deterministic allowlists and hold rules outside the model, preserve route provenance, prevent silent failover to a weaker or unauthorized provider, and revalidate every material model, vendor, prompt, data, or integration change.

Treat every route as a governed configuration

Rafael inventories the use case, input class, vendor, model and version, region, prompt, retrieval corpus, tools, output, reviewer, downstream action, and fallback. The same prompt sent to two providers creates two data routes and two evaluated systems. A provider alias that changes its underlying model also creates change risk. Record approved, prohibited, experimental, fallback, and retired routes so runtime behavior can be reconciled with governance.

Route from explicit attributes

Use deterministic fields such as use-case ID, PHI status, data residency, language, document type, input length, required capability, tool need, latency class, and risk tier. The model should not decide whether it may see more sensitive data or gain a stronger tool. Default to a visible hold when no approved route matches. Reject user instructions or retrieved content that attempt to select a provider, expand permissions, or disable review.

Validate each model-provider pair

Run the representative and critical-failure tests for each route. Compare output quality, source support, abstention, accessibility, latency, availability, logging, correction, export, and stop behavior. A model that passes summarization may fail extraction, another language, or tool use. The voluntary AI RMF Core treats third-party components, monitoring, human oversight, and change management as connected risk concerns. A benchmark or vendor reputation cannot replace use-case acceptance.

Verify data and contract boundaries

HHS business-associate guidance determines status from actual function and PHI activity. HHS cloud guidance says a cloud provider maintaining ePHI can be a business associate even without the encryption key. For each route, verify the contract, BAA when applicable, subprocessors, regions, retention, training, secondary use, support access, incident notice, export, deletion, and termination. The FTC staff article reinforces that providers should honor privacy and confidentiality commitments.

Design safe fallback

Specify which failures permit retry on the same route, a tested alternate route, manual work, or a hold. Never send PHI to an unapproved provider because the preferred model is unavailable. Preserve idempotency and prevent duplicate messages, records, claims, or payments across retries. The fallback must meet the same authority and evidence gates for its narrower approved scope. Test missing credentials, regional outage, model retirement, quota exhaustion, and corrupt output.

Keep provenance across delegation

Record router version, attributes used, rule selected, providers considered, selected model, data sent, tools available, output, reviewer, edits, downstream action, retries, and fallback. If one model calls another, each delegation appears in the chain. Logs should reveal whether the route matched the approved rule without unnecessarily copying PHI. Staff need a way to see and challenge the route when it affects review.

Work through a route inventory

Rafael locks 28 fictional model-provider routes. Twenty-two have an approved purpose, data class, provider terms, region, validation, tool boundary, human review, fallback, provenance, and stop test: 22 of 28, or 78.6%. Two routes silently fall back to an unapproved provider, one lacks a region answer, one changes model aliases without notice, and two cannot reproduce why the router selected them. All six remain disabled.

Monitor cost and quality without hiding risk

Track route volume, input class, quality failures, abstentions, latency, errors, retries, cost, quota, human edits, incidents, and fallback use. Segment by model and version. A cheaper route is acceptable only when it meets its approved requirements; a faster route cannot bypass privacy or qualified review. Investigate unexpected routing and lock the affected cohort before changing rules.

Use a routing approval checklist

  • Which attributes select the route?
  • Can untrusted content influence provider or tool choice?
  • Is every provider approved for the exact data and purpose?
  • Has each route passed the same critical tests?
  • What happens during quota, outage, or retirement?
  • Can staff reconstruct every selection and retry?

Prevent fallback from expanding risk

A fallback provider may have different retention, training, regions, subprocessors, authentication, logging, accessibility, content limits, and model behavior. Treat it as a separately approved route rather than a substitute URL. The router checks the data class, purpose, tenant, user, action, and current provider approval before sending the request. When no eligible route exists, the workflow holds or uses the documented manual alternative.

Do not send a failed request to several providers automatically unless the approved design requires it and accounts for every disclosure, charge, and duplicate output. A timeout may be ambiguous; the first provider could still process the content. Use request identifiers and provider status evidence to decide whether a retry is safe, and keep the user informed about the held state.

Test the router as its own system

Build cases for each allowed route, prohibited data class, provider outage, quota exhaustion, latency spike, wrong tenant, stale configuration, model retirement, conflicting provider result, and missing audit log. Verify which provider received the request, which version ran, why the route was selected, what evidence returned, and whether downstream review matched that route's limitations.

Include adversarial inputs that try to select a provider, lower a review tier, or pass data through an unauthorized fallback. Routing attributes come from trusted system fields, not from user text or retrieved content. Lock all route rules and provider configurations with the evaluation result.

Reconcile quality, cost, and continuity separately

Leadership may compare providers on cost and speed, but those measures cannot override critical safety, privacy, accessibility, or correctness gates. Report the number of eligible requests, routed requests, abstentions, failures, retries, duplicates, provider changes, severe errors, reviewer corrections, and downstream actions by route. A cheaper average is not a release rationale when one route creates an uncontained high-impact failure.

When a provider leaves service, close credentials, queues, retained data, logs, and unresolved requests, then update every routing rule and fallback test. Preserve historical route evidence for records, incidents, corrections, and disputes that still depend on it.

Make route changes visible to the people who review the output. Show the provider or approved route identifier, model version when available, source evidence, material limitations, and whether a fallback occurred. Reviewers need to know when behavior may differ from the familiar path. For high-impact work, require fresh approval after a route change rather than carrying forward an earlier decision made on another configuration.

Keep an accessible manual route available when a user cannot safely use the provider selected by the automated router.

Name its owner and response time.

Related resources

Sources