Fail Closed or Fail Over: Designing Pre-Flight Controls for Multi-Provider LLM Routing

OpenRouter's automatic failover keeps agents running when an upstream provider goes down. That same mechanism means the serving provider is chosen after your code runs, and is not exposed in the response by default. This article examines where enforcement must happen, and what fail-closed means in an agent that routes through a gateway.

By Tahir Mahmood, Co-founder & CTO, OpenBox · Last updated 14 September 2026

When an OpenRouter request leaves with the default routing configuration, the gateway load-balances across available providers and silently falls back to the next option if the first is unavailable. This is what keeps gateway-routed agents reliable under provider outages. It is also the reason that post-hoc provider checks cannot substitute for enforcement: by the time you know which provider handled a call, the prompt has already been delivered.

The question this raises for any system with genuine provider constraints is not how to observe which provider was used, but how to prevent an unapproved provider from ever receiving the prompt. The answer depends on what can be decided before a request is constructed, and what can only be determined after.

The Reliability Tension That Creates the Governance Problem

The OpenBox Proof of Routing documentation characterises the situation this way: OpenRouter makes its users two promises that pull against each other. The first is trust: prompts only go to the providers you choose. The second is reliability: routing across 80-plus providers with automatic failover when one is down.

These two goals pull against each other. Failover works by choosing the serving provider at request time, per call. That per-call selection is what makes the reliability guarantee real. It is also why the provider's identity is not exposed in the default response: the gateway picks it, serves the call, and returns the result without surfacing which upstream provider handled the request.

Per the OpenBox Proof of Routing documentation: "The provider serving any single call is chosen at request time, per call, and never appears in the response." That statement reflects the default response behaviour. OpenRouter now offers an opt-in Router Metadata mechanism (via the X-OpenRouter-Metadata: enabled request header) that can surface the serving provider, region, and routing decisions directly on the response. But without either Router Metadata or a post-call generation record lookup, provider identity is not visible to the caller.

Which provider served a call can be determined from the gateway's generation record after the answer (via GET /api/v1/generation?id=<gen-id>), or by enabling Router Metadata to receive it inline. Without either, the governance question about provider identity can only be answered in arrears.

That is evidence, not enforcement. Knowing after the fact which provider served a call does not un-send the prompt.

What OpenRouter Lets You Write Into the Request

OpenRouter's provider object contains several routing controls, including data-collection policies, ZDR enforcement, and model-specific constraints. Two central fields for provider allowlisting are:

• only: a list of provider slugs the request will accept. When account-wide and request-level constraints name no common provider, per the OpenRouter provider routing documentation, the request fails with a 404.

• allow_fallbacks: a boolean, defaulting to true. Set to false, it prevents the gateway from falling back outside the named list if none of the named providers can serve the call.

The combination { "only": ["openai"], "allow_fallbacks": false } produces a closed constraint: the call succeeds only if OpenAI can serve it, and fails otherwise. The gateway itself enforces the restriction, not your application code.

The question is where the decision to attach those fields should live. If routing constraints are hard-coded into application logic, every change to the allowed provider set requires a code release. A per-session policy update, a trust-score-driven narrowing, or a per-agent constraint introduced after deployment is not possible without re-deploying code. Writing the constraint into a policy evaluated before each call solves that.

The Pre-Flight Routing Mechanism

With preflightRouting on (the default in the OpenBox OpenRouter SDK), every model call carries a routing claim before the request is built. A policy reads that claim and can act on it while doing so still changes where the prompt goes.

Per the OpenBox Routing Policies documentation: "A routing policy decides where a prompt may go, before it goes. It is an ordinary policy that reads the routing claim on a model call and either allows it, refuses it, or attaches the routing to use instead."

The routing claim includes the following attributes:

The pre-flight point is what makes enforcement possible rather than merely observable. Per the OpenBox documentation: "Before the call: The routing the request will use is stated on the call itself, so a policy can refuse or narrow it while that still changes the outcome." Once the request has left the process, a policy can still record and detect but cannot un-send the prompt.

Refusing and Redirecting: The Two Policy Arms

A routing policy operating on the pre-flight claim has two options: refuse the call outright, or attach a narrower routing directive.

Refusing

The simpler path. A policy can block any call that names no allowlist:

result := {

    "decision": "BLOCK",

    "reason": "routing must be constrained",

} if {

    routing

    routing.attributes["openbox.routing.declared"] == false

}

The call has no spans, because no request was ever built under it. The governance record shows: call refused, reason stated.

Redirecting

The policy returns BLOCK plus a patch containing the routing to use instead. The SDK writes that routing into the outgoing request, and the corrected call proceeds:

result := {

    "decision": "BLOCK",

    "reason": "this agent may only send prompts to openai",

    "patch": {"new_input": {"provider": {"only": ["openai"], "allow_fallbacks": false}}},

} if {

    routing

    {lower(p) | some p in object.get(routing.attributes,

     "openbox.routing.requested_only", [])} != {"openai"}

}

Per the OpenBox documentation: "The SDK writes that allowlist into the outgoing request, so OpenRouter itself refuses rather than falling back to a provider you did not approve."

The redirect is spelled as a BLOCK because the verdict is what the governance layer records against the call. A call that is being re-routed did not run as the caller specified. The shape is always BLOCK plus what to do instead.

Narrower Only, and What Happens When the Sets Are Disjoint

The routing directive has one architectural constraint that matters for system design: it can only narrow what the caller already named.

Per the OpenBox SDK Reference documentation: "A routing directive is the one directive the SDK applies on its own initiative, and it can only ever narrow: only and models intersect with what the caller named, and allow_fallbacks is false if either side says so. A policy can therefore never route a prompt somewhere the caller did not allow; it can only constrain to fewer places."

This is a design choice, not a limitation to work around. A governance operator can restrict what callers do; they cannot expand it.

The interesting edge case is when the constraints produce an empty intersection. If the policy approves only one provider and the request named a different one, the intersection is empty. Per the Routing Policies documentation: "nothing satisfies both, and the call is refused rather than resolved in either party's favour."

The call fails closed, the record states why, and neither party's choice is silently honoured. A disjoint constraint produces an explicit failure, not a silent resolution.

Three Execution States, Not Two

When a routing policy redirects a call, the governance timeline shows three distinct states. These matter for audit because they correspond to three different facts in the record.

Per the OpenBox Routing Policies documentation, the timeline for a redirected call reads:

llm_call     started    BLOCK   "not to that provider"

llm_call     completed  BLOCK

llm_routing  started    ALLOW

  └─ routing to openai

llm_routing  completed  ALLOW

llm_call     started    ALLOW

  └─ POST openrouter.ai/api/v1/responses  200

llm_call     completed  ALLOW

The documentation notes: "The refused attempt has no spans, because no request was ever built under it. The routing step appears only when a policy actually redirects

The three states preserve honesty in the record:

• Refused attempt: BLOCK verdict, no request built, no spans.

• Routing step: the policy-directed routing runs as its own event.

• Corrected call: an ALLOW, running against the approved provider.

A call the system re-routed is not recorded as having succeeded at the original destination. The distinction between a refused call and a corrected call is visible in the session timeline, not collapsed into a single success event.

What the OpenBox SDK Enforces Pre-Flight, and What It Observes

Within OpenBox's current routing-policy implementation, provider allowlisting is the one constraint that is enforced before a prompt is sent. The provider.only field travels in the outgoing request. When the approved list and available providers have no intersection, the gateway fails the call before sending the call to an upstream provider.

Data-residency constraints work differently inside the OpenBox SDK. The Routing Policies documentation is explicit: "An approved-region list cannot travel in the request: OpenRouter honours provider.only but has no region parameter." The OpenBox SDK seals an approved-region list into the routing record before the call, compares it against what the gateway reports afterwards, and records a breach when the regions do not match.

It is worth noting that OpenRouter itself has separate platform-level geographic controls outside the SDK's approved-region mechanism. Its In-Region Routing feature (Business and Enterprise plans) routes requests to regional endpoints via base URLs such as eu.openrouter.ai and us.openrouter.ai, and guardrails with allowed_data_regions can reject requests with a 403 before inference. Those controls operate at the OpenRouter platform level, not through the provider request body, and are not what the OpenBox SDK's residency policy mechanism uses.

The following table from the OpenBox Proof of Routing documentation captures the distinction between the two constraints within the OpenBox SDK:

A residency breach under the SDK mechanism is caught on the call after it lands, and a policy can halt the session from there. It is detection and session termination, not prevention. The Routing Integrity panel states this distinction in the dashboard view.

Choosing Where to Write the Rule

The split between the two enforcement arms is mechanical. The choice follows from what you need:

Both arms live in the same policy file. The difference is which span the rule reads: pre-flight rules read the routing claim; post-hoc rules read the provenance record span sealed after the answer arrives.

How This Connects to the Governance Layer

OpenBox, an AI agent governance platform, wraps the full execution cycle of agents built on the OpenRouter Agent SDK through a Trust Lifecycle: Assess, Authorize, Monitor, Verify, and Adapt. The routing enforcement described above sits in the Authorize stage, alongside Guardrails and Behavioral Rules.

The OpenBox SDK wraps the OpenRouter Agent SDK, governing callModel and your tool functions. With preflightRouting on, every model call carries the routing claim before the request is built. A Routing Policy, written in OPA Rego, evaluates that claim and can return a directive. The SDK then writes the approved routing into the outgoing request, so the gateway enforces the constraint rather than your application code.

All enforced operations produce one of five governance decisions: ALLOW (operation proceeds, trust score improves), CONSTRAIN (proceeds with constraints applied), REQUIRE_APPROVAL (paused for a human decision), BLOCK (operation denied, session continues), and HALT (session terminated). The precedence is HALT > BLOCK > REQUIRE_APPROVAL > CONSTRAIN > ALLOW. A routing refusal is a BLOCK: the session continues, and a corrected call can proceed if valid routing exists.

After the session closes, the routing record is sealed into the session's cryptographic attestation, and the Routing Integrity panel shows which providers served which calls, whether each stayed inside its stated allowlist, and what each dishonored call cost. A call with no allowlist is counted as unconstrained, not as a pass. The distinction is in the denominator, not in a warning row at the bottom of the panel.

Frequently Asked Questions

Does allow_fallbacks: false alone enforce a provider allowlist?

Not by itself. allow_fallbacks: false prevents the gateway from falling back to other providers, but it applies only to providers already in scope for the request. Without an only list, the gateway still uses its default load-balancing selection. The combination of only and allow_fallbacks: false is what produces a closed constraint: the call succeeds only if a named provider can serve it, and fails otherwise.

What happens if a policy routing directive already matches the caller's original constraint?

The directive still intersects with the caller's named list, and the result equals the caller's original constraint. The routing step does not appear in the timeline because no narrowing occurred. The call runs as a single llm_call event.

Can a routing policy widen the approved provider set?

No. The narrower-only rule is enforced by the SDK. only intersects with the caller's named list; it does not union with it. A policy cannot add a provider the caller did not already specify.

In the audit record, how does a refused call differ from a redirected call?

A refused call carries a BLOCK verdict on both its start and completion events, with no spans (because no request was built). A redirected call carries a BLOCK on the original attempt, followed by a separate llm_routing event, followed by an ALLOW on the corrected call. The timeline shows both outcomes; neither is collapsed into the other.

Can the serving provider be identified without querying the generation record?

Yes. OpenRouter's opt-in Router Metadata (enabled via the X-OpenRouter-Metadata: enabled request header) can surface the serving provider, region, and routing decisions directly on the response. The OpenBox SDK additionally reads the generation record after the answer and seals it into the session's cryptographic attestation, producing a tamper-evident record that can be re-checked at the gateway independently of the response.