Observability Is Not Proof: What It Takes to Make AI Routing Auditable

A routing log tells you where a prompt went. Answering whether it was permitted to go there is a different question entirely.

By Tahir Mahmood, Co-founder & CTO, OpenBox  ·  Last updated 14 September 2026

Consider a scenario that is becoming common in enterprise AI deployments. An agent routes model calls through a gateway spanning dozens of providers. The telemetry console shows which provider served each prompt, the latency, the cost, and the region where the data was processed. A compliance team then asks: was that provider on the approved list for this particular agent and this particular call?

The telemetry console cannot answer that question. Not because the data is missing, but because the data it holds describes an outcome. The compliance question is about a promise. Those are structurally different things, and confusing them is what leads teams to build reporting pipelines they later discover they cannot use as evidence.

The question telemetry cannot answer

OpenRouter already exposes per-call routing data through its Generation API. For each call, the record available at GET /api/v1/generation?id=<gen-id> includes the provider that served the request, the region in which the data was processed, the cost, whether the caller's own key was used, and provider response details that can include information on the attempts that preceded the successful response. This is structured, real data, useful for operational and billing purposes.

The gap is narrower than it appears and more structural. A routing record describes the outcome of a call. It does not contain the caller's policy intent. A gateway can report that a prompt was served by Azure. It cannot answer whether Azure was an approved provider for that call, because no part of the request told it what the caller approved. It can report that the call ran in a global region. It cannot determine whether that satisfied a residency policy, because residency policy was not part of the outgoing request.

The gateway cannot derive the comparison because it was never given what to compare against. That is the gap that turns a detailed routing log into something that cannot, on its own, function as compliance evidence.

Four properties that "auditability" is expected to cover

These four terms are routinely collapsed into a single claim. Keeping them distinct is what makes an evidence argument precise.

A gateway can provide telemetry without attestation. A system can produce attested records without enabling source verification. And a system can produce attested, source-verifiable records without ever having captured what was permitted. The four properties are additive. A routing evidence system that aims to carry weight with an auditor needs all of them.

The ordering property that changes what is provable

The core technical requirement for credible routing evidence is sequencing. Caller intent must be recorded before the request leaves the process. The gateway's routing outcome is only available after the response arrives.

When each is captured at the time it becomes known, from independent sources, the resulting comparison has a specific property: neither record could have been chosen to fit the other. Intent was recorded before the outcome was known. The outcome was obtained from a source that was never told what the intent was. The comparison is therefore between two independently determined values.

A system that stamps both intent and outcome together after the fact cannot make that claim. The verifiable character of a comparison is inseparable from the order in which the values were captured.

How the OpenBox and OpenRouter integration builds the record

The Proof of Routing integration implements this lifecycle for every governed model call:

1. Routing intent is declared before the request is built. A policy can read this claim and refuse or narrow the routing at that point, before any prompt is sent.

2. The approved provider list is written into the outgoing request as provider.only. OpenRouter refuses, rather than falling back to an unapproved provider, if no approved provider can serve the call.

3. After the answer arrives, OpenBox retrieves the generation record from OpenRouter's Generation API. The comparison between declared intent and observed outcome is computed and recorded as the span attribute openbox.routing.honored.

4. That span and the full set of routing attributes are hashed into the session's Merkle tree. The session root is signed. OpenRouter's generation ID is retained in the record.

5. The result is a verifiable run receipt: routing facts sealed under the signed session root, with the generation IDs needed to re-check the underlying claims directly at OpenRouter.

One timing detail matters in practice. The generation record is written by OpenRouter shortly after the response, not simultaneously with it. The SDK collects it asynchronously with backoff and drains pending provenance before the session closes. Calling await openbox.close() is required; omitting it risks closing the session before the generation record has arrived.

The cryptographic layer and what it precisely proves

Per-session attestation works as follows, per OpenBox's attestation documentation. Each governance event is hashed with SHA-256. The event hashes are combined into a Merkle tree using sorted-pair hashing. The session root is signed using ECDSA NIST P-256 via AWS KMS by default, with a dedicated key per agent. Each session produces a proof certificate containing the Merkle root, the signature, and the event count.

What this establishes is specific. The attested record was not altered after the session closed. The routing figures a dashboard displays are the ones sealed when each call ran, not a summary recomputed later. This is tamper-evidence: the property the signature establishes is integrity after sealing. It does not guarantee that what was recorded at the time of recording was accurate.

This distinction matters when communicating with auditors. Tamper-evidence and proof of truth are different claims. Keeping them separate is what makes the evidence argument credible rather than overclaimed.

Source verification via the generation ID

The generation ID is what allows routing claims to be checked without involving OpenBox. Each governed model call retains the identifier OpenRouter assigned to it. Anyone with the run receipt and appropriate OpenRouter API access can fetch the corresponding record directly from the gateway:

curl -H "Authorization: Bearer $OPENROUTER_API_KEY" \

  "https://openrouter.ai/api/v1/generation?id=<gen-id>"

The two verification mechanisms serve different purposes and complement each other. The cryptographic signature establishes that the OpenBox record was not altered after the session closed. The generation ID provides a path back to the upstream source, so anyone can check whether what OpenBox recorded matches what the gateway actually observed. These are independent checks on the same routing claim.

OpenBox is explicit on this point: the gateway is the source of truth for the underlying routing facts, not OpenBox itself. The integration reads OpenRouter's generation record after OpenRouter hands it over.

Coverage: what the honoured rate is actually measuring

A routing record that reports every call honouring the provider allowlist is meaningful only if those calls actually declared an allowlist. An unconstrained call, one that named no provider list, is excluded from the honoured rate, not counted as a pass. The Routing Integrity panel makes this explicit: the openbox.routing.honored attribute is absent on unconstrained calls, not true.

The same discipline applies to every other check. An unreported region is unchecked, not approved. An unpriced call is a gap in the evidence record, not a zero-cost call. A call that used openrouter/auto is unchecked for model substitution, because delegating model selection was the intention.

Coverage tells you how much of the traffic the rate speaks for. A perfect rate over a narrow constrained window and a perfect rate over full traffic are different claims. Reading coverage before any rate is what prevents a compliance dashboard from reporting a promise that was never made.

What a run receipt does and does not prove

A run receipt is one session's routing evidence on a single shareable page. It reports the providers that served each call, the regions where data was processed, whether routing matched what was requested, and the gateway's generation ID for each call. Prompt and completion content is not in the record, which is what makes a shared page possible.

What the receipt reports is what OpenBox recorded and sealed. What it does not prove is that those recordings were accurate at the time they were made. For the underlying routing facts, the generation IDs are what remove that dependency: a reader can re-check the source without OpenBox in the loop.

A receipt covering a session where a policy refused all calls is often the most informative one. It attests that no model call completed and nothing was served. That is the provider allowlist working as intended, and the receipt is the document that demonstrates it.

Turning telemetry into evidence

A routing log covers one side of a compliance question. It records where a prompt went. The question asks whether it was permitted to go there. Joining those two sides, recording intent before execution and outcome after, sealing the comparison in a form that cannot be quietly edited, and retaining the identifiers that allow independent verification: these are what turn routing telemetry into an auditable record, evidence an auditor can examine and challenge.

The conceptual requirement is narrow: both ends of the proposition must be present, captured in the correct order, and independently checkable. Retaining more metadata does not, by itself, satisfy that requirement. The evidence model is defined by sequencing, not volume.

OpenBox implements this for agents on the OpenRouter framework through Proof of Routing. Teams working through the practical setup will find the Getting Started guide the right entry point. For the broader governance context that routing evidence sits within, the Complete AI Agent Governance Guide covers the full picture.

Frequently asked questions

Can routing claims in a run receipt be verified without trusting OpenBox?

Yes, for the underlying routing facts. Each call in a run receipt retains the generation ID that OpenRouter assigned to it. Anyone with OpenRouter API access can fetch the corresponding record directly from the gateway without OpenBox in the exchange. The signature separately shows the OpenBox record was not altered after the session closed. Two independent checks on the same routing claim.

What is the difference between the provider allowlist check and the approved-region check?

The provider allowlist travels in the request as provider.only and OpenRouter refuses if no approved provider can serve the call. The region check is observational. A standard request carries no generic region parameter, so OpenBox records what the gateway reports after the response and a policy can act on a breach. OpenRouter separately offers US and EU In-Region Routing on Business and Enterprise plans as a distinct capability.

Does a 100% honoured rate mean all traffic respected the routing policy?

Only if that traffic declared an allowlist. Calls that named no allowlist are excluded from the honoured rate, not counted as passes. Coverage tells you what share of total traffic was constrained; the rate tells you how many of those calls kept their promise. Read both figures together to understand what the rate actually means.