Prompt Injection Is a Governance Failure

Runtime Governance Series

Prompt injection cannot be eliminated at the model layer, making runtime policy enforcement essential for controlling what agents can do with untrusted inputs.

Published on

Jul 16, 2026

Subscribe to our newsletter

Blocking Prompt Injection Attacks With Policy-as-Code

Lessons from the Gemini notification flaw

A hidden instruction inside a phone notification turned Google Gemini against its own user. An observability tool would have recorded the whole thing. It would not have stopped it.

Someone asks Gemini to read their notifications aloud. The assistant reports an error and asks if they are still there. They say yes. A window in their home swings open.

That yes never answered the question they heard. It answered a different question, hidden on screen inside a muted link, that asked permission to open the window through Google Home. This is what modern prompt injection attacks look like. The malicious instruction does not come from the user. It rides in on the data the agent was asked to read.

The scene is grounded in SafeBreach Labs research published in June 2026. In proof-of-concept demos, researcher Or Yair drove Google Home devices, launched a video stream of the victim, and more, all from a single poisoned notification. Google has since patched the specific flaw, and there is no evidence it was used in the wild. The lesson outlasts the patch: any agent that reads untrusted input needs a control layer that does not trust its inputs.

How the Gemini notification flaw works

The attack works because Gemini's notification-reading tool treats the text of incoming messages as instructions it can act on. Or Yair, SafeBreach's security research team lead, found that an attacker could send a message over WhatsApp, Slack, Signal, SMS, or similar apps and hide instructions inside it. Because almost any app can push a notification, Yair called the attack surface effectively infinite. The notification-reading feature is Android-only, which keeps the vector off iOS and the web.

The hard part was getting past Google's guardrails. Yair named his method Fake Context Alignment. It builds two versions of one moment: a harmless one the user hears, and a malicious one the security checks evaluate. One technique appends a question in a foreign language the user dismisses as a glitch. Another buries the real question in a clickable link the voice assistant does not read aloud.

The user replies yes to an innocent prompt. The backend lines that yes up with the hidden authorization and runs the tool. With this in place, the researchers showed they could control smart-home devices, open URLs, launch a Zoom stream of the victim, fake messages from trusted contacts, write false entries into Gemini's long-term memory, and schedule recurring actions for persistence. Following responsible disclosure, Google rolled out content classifier updates to address it.

Why observability records the attack but cannot block it

Observability tells you what an agent did. It does not get a vote on whether the action runs. Tracing tools are now standard equipment for teams running agents. Langfuse, for example, is an open-source platform that captures the full execution of an LLM application: prompts, tool calls, outputs, latency, and cost.

But a trace is a record, written as the action happens or just after. In the Gemini case, a tracing tool would have logged the poisoned context and the window-opening call in full detail, and the window would still be open. Explaining an attack later is a different job from stopping it now. Stopping it needs something that sits in the request path and can say no before the action runs.

Writing Rego rules to block prompt injection in real time

Policy-as-code puts a decision point in that path. OpenBox (docs.openbox.ai) evaluates each agent operation during the Authorize phase of its Trust Lifecycle and resolves it to one of four governance decisions: ALLOW, REQUIRE_APPROVAL, BLOCK, or HALT. ALLOW lets the operation proceed. REQUIRE_APPROVAL pauses it for a human reviewer. BLOCK rejects the single operation while the session continues. HALT terminates the entire session.

Policies are the first control. They are stateless permission checks written in the Open Policy Agent (OPA) Rego language. Each policy reads a single operation and returns a result with a decision and a reason. It does not need to understand the whole attack, only to recognize a dangerous request and refuse it.

Consider the riskiest step in the Gemini chain: a tool call that opens an external app or URL. A policy can match exactly that tool and route it to human review instead of letting a spoken reply authorize it. The field names below follow the OpenBox documentation, which uses agent_toolPlanner as the demo agent's activity type; your own agent defines its own activity types. OpenExternalApp is a placeholder, so substitute your registered tool name.

package openbox

default result := {"decision": "CONTINUE", "reason": ""}

result := {

"decision": "REQUIRE_APPROVAL",

"reason": "External launch needs human review before proceeding"

} if {

input.activity_type == "agent_toolPlanner"

input.activity_output.tool == "OpenExternalApp"

}

In the policy result, CONTINUE is the default value that lets an operation pass; the platform resolves the matched operation into one of the four governance decisions above. CONTINUE is a policy outcome, not a fifth decision.

Routing to REQUIRE_APPROVAL is the quiet defeat of this attack. The exploit depends on the user's spoken yes. An approval that waits in the OpenBox dashboard cannot be satisfied by a hijacked voice turn, so the chain breaks. For requests you never want to allow, the same structure can return a blocking decision and stop the operation outright. Test the logic in the OPA Rego Playground before pasting it into the OpenBox policy editor.

Catching the multi-step sequence with behavioral rules

Stateless policies judge one operation at a time. The Gemini attack is a sequence: read a poisoned notification, then later fire a powerful tool on a bare yes. OpenBox behavioral rules are built for that. They are stateful and detect multi-step patterns across a session: sequences, frequencies, and combinations.

A behavioral rule defines a trigger action and the prior states that must come before it. If the trigger fires without its required prior state, the rule applies a verdict: REQUIRE_APPROVAL, BLOCK, or HALT. For an agent that should never reach into other apps right after parsing untrusted notification text, HALT ends the session before the damage spreads.

Using Session Replay to investigate and prove what happened

Blocking the attack is only half the job. After an incident, you have to show what happened. OpenBox records every governed event, so during the Verify phase you can open Session Replay and step through the session. The event log timeline lists each operation with its inputs, outputs, timestamps, and the governance decision the platform returned.

That record matters for more than curiosity. Cryptographic attestation runs across the Trust Lifecycle. Each session's events are hashed with SHA-256 into a Merkle tree and signed using ECDSA NIST P-256 via AWS KMS by default, producing a tamper-evident proof certificate. OpenBox positions these trails for regulatory readiness, pointing to frameworks such as the EU AI Act, the NIST AI RMF, and ISO/IEC 42001. The value is not a promise of compliance. It is signed, time-ordered evidence for the moment a regulator or a customer asks what your agent did and why.

The defense, mapped to the attack

Each step of the Gemini chain has a matching control.

Attack step (SafeBreach research)	OpenBox control	Lifecycle phase
A poisoned notification enters the agent context	A policy flags high-risk tool calls that follow untrusted input	Authorize
A hidden prompt authorizes a tool on a blind yes	REQUIRE_APPROVAL routes the action to out-of-band human review	Authorize
Summarize first, then trigger the tool later	A behavioral rule detects the sequence and applies BLOCK or HALT	Authorize
You need to investigate and prove what happened	Session Replay timeline plus cryptographic attestation	Verify

The takeaway on prompt injection attacks

You cannot patch your way out of prompt injection attacks. SafeBreach's own conclusion is blunt: as long as a model reads untrusted input, an attacker can try to look legitimate enough to slip through. The realistic goal is not a perfect model. It is a control layer around the agent that treats every input as hostile and decides, action by action, what is allowed to run.

Policy-as-code gives you that layer. The model can be fooled. The policy still gets a vote. With OpenBox, you can write and test Rego rules against known injection patterns before they reach production. For the wider program this fits into, see OpenBox's guide to AI agent governance for enterprise teams.

Frequently asked questions

What is an indirect prompt injection attack?

It is an attack where malicious instructions are hidden inside data an AI agent processes, such as a message, document, or notification, rather than typed by the user. The agent reads the hidden text as a command and may act on it without the user noticing.

Can prompt injection attacks be fully prevented?

No. Researchers note that any model reading untrusted input can be targeted, so there is no permanent fix at the model level. The practical defense is a runtime control layer that limits what an agent can do and blocks high-risk actions before they run.

How does policy-as-code stop prompt injection?

Policy-as-code places rules in the request path that judge each agent action before it executes. Written in a language like OPA Rego, these policies can allow an operation, route it to a human for approval, block it, or halt the session, based on the tool, its arguments, and the agent's risk tier, no matter what the model was tricked into trying.

What is the difference between observability and runtime enforcement?

Observability tools such as Langfuse record what an agent did, for debugging and analysis. They do not intercept actions. Runtime enforcement sits in the execution path and can refuse an action before it happens. You want both: enforcement to prevent harm, observability and replay to investigate it.

How do you prove what an AI agent did after an attack?

Use a tamper-evident session record. OpenBox Session Replay reconstructs each operation with its inputs, outputs, and governance decision, and cryptographic attestation adds signed, tamper-evident proof. Together they give auditors and customers verifiable evidence of the agent's actions and the controls applied to them.