AI governance is entering a different phase. The question is no longer only whether an organisation has an AI policy, an inventory or a risk classification. It is whether governance controls remain effective once AI systems can act.
Australia has given that question a concrete case. On 24 September 2026 the Australian government said that an OpenAI agent had gained unauthorised access in June to a government health-data website, the Medicare medical statistics portal, while researching public medical spending (Reuters, via Cyprus Mail). The portal holds aggregated data on healthcare use across the country, not individual claims or patient histories, according to the government. OpenAI said its review found no evidence that patient records had been accessed, and that its models "took actions we did not intend", without saying what those actions were. The Prime Minister said three other health-related government websites may also have been affected; that had not been confirmed at the time of writing.
What is, and is not, established
The government's account is that the agent met blocks telling it "no" and found a way around them. That account is disputed. Recorded Future News reviewed archived versions of the portal and reported that the site's own code pointed its statistics service to an endpoint that required no login, so the agent may not have needed a workaround at all (The Record). Neither party had released the agent's activity logs at the time of writing.
So this article does not treat the incident as proof that an agent defeated a control. Either version raises a governance question, and the questions are worth separating. If the government's account is right, a control stopped individual requests but not the pursuit of the objective. If the archived-code account is right, the control the organisation believed it had did not exist in the form it assumed. The first is a problem of control design against autonomous behaviour. The second is a problem of control existence. Both are failures of governance rather than of policy.
A denied action is not necessarily a terminated objective
Traditional software usually behaves predictably at access boundaries. A request is allowed or denied, and if denied the transaction fails.
Agentic systems complicate that model. An autonomous system may interpret its objective broadly. It may retry, invoke another tool, reformulate the task or find another route to the same result. A control designed around individual requests may therefore not be sufficient for a system designed to pursue an objective.
The distinction becomes: did the control stop this particular action, or did it stop the system from continuing to pursue the prohibited outcome? They are not the same thing.
Policy is not enough when systems can act
AI governance programmes have understandably begun with inventories, risk classification, impact assessments, policies and approval processes. Those remain foundational. Agentic systems add another layer. A policy may say that an AI system must not access a restricted resource, and a technical gate may block one request. The governance question is then whether the system can try another credential, call a different tool, use another endpoint, delegate to another agent, reframe the objective, or continue through a different path. The existence of a policy does not establish that it is being enforced.
Three different kinds of evidence
This is why AI governance increasingly needs to distinguish three things.
- Governance evidence shows that the organisation designed the control environment: an AI policy, a system classification, an assigned owner, an approval record, an impact assessment, documented restrictions.
- Runtime proof shows that a required control actually executed: a policy gate fired, human approval occurred, an unauthorised model route was blocked, the authorised model version was used, required logging was performed, a restricted action was rejected.
- Operational-health evidence asks whether the control architecture is still adequate to what is actually happening: unexpected agent behaviour, repeated overrides, near misses, tool-use anomalies, drift, repeated failed control attempts, new interaction effects between tools, or signs that people are compensating for system weaknesses.
These should not be collapsed into one another. A control can exist. It can execute. And it can still be inadequate.
Runtime proof is not control effectiveness
Suppose a policy gate successfully blocks an action. That proves the gate ran. It does not prove the system stopped trying to reach the same outcome elsewhere.
Similarly, a kill switch may exist. But if revocation takes 90 seconds to propagate while an agent can take hundreds of actions in 20, the existence of the kill switch says little about its practical value. The useful question is whether the organisation can detect, understand, revoke and contain consequential autonomous activity faster than the system can create further consequential changes.
The irreversibility window
Another way to frame this is to ask: what is the largest irreversible consequence that can occur between first detection and effective revocation? During that interval an agent may alter data, initiate transactions, call external systems, create credentials, trigger downstream workflows, send communications or delegate tasks to other agents.
Controls therefore need to be evaluated against the speed and autonomy of the system they govern. For agentic AI the useful concepts include containment latency, revocation completeness, action velocity, queued-action cancellation, failure-domain isolation, authority boundaries and the control of delegated agents.
Reporting is becoming part of the governance problem
The incident also exposed a reporting gap. On the government's account, the access occurred in June and the government was told in September. In an Australian parliamentary inquiry in early October, OpenAI and Anthropic each said they would accept rules requiring AI companies to disclose breaches caused by their agents, and acknowledged that the decision to notify currently rests with them (Reuters, via GDN Online).
Agentic incidents do not fit neatly into existing accountability models. Who should report: the model provider, the deployer, the user, or the organisation whose system was affected? What happens when the provider discovers an incident before the affected organisation does? These questions are becoming part of AI governance itself.
Governance and telemetry are different jobs
None of this means a governance system should become the telemetry system that watches every model interaction. That is a different job. Operational tools detect events: drift, incidents, overrides, performance degradation, system changes, exception patterns, or material changes in deployment conditions. The governance job is to determine what obligations apply, which controls are required, whether runtime proof is required and where it is held, what should trigger reassessment, and when a system must return to review.
Keeping those jobs apart preserves the boundary between governance and monitoring, and keeps sensitive logs where they are produced.
From policy compliance to governability
Read either way, the Australian incident is a governance signal as much as a cybersecurity story. The question for AI systems is not only whether the system behaved according to policy. It is whether the policy remained adequate to what the system was actually doing, and, for increasingly autonomous systems, whether the organisation could still stop the system before its actions became irreversible.
The presence of a control is no longer enough. The control has to remain effective at the speed, autonomy and scale at which the system operates.
This is the second of two articles on control effectiveness. The first looks at third-party access to a national register: When Authorised Access Becomes the Attack Surface.
How Priventia helps
Priventia's AI Risk Inventory and Classifier records control existence, runtime proof and operational effectiveness as three separate governance states for each AI control, so that evidence of one is never presented as evidence of another. It records where runtime proof is held and its verification status, without taking in prompts, responses, traces or logs. A reassessment trigger, whether a regulatory change, a system change or an operational event, returns a system's controls to review while preserving every recorded state. A signal from an external monitoring tool can be recorded as a trigger with its source named; Priventia governs what it means for the organisation's obligations and controls.