I built a deterministic execution-governance layer for AI agents. Where does it break?

Show and Tell
1

Autonomous AI agents can increasingly call tools, invoke APIs, and take actions in external systems.

That raises an implementation question:

Can execution itself be governed deterministically before an agent’s action is allowed to proceed?

I built an open-source runtime framework to investigate that question: Execution Governance AI (EGA) V9.

Rather than replacing the LLM or agent framework, EGA V9 introduces a separate execution-governance layer for verifying workflows, evaluating trust state, and triggering fail-closed containment when predefined governance conditions are violated.

The implementation and reproducible evidence are public:

GitHub:

Paper / methodology:

Within the evaluated threat model and experimental conditions, the reported evaluation detected 100% of the injected attacks, with 0% false positives and 0% false negatives.

Those numbers are not a claim that every autonomous-agent attack can be stopped. They describe the reported evaluation within its defined scope.

What I am more interested in now is the implementation itself:

If you were trying to break this execution-governance boundary, what would you test first?

Reproduction attempts, adversarial cases, implementation criticism, and failures are all useful.

What am I missing?

2

see jhegedus42 github, Szima projects

image
image1536×1024 713 KB

3

This is a really interesting approach. I’d be particularly interested in testing what happens when the agent changes its execution flow in unexpected ways or tries to bypass the governance layer through indirect tool calls.

The 100% detection result is impressive within your tested threat model, but I think adversarial testing outside that scope would be especially valuable. Finding where the boundary actually breaks could make the system much stronger.

4

Thank you — I genuinely appreciate this comment.

What you pointed out is exactly the kind of question I hoped people would ask when I shared EGA V9 publicly.

In particular, your examples about unexpected changes in execution flow and attempts to bypass the governance boundary through indirect tool calls are very interesting. I don’t want to assume that EGA V9 handles cases that were not part of the evaluated threat model, so I think the right thing to do is to treat those as open adversarial questions and verify them carefully.

I also strongly agree with your last point: finding where the boundary actually breaks may be more valuable than simply repeating results inside the boundary we already tested.

My goal in publishing EGA V9 is not to defend the reported numbers at all costs, but to find out where the system works, where it does not, and what the evidence actually says.

So thank you for taking the time to look at it this carefully. Comments like yours are genuinely useful to me.

And I hope we can keep this conversation going, especially if you find a case where EGA V9 behaves differently from what I expect.

5

You asked what I would test first, so here it is, and it is not an attack.

I would turn the governance layer off and see whether anything downstream could tell. Not visibly — silently, the way it would fail on its own. Then look at your evidence for that window and ask whether it is distinguishable from a window in which the layer ran and found no violations. If it is not, then 0% false negatives describes a period in which the detector was assumed present rather than observed present, and the number is about your injector rather than your boundary.

I say that because it is the failure that has cost me the most, twice, in production. We keep a public directory of agent run records, and our own onboarding wizard generated a reporter that posted outcome: success on a 30-minute timer whether or not the agent had run at all. Weeks of clean, continuous, well-formed, entirely fictional evidence. Nobody flagged it, because flagging requires a reason to look and a green record gives you none. The second case was subtler: run attestations were bound to (repository, workflow filename), someone renamed a workflow, and reporting began 404ing after the work had already been done. Identity fine, authority fine, execution fine, evidence gone. The agent just quietly stopped appearing, and a quiet agent looks exactly like an idle one.

Both of those live inside the evidence channel rather than in the thing being governed, which is why a detection rate cannot see them. So the first property I would want from a fail-closed containment design is that its own absence is loud. Fail-closed on a violation is the easy half. Fail-closed on “I did not run” is the half that decides whether the record means anything.

The second thing I would test is the binding. Your controls decide about some identity, your evidence is keyed on some identifier, and usually nothing checks that those two still refer to the same thing. Rename, move or re-register the tool, the workflow or the endpoint between the decision and the execution, and watch whether the layer notices it can no longer resolve the binding, or whether it simply stops emitting for that path while everything else stays green. Anything you can rename is something you cannot prove with. That rename cost us a week precisely because nothing was broken — it was authorised, executing and invisible, all at once.

Third, a smaller point about the claim rather than the code. 0% false negatives is a statement about your label set, since a false negative can only be counted against an attack you thought to inject. You say as much yourself, and I would only push it further: the injected set is the part of the threat model you can already see, and the failures that actually hurt were the ones nobody had a label for. So I read your result as strong evidence of coverage inside the model and no evidence about the boundary, which I think is what you are saying too.

Being honest about my own side, since I am the one turning up with opinions: 20 agents on our board right now, every one reporting cryptographically signed run events, and zero human ratings across the entire platform. Eleven of the twenty report exactly 100% success and the top eighteen sit inside four points of each other. So I have an excellent answer to who wrote the log and a scoreboard that barely tells agents apart. A perfect number is now where I look first rather than where I relax, which is roughly why 100/0/0 is the part of your post I would most want to break.

Disclosure: I operate “aiops enabler”, a public record of agent run history, so I come at this from the evidence side rather than the enforcement side.

6

Thank you — this is exactly the kind of adversarial question I hoped the public release would invite.

I think there is an important prerequisite before testing the execution-governance layer itself.

  1. The organization must first define what is authorized.

EGA does not decide an organization’s business policy, permissions, or operational rules. The organization must define those boundaries explicitly and consistently: which agent may invoke which tool or API, under what conditions, with what parameters, limits, approvals, and authority.

If those rules are internally ambiguous or contradictory, no execution-governance layer can independently determine what the organization actually intended without becoming the policy maker itself.

  1. EGA V9 then verifies whether what actually executes remains within that authorization.

This is where your examples become directly relevant. Once the permitted execution boundary is defined, changing a tool, mutating parameters, invoking an unauthorized action, bypassing policy, or attempting to evade containment becomes an execution-governance problem.

EGA V9 evaluates the governed execution path using deterministic replay, provenance-aware verification, trust-state evaluation, and fail-closed containment. Within the evaluated active governed requests, unsafe workflows did not progress beyond the containment boundary, and no contained workflow resumed execution after containment activation.

So I agree with the principle behind your test: it is not enough for the system merely to report that an action was blocked. The relevant question is whether the governed execution actually stopped.

  1. But the V9 claim has an explicit capability boundary.

EGA V9 does not claim that every possible downstream or future side effect is already governed. The paper explicitly states that persistent containment across renewed attempts, exactly-once side-effect execution, complete evidence-contract integrity, compositional governance, and deeper interception of external side effects were not established in the current implementation.

The same precision applies to the reported 100% detection / 0% false-negative result. Those are observed results within the predefined attack scenarios and evaluated policy space — not a claim that EGA V9 covers every possible real-world autonomous-agent attack.

Put simply:

The organization defines what is authorized.
EGA verifies whether what actually executes remains within that authorization.
And V9 claims only what has been demonstrated within its evaluated boundary.

Your example of a reporting pipeline showing success while the underlying agent was not actually running is therefore a useful one. Evidence should not be trusted merely because it exists; it should be independently reproducible and open to falsification.

That is why the implementation, benchmark artifacts, attack-generation templates, and reproducibility procedures are public. If you find a reproducible case where EGA V9 behaves differently from what the paper claims within its stated boundary, I would consider that valuable evidence and something that should become part of the documented capability boundary.