AI agents – or Agentic AI – are revolutionising the corporate world: from automation to data analysis to autonomous decision-making. But with the speed at which AI agents are being introduced, the risk is also growing: who is actually controlling what these agents do? And how can companies ensure that AI does not become a compliance or security risk? In this article, you will learn how the principle of 'In Dubio Pro Securitate' and modern governance approaches help to deploy AI agents safely and in compliance.
How does the security gap of AI agents arise?
Many companies use AI agents without really knowing their access rights, behaviour or risks. Typical mistakes:
- AI agents are granted overly broad access rights ('Least Privilege' is ignored).
- There is no continuous monitoring or audit trails.
- Compliance requirements (e.g. ISO 27001, SOC 2) are not applied to AI agents.
- Security incidents are piling up: chatbots are concluding contracts at fantasy prices, agents are installing keyloggers or leaking internal data.
- Shadow AI is emerging, similar to Shadow IT, because no one really has an overview of the agent fleet.
Agents must fundamentally be treated as unsafe until their safety has been formally proven – because the costs of an approved but unsafe AI action are far higher than blocking a useful but unproven action.
Erik Meijer, Speaker at the Enterprise AI Summit 2026 and internationally recognised computer scientist, software architect and innovator in the field of programming languages and formal methods
From presumption of innocence to presumption of safety: 3 safety principles for Agentic AI
1. In Dubio Pro Securitate
In the classical legal system, the principle applies: “In case of doubt, for the accused.” For AI agents, this principle must be reversed: In case of doubt, for safety (“In Dubio Pro Securitate”). This means that every action of an AI agent is initially considered potentially unsafe until the opposite is proven.
Why is this necessary?
- AI agents can make far-reaching decisions in fractions of a second (e.g. concluding contracts, sending data, configuring systems).
- A single error can cause enormous financial, legal or reputational damage.
- The speed and complexity of AI systems exceed human control capabilities.
2. Deferral and formal verification as a protection mechanism
Deferral means that an AI agent does not act directly, but first only generates a proposal (plan, code, action). This proposal is then checked before execution.
Formal verification is a mathematical procedure in which it is checked whether the agent’s proposal meets all defined rules and safety requirements. This is done, for example, with so-called SAT solvers or SMT solvers, which automatically search for counterexamples.
Example: An AI agent proposes to sign a contract. Formal verification checks whether all contract terms, price floors and compliance requirements are met. If a rule is violated, the proposal is rejected and the agent must generate a new plan.
3. Separation of proof finding and proof checking
A central principle for safe AI agents is the clear separation between proof finding and proof checking.
- Proof finding: The AI model creates a proposal (e.g. code, plan, action) that is supposed to meet the desired requirements. Here, the model can act creatively and flexibly, but errors or rule violations are possible.
- Proof checking: An independent, deterministic system subsequently checks whether the proposal really complies with all safety and compliance rules. Only if the check is passed is the agent allowed to act.
Why is this important?
- Avoiding “Blind Trust”:
AI models are powerful, but not infallible. They can make mistakes, overlook security vulnerabilities, or even – intentionally or unintentionally – circumvent rules. If the same system that makes the proposal also decides on its safety, there is a high risk of errors or manipulations. - Independence and Traceability:
The proof verification is independent of the AI model and operates deterministically. This means: For the same proposal, there is always the same verification result. This creates transparency and traceability – key prerequisites for audits, compliance, and trust. - Efficiency through Repetition:
Since the generation of proposals by AI agents is very cost-effective, unsafe or erroneous proposals can simply be discarded. The agent generates new proposals until one passes the verification. This increases security without slowing down the pace of innovation.
Practical Example: An AI agent proposes a code change. Only when a formal verification system confirms that no compliance rules are violated is the code executed. Otherwise, a new proposal is generated.
This separation ensures that AI agents act innovatively, but never uncontrollably or insecurely. A must for any company that relies on AI.
5 concrete steps for secure AI agent governance
1. Zero Trust and Least Privilege for AI agents
A central principle of modern IT security is the Zero Trust model. Applied to AI agents, this means: No agent is automatically granted trust or extensive rights. Every access and every action by an agent must be explicitly permitted and traceable at all times.
The principle of “Least Privilege” complements this approach by ensuring that each agent only receives the minimum necessary rights required for its specific task. This prevents an agent – for example, a reporting agent – from accessing sensitive customer data or administrative functions if it does not need them for its work.
In practice, all agent identities are managed centrally, for example with DevSecOps platforms such as Container8. Permissions are regularly reviewed and adjusted to ensure that no unnecessary rights remain. If there is suspicion of misuse or misconduct, an agent's rights can be revoked immediately to prevent damage.
2. Operationalising formal verification
To further increase the security of AI agents, every plan proposed by them is subjected to formal verification before execution. This means that the proposal is checked against all relevant rules and compliance requirements using mathematical methods and automated tools such as Policy-as-Code or SAT solvers. Only when the plan passes this verification and no rule is violated is the agent allowed to actually execute the action.
A clear example is an AI agent that wants to trigger a payment: The verification checks whether the amount is within the permitted range, all compliance requirements are met, and no data leaks occur. Erroneous or risky actions are thus detected and blocked early on, before they can cause damage.
3. Compliance “Shift Left” – Governance as Code
Traditionally, compliance checks often only take place at the end of a development process. In the age of AI and DevOps, this is too late.
The “Shift Left” approach integrates compliance checks early in the development process, ideally in an automated manner. This is achieved through “Policy-as-Code”: rules and policies are formulated as code and integrated directly into the CI/CD pipeline. Developers and agents receive immediate feedback on whether their changes are compliant. This significantly reduces the effort required for manual audits and accelerates the market launch of new features without compromising security.
4. Continuous Monitoring and Audit Trails
Another key component is the continuous monitoring of all agent activities. Every action taken by an AI agent is recorded comprehensively, ensuring that it can be traced at any time who did what and when.
Modern monitoring systems analyse these activities in real time and immediately detect unusual or unauthorised behaviour. For example, if an agent attempts to access data outside its scope of responsibility, the system raises an alarm and automatically blocks the action.
Central dashboards provide a complete overview of all agent activities and enable rapid escalation in the event of policy violations. This transparency is essential not only for security but also for audits and compliance checks.
5. Peer Review and Manual Control for Critical Actions
Especially for highly sensitive or financially critical actions, automation alone is not sufficient. This is where the four-eyes principle comes into play: at least two people or systems must approve an action before it is executed. This ensures additional security and prevents a single agent – or a single human – from making critical decisions alone.
In practice, thresholds are defined above which peer review becomes mandatory, for example for particularly high amounts or when accessing particularly sensitive data. This creates a balanced combination of automatic and manual control that guarantees both efficiency and security.
Summary
- AI agents are a massive lever – but also a massive risk if governance is lacking.
- The principle of “In Dubio Pro Securitate” protects companies from costly errors and compliance violations.
- Formal verification, Zero Trust, and Policy-as-Code are the keys to the secure operationalisation of AI agents.
- Continuous monitoring and audit trails create transparency and traceability.
- Companies that act now secure a decisive competitive advantage.
Do you want to know how to deploy AI agents in your company securely and in compliance?
Contact our XALT AI experts: We analyse your agent landscape, identify compliance gaps, and show you the way to Zero Trust Governance.
