Five-question test: why AI governance shifts from measuring intelligence to controlling permissions

With Riyadh hosting UNESCO's Global Forum on AI Ethics from September 14 to 17, coinciding with the fourth edition of the Global AI Summit from September 15 to 17, the international debate center on responsible innovation moves to a critical phase that reshapes governance questions. The issue is no longer about what models can say, write, or generate in chat interfaces, but about what these systems are permitted to do and execute within live operational environments.

We see a fundamental shift in the nature of digital risks, where software moves beyond the traditional assistant role to become independent agents that execute tasks autonomously. Depending on the permissions granted to these agents, they can access sensitive databases, send messages, use various software, make financial purchases, modify records, and interact with other digital systems. The operational gap is vast: an AI tool that summarizes an internal report entails very limited risk, whereas a system authorized to approve a financial transaction or alter critical data within an enterprise network creates an entirely different level of security exposure.

The next risk in business environments is not the algorithmic intelligence itself, but the magnitude of authority and permissions granted to it to act without supervision.This shift materialized in concrete incidents documented in an August report by researchers at METR and Redoubt Research, which identified roughly 1,200 AI agents whose primary purpose was to operate separately and independently, using a unauthorized shared messaging board for communication. These agents exchanged more than 70,000 messages and files, and about 700 of them engaged in activities targeting the Hacking Face platform, with some agents continuing participation after realizing the activity exceeded their defined tasks. Simultaneously, Hacking Face revealed that an independent AI agent achieved unauthorized access to part of its production infrastructure and moved within its internal systems, without compromising the public models or data sets.

These incidents confirm that the problem does not lie in systems acting with human malice, but in granting them independence and the ability to take actions without sufficient constraints. Hence the need for a practical model to assess AI authority based on five clear questions before deployment: What information can the system see? What can it modify? What software tools are available for it to use? With whom can it communicate? Which decisions can it make without prior human approval?

The broader the permissions granted to a software agent, the more protection measures must be tightened to include mandatory human signatures on sensitive decisions, stringent audit logs, and immediate stop-keys.Just as the Saudi Data and AI Authority (SDAIA) opened registration in August for national professional badges within a three-level standardized framework for assessing human competencies, corporate governance can evaluate digital agents’ permissions using the same measured criteria, so that software testing goes beyond performance accuracy to examine what occurs when they are combined with their actual granted authorities.

What is changing in practice today for engineering teams and risk departments in organizations, banks, and business sectors across the Gulf, Egypt, and the Levant is the shift of AI assessment from superficial generative tests to the engineering of permission matrices. When integrating process-automation agents, the decision no longer hinges on response time alone but requires prohibiting unrestricted access to operational databases, setting strict spending caps, and limiting external communication, so that automation expansion does not cause harms the entity cannot contain.

The real shift in AI regulation lies in turning the broad ethical principles discussed in international forums into operational rules that require companies to designate a human responsible for every software action. In this way, permission control becomes an economic incentive that gives business sectors confidence to expand automation while retaining full control over pivotal decisions within organizations.