Agent operating structure lifts model performance from 30 to 81 percent
Most organizations that build AI agents begin their journey by searching for the largest model on the market, starting from the assumption that performance accuracy is directly linked to model weight and raw computational power. In AI Gossips we followed a controlled evaluation experiment published by IBM Research in early November 2026, which revealed that this assumption overlooks the decisive variable in real work environments: the operating structure and control that drives the model and coordinates its tool invocations and regulatory rules.
The researchers subjected an open-weight model, GLM-5.3-flash, to ninety tasks distributed across thirty scenarios within the AppWorld benchmark, which measures models’ ability to handle nine consumer applications and 457 APIs covering email, payments, and file management. The test ran with a single attempt per task using the Harbor framework, with the model frozen and only the operating structure changed among three software frameworks: CUGA, developed by IBM; Hermez from NuS Research, designed for always-on personal assistants; and Deep Agents from LangChain, aimed at multi-step research tasks.
Results showed that changing the operating structure alone raised task completion from 30 % with Deep Agents to 46.7 % with Hermez, then to 81.1 % with CUGA.This gap doubles when measuring scenario consistency across all branches, as CUGA passed twenty scenarios in full, compared with eight for Hermez and only two for Deep Agents. The outcome does not diminish the other frameworks; it demonstrates that the operating structure must be tailored to the work environment. CUGA uses an integrated loop that breaks tasks step by step, passing the relevant application tool at each step instead of confusing the model with all interfaces, running code in Python within an isolated environment, reviewing code before execution to adjust data, and auditing page-numbering processes automatically.
The practical shift in this proposal lies in moving compliance rules from the textual command window to a solid operating layer. Development teams have traditionally written business constraints, such as financial payment approvals or preventing customer data leaks, inside the system’s directive commands, yet the text remains flexible instructions that the model can misinterpret or override under conversational pressure or injection attacks.The governance-by-construction system offers a solution by imposing absolute software policies that guard tool calls and enforce mandatory human approval before any transformation is executed.Policy-lab experiments demonstrated this tangible difference: when instructed to pay dues, the unrestricted agent paid ninety dollars, including a disputed invoice in the collection, while the policy-controlled agent halted the transaction for review and paid only ten dollars, the actual amount required.
This shift has practical implications for financial institutions and companies in the Gulf and Egypt; technical teams spend high budgets on subscriptions to closed commercial models to avoid agent mishaps, while the experiment shows that fast open models achieve double the completion rates when surrounded by robust management and security structures. The open-source framework Apache 2.0, licensed for local deployment entirely isolated from the public internet, also meets the sovereign-computing requirements imposed by regional regulators to protect sensitive data within national borders without relying on external cloud servers.
Accordingly, developers’ skill priorities shift; the focus is no longer solely on crafting directive commands, but on engineering software security barriers, managing consumption budgets, and building isolated environments that prevent an AI agent from making costly financial or administrative mistakes regardless of model behaviour.