Growing Structure Not Context: How Code Turns Small AI Agents into Experts?

The architecture of large language model agents today relies on loading the context window with everything: from prompt instructions and interaction logs to attempts to infer programmatic control paths at each individual execution step. This approach, despite being common in current development environments, forces the model to repeatedly recompute the same control decisions, leading to inflated inference costs and reduced efficiency of sequential tasks. A new research paper titled “Growing Structure Not Context” proposes an engineering reversal of this equation by converting the repetitive control logic from in-context text to reusable executable code, and limiting language model calls to the semantic reasoning specific to each task.

The researchers presented a training framework they named “Growing Harness,” a model that relies on failure-driven prompting to build the agent’s structure incrementally. The system starts from an initial architecture devoid of pre-programmed strategies, providing only static APIs to the model and tools without any embedded control logic for solving tasks. When an error occurs during execution, function-level trace logs capture the point of failure and constrain it to a specific code segment, allowing a joint optimization system to address a batch of failures together.

The core innovation lies in an evaluation gate that follows a “success-first” principle, automatically rolling back any code changes that might impair the agent’s previously acquired capabilities.Accepted modifications accumulate in a shared code structure, allowing the control architecture to crystallize and evolve from task feedback rather than stacking examples in the model’s transient memory.

The test results on the «BrowseComp-Plus» and «WebArena-Verified» platforms, using three operational models ranging from 4 billion to 120 billion parameters, clearly demonstrated the superiority of this approach. The framework achieved the highest average success in five of six experimental environments, lagging by only 0.7 percentage points in the sixth environment. More importantly, compared with the traditional tool-calling system, the framework succeeded in reducing language-model calls by a range of 76.0 % to 91.8 % and cutting agent inference cost in deployment environments by a range of 74.4 % to 98.6 %. Furthermore, the experiment on the «WebArena-Verified» platform showed a stable success rate between 44.7 % and 45.3 % across various model sizes, whereas the success rate of traditional tool calls declined to just 6.7 % when using a 4 billion-parameter model.

This shift carries direct technical and financial implications for software engineering teams and companies in the Gulf, Egypt, and the Arab region. Banking institutions and entities building internal automation systems face a double dilemma: either paying escalating bills for invoking massive cloud models amid data-sovereignty sensitivities and cross-border transfers, or running small local models on their own servers that typically struggle to handle browsing tasks and tool calls accurately. Reducing inference cost by up to roughly 98 % while enabling a small local model of 4 billion parameters to achieve efficiency comparable to large models through code accumulation makes deploying self-contained agents within closed work environments economically and operationally viable without draining infrastructure.

Your move as a technical leader or systems developer is to stop inflating agent context windows with lengthy prompts and start building code structures that surround the model, learn from its errors, and turn them into permanent code.The real efficiency of specialized agents will not come from stuffing context windows, but from isolating repetitive control in auditable, testable code and leaving pure semantic reasoning to the model.