Generator and Selector: Instant-step evaluation boosts accuracy of terminal task agents by up to 10 percent

Expanding computation at inference time drives software agents to new performance levels, but the cost of errors in long-running tasks remains the biggest obstacle to their operational adoption. A research paper deposited by researcher Jalaaj Bahandari and colleagues, including Eric Sheng and Rubish Srivastava, on the arXiv platform in early November 2026 showed that applying a “top-N” sampling mechanism at the single-step level yields a qualitative leap in the efficiency of large language model agents when executing complex, long-horizon tasks. The idea works by decomposing decision-making at each interaction round, so that no single model both generates the action and executes it directly in the software environment without prior scrutiny.

The research team adopted a generator-selector architecture using two independent language models. At each turn, the generator model proposes a fixed number of alternative work paths, while the selector model evaluates these options and picks the best one for actual execution. This methodology builds on a previous study published by Zenolina and colleagues in 2025 that relied on specialized critic models to guide search at inference time, whereas the new study tests the capability of open-weight, off-the-shelf language models to compare candidate actions and select the most suitable without needing additional critic model training.

Experiments on long-term downstream tasks revealed that instantaneous selection raises execution accuracy by between 5 and 10 percent, while also improving result stability across repeated attempts.The gains were not limited to the final success rate; analysis of agent trajectories showed a clear shift in decision-making patterns, manifested as fewer attempts needed to complete the task, tighter adherence to problem instructions, and the emergence of self-inspection, self-correction, and code-testing behaviours before proceeding to subsequent steps, behaviours that traditional systems, which act rashly via immediate commands, lacked.

This shift has direct practical implications for engineering teams and tech firms in the Gulf, Egypt, and the Levant that build agents to automate cloud infrastructure and manage complex technical operations. Instead of relying exclusively on costly closed-source models or expending resources to train specialized evaluation models, the framework enables running open-weight models locally within sovereign data centres while distributing workload between generation and ranking. Although generating multiple options per step increases instantaneous compute cost, avoiding random repetition and drift in downstream tasks reduces overall waste and gives organizations execution reliability that sensitive operational applications have previously lacked.

The research findings confirm that boosting agent intelligence is not solely about enlarging model sizes; it is tightly linked to how the inference step itself is engineered. Inserting an evaluative filter between proposing an action and executing it gives autonomous systems a chance to backtrack and review, a decisive difference between an agent that wastes resources correcting accumulated errors and a software system that understands the consequences of its step before sending it to the endpoint.