Continuous Search outperforms model size, research paper solves the dilemma of diagnosing agent failures in long-horizon tasks

Engineering teams that deploy AI agents in complex work environments face a growing operational challenge: the difficulty of tracing the causes of crashes and errors, especially when execution traces span long sequences of steps. As the task horizon expands and massive execution logs accumulate, direct human review becomes impractical, while automated diagnostic tools based on large language models suffer from low accuracy and limited ability to pinpoint the root cause of failure precisely and reliably.

The new paper titled “Root Cause Identification as a Research Problem: Continuous Search for Agent Failures in Long-Horizon Tasks” revealed, by a research team led by Harsh Raj, David Lee, Anas Mahmoud and seven other researchers, a structural flaw in the currently used methods for automated fault diagnosis. The prevailing methodologies rely on a language model acting as a judge that examines the execution trace in a single evaluation round. Although this approach succeeds on short traces, the models fall into the trap of quickly settling for a superficially plausible explanation, leaving decisive evidence distributed across long execution logs unexamined.

The core problem lies in the fact that information related to the root cause of an error is often scattered, temporally dispersed, and completely separated from the failure point visible to the user.To address this flaw, the researchers introduced a framework called “Continuous Search,” an iterative system that pushes the evaluating model through successive rounds to keep exploring and searching for undetermined diagnostic evidence instead of leaping to an early conclusion.

Because existing benchmarks lack large, real execution logs, the team developed a new test suite called “MegaRCA-Mix” comprising fifty failure traces that underwent meticulous human documentation and verification, covering long and complex tasks. The results showed the Continuous Search framework outperforming across various test sets and model families; it increased the F1 score of the GPT-5.5 model on the MegaRCA-Mix benchmark by more than 40 percent, jumping from 0.349 to 0.498.

The most technically striking observation is that smaller models within the same family outperformed larger, higher-ranked models when equipped with the Continuous Search methodology, demonstrating that an intelligent search strategy outweighs mere scaling in model size.

This conclusion has a direct practical impact on software and AI engineering teams in the Gulf, Egypt and Jordan that are currently building autonomous agents to automate processes in banking, logistics and government services. Instead of draining operational budgets on calling massive, costly models to attempt diagnosing agent errors in a single inference request, these results call for redesigning internal monitoring and remediation systems; smaller, cheaper models can be relied upon in directed, iterative inspection cycles, reducing inference costs and giving Arab platform engineers genuine ability to isolate hidden errors in complex workflows before they disrupt customer service.