Skip to content

“Every Day Bench” puts live-web agents up against a daily-task test, with the strict success ceiling at 54% for regional teams

Share
“Every Day Bench” puts live-web agents up against a daily-task test, with the strict success ceiling at 54% for regional teams

A new research paper published by scholars on the arXiv platform tests the ability of artificial-intelligence agents to perform daily digital tasks on the live web, revealing that the highest strict success rate among 23 agent-model configurations did not exceed 54.55 %, which places autonomous automation plans of organisations in the Gulf and Egypt in a cautious operational reality that demands a reassessment of full reliance on agents in live work environments. The study, titled “Every Day Bench”, shows that moving from mere information retrieval to delivering outputs suitable for final use remains the biggest obstacle for web-based models.

The new benchmark is based on 77 tasks devised by human researchers and covers eight domains of everyday life and work environments, ranging from product selection and travel planning to conducting research and building digital artefacts. The benchmark does not impose a mandatory workflow or a predefined answer; instead it sets success criteria before execution, leaving each agent free to choose its results and paths, thereby mirroring the complexity of the live web where multiple correct outcomes exist and page data change over time.

To evaluate the outputs accurately, the evaluation methodology combinedthree integrated pillars: deterministic checking of explicit constraints and pre-specified conditions, verification of claims by referring to the web pages cited by the agent and retrieved at the moment of evaluation, and rule-based assessment of the final output quality. The researchers demonstrated the robustness of this protocol by matching results with human judgments, re-evaluating under fixed evidence, and testing controlled failure cases in source linking.

The recorded results showed a marked variation among models, with both “Claude Opus 5.5” and “Claude Fiable 5.1” leading the highest strict pass rate at 54.55 %. In contrast, “GPT-6 Sol” performed between 29.87 % at low effort level and 45.45 % at maximum operational effort in controlled-effort experiments. The tests also indicated thatReliability varies fundamentally with the nature of the task, with “GPT-5.6 Sol” topping research and retrieval tasks, “Fiable 5.1” leading planning tasks, “Opus 5.5” and “Fiable 5.1” tying in comparison tasks, while “GPT-6 Astra” and “Sonnet 5.5” share the lead in generating custom outputs.

What this means for the region

The test results reveal to CTOs and development teams in the Gulf, Egypt and the Levant that betting on fully automated independent web agents for customer service or live reporting still carries an error rate above 45 % even with the strongest available models. Operationally, this gap necessitates maintaining strict human verification layers and refraining from delegating critical decisions or sending live transactions to agents alone at this stage.

The uneven distribution of capabilities also reshapes model-selection criteria for regional automation projects; relying on a single model for all workloads raises cost without guaranteeing quality, as the data show certain models excel in research and retrieval while others outperform in planning or custom output generation, prompting engineering teams to adopt a hybrid routing architecture that assigns the most reliable model to each task based on the required deliverable.

Don't miss the next story

Subscribe for updates