BotLd test probes inference limits, asking why models cannot package their abilities cheaply

The infrastructure manager quickly discovers that invoking a large language model millions of times to classify a query or sort a recurring transaction is a financial drain that no budget can sustain, but the question posed by the latest research papers posted on the arXiv platform on October 6, 2026 was precisely: does the intelligent agent have the ability to engineer a cheaper self-alternatives and turn its general cognitive capabilities into integrated, repeatable tools?

The concept, which researchers Ankit Sunthalia, Haritz Buerto, Alexander Rubenshtein, Martin Gobre, and Song Jun Oh have named “packaging,” is based on enabling agents to turn their broad capabilities into task-specific solutions that precisely balance answer quality with cumulative cost. To test this hypothesis, the team designed a new evaluation benchmark called “BotLd,” in which models are given an entirely unlabelled workload and are required to complete it under strict time, compute, and API-budget constraints, while retaining the freedom to choose their approach, whether by training a dedicated smaller model or by writing a reusable program.

The results, which covered ten models across three tasks, revealed a sharp gap between the boasting of general intelligence and the ability to reduce costs.The experiments showed that superiority on traditional zero-shot tests does not necessarily translate to efficiency in the packaging process, as 48 out of 60 packaging rounds recorded performance below the lower bound of the 95 % confidence interval for the original model’s performance in its zero-shot setting. Moreover, 31 rounds failed to keep pace with conventional distillation approaches directed at small models within the same token budget, confirming that models’ closeness on general performance metrics collapses entirely when they are asked to engineer their own economically independent solutions.

But when packaging succeeds, the financial savings become sweeping and disruptive to operating economics. In a task classifying query suitability for a product, the “Opus 5” model retained roughly 82 % of its score on the macro-F1 metric while reducing the declared cost by a factor of 657 compared with the cost of a direct single query. It even successfully rivaled the “Jef” model, a system specifically designed for cheap, repeated inference tasks, with “Opus 5” recapturing about 94 % of “Jef”’s performance at a quarter of the total expected operational cost for the entire workload.

This technological shift reshapes the digital spending equation for tech companies and banks in the Gulf and Egypt.Thus, instead of continuously paying fluctuating open-inference bills to global cloud providers for data classification and daily record sorting, packaging research opens the door to converting inference budgets into a one-time investment in training small models or crafting lightweight code that performs the task locally. However, the cautious results of the “BotLd” benchmark compel technology leaders in the region not to chase promises of fully automated distillation, as blind reliance on agents to summarize themselves led to a noticeable drop in result accuracy in more than half of the cases.

Calculate the cost of your repeated calls precisely and compare it to the cost of building a one-time lightweight model; the gap between the two invoices may entirely reshape your project's profitability. What the “BotLd” results demonstrate is that the shift from direct consumption of large models to packaging knowledge within embedded software is the next real arena of competition, yet the success of this path remains contingent on engineering governance that ensures financial savings do not come at the expense of decision quality.