We Act architecture solves agents’ catastrophic forgetting by dynamically switching model weights

Intelligent agent systems with long-running tasks face a recurring technical dilemma when trained on multiple skills, as updating shared parameters to learn new capabilities causes previously acquired skills to degrade gradually through catastrophic forgetting. Although executing complex tasks requires varying capabilities at each stage of the operational pipeline, traditional practices based on joint training or fixed LoRA merging impose a static parameter setting throughout execution, limiting the agent’s flexibility and stripping the system of moment-to-moment adaptability to divergent requirements.

A research team including Roy Shi, Shunjian Yin, Jiatseng Sang, Zichuan Liu, Shuian Zhu, presented a new paper on arXiv titled “We Act: Dynamically Switching Weights as Actions for the Smart Agent”. The proposed architecture trains independent LoRA adapters, each devoted to a specific capability, on top of a frozen base model, and then incorporates a selection mechanism for the appropriate adapter within the agent’s available action space. These internal actions leverage the model’s native tool-calling interface, enabling the combination of capabilities and the on-demand invocation of specialized weights within a unified context, allowing the agent to treat weight switching as a direct functional action just like any programmatic call to an external tool.

The experimental results on the Qwen3-32B model across eight benchmark tests in deep-search tasks showed a clear superiority of the new architecture; it recorded a 6.2 percent increase in average accuracy compared with fully fine-tuned models, despite using less training compute. The system also achieved a qualitative accuracy jump of 27.5 percent relative to the strongest traditional methods based on fixed LoRA (Low-Rank Adaptation) adapters. These data confirm that forcing the model to keep a single set of weights throughout the long search trajectory was the main bottleneck for agent accuracy, compared with allowing the agent to adjust parameters itselfThe experimental results on the Qwen3-32B model across eight benchmark tests in deep-search tasks showed a clear superiority of the new architecture; it recorded a 6.2 percent increase in average accuracy compared with fully fine-tuned models, despite using less training compute. The system also achieved a qualitative accuracy jump of 27.5 percent relative to the strongest traditional methods based on fixed LoRA (Low-Rank Adaptation) adapters. These data confirm that forcing the model to keep a single set of weights throughout the long search trajectory was the main bottleneck for agent accuracy, compared with allowing the agent to adjust parameters itself when moving between analysis and retrieval steps.

The new architecture decouples the direct link between expanding the agent’s capabilities and fully retraining the base architecture.Researchers have demonstrated the system’s effectiveness in continuous adaptation by adding a new computational adaptor dedicated to writing and generating code to expand the agent’s task scope, alongside updating the existing writing and editing adaptor to improve the quality of the final reports produced. These two updates were applied without altering the frozen core model or changing any of the other independent adaptors, providing practical evidence that agents can be developed through accumulative modular units that evolve continuously without jeopardizing existing knowledge and skills.

This architectural shift requires an immediate engineering review of the technical team priorities in Gulf banks and digital institutions in Egypt and the Levant, which are currently building agents for automating financial auditing, legal contract vetting, or marketThis architectural shift requires an immediate engineering review of the technical team priorities in Gulf banks and digital institutions in Egypt and the Levant, which are currently building agents for automating financial auditing, legal contract vetting, or market data analysis. Instead of draining expensive cloud computing budgets on full retraining of large models, or constructing complex chains of multiple models that double response time and increase the likelihood of failure, developers can train lightweight, independent adapter packages and install them on a single base model. This approach allows a noticeable reduction in inference and on-premises operating costs, and shifts the software workload toward designing action spaces and invoking tools, rather than entering a vortex of managing the catastrophic forgetting of unified models.

Enabling a smart agent to switch instantly among its specialized weights shifts software model architecture from rigid, closed blocks to a flexible, cumulatively extensible system, where efficiency results from managing specialized skills rather than inflating transaction sizes.