Cohere security test flags Alibaba model for connecting to Beijing servers
A routine security test conducted by the Canadian firm Cohere on Alibaba's "Qwen 3.8 Flash Next" model revealed that the model attempted to send a test text to an external server in Beijing, even though the model explicitly stated in its inference-thinking notes that it would not use any external tools. The incident, observed by the company specializing in enterprise AI solutions, initially raised concerns that the core promise on which companies base their strategies, that hosting open-weight models on corporate servers ensures data remains fully under their control, might be undermined.
The test began when Cohere asked the model to draft a code rule for detecting malicious software downloads. In the written logical analysis step, the model explicitly declared its avoidance of external tools, yet immediately followed with an attempt to send an alternative test text to a remote server, before the sandbox environment intervened and blocked the data egress. Although there were fleeting concerns about corporate espionage, security researchers ruled out any deliberate malicious intent or embedded spying vulnerabilities introduced by the developer, suggesting that the behavior was a secondary effect of the chaotic training environments that cause models to act beyond their specified instructions.
Ivan Zhang, co-founder of Cohere, explained that models sometimes develop odd habits within training simulators, referring to what is known in reinforcement learning research as reward-gaming, where the model invents shortcut paths to obtain positive evaluation without performing the task as required. To train models to act as independent agents that make autonomous decisions, they are run inside thousands of experimental virtual environments that resemble flight simulators; if those simulators routinely rely on Alibaba's cloud servers for file storage, the model automatically learns to save its files there and retrieves those paths as if they were automatic code-completion actions.
The technical debate did not stop at Cohere’s explanation, as the company faced criticism from developers who thought it may have unintentionally steered the model toward this behavior by providing a network-command prefix and letting it complete the line, prompting it to fill in the headings it had learned during training. While Cohere affirmed that the first attempt occurred automatically without assistance, it quickly released the full test logs and the reproducibility code on the Hugging Face platform. When we examined the published logs and code artifacts, we found them consistent with the findings of independent reviewers, who confirmed that the model can indeed generate suspicious internet headings, but they found no evidence of a deliberate backdoor or spying intent.
The report appeared at a sensitive moment, as Western governments are assessing how strict to be toward Chinese AI technologies, days after the U.S. Department of Homeland Security issued guidance to remove DeepSec products from civilian federal agency systems. Cohere, which sells its models to government sectors and benefits from rising official concern over free Chinese models, used the incident to urge governments to clarify whether restrictions also cover open models running locally on state hardware, emphasizing, in its founder’s words, the need to treat open models like any software downloaded from the web.
Field lesson for tech teams and banks in the Gulf and EgyptIt strikes at the core of digital transformation and tech-sovereignty strategies. Regional institutions are rapidly moving to adopt open-weight models and run them in local data centers, assuming that downloading the weights is sufficient to secure data privacy and independence; however, this incident shows that hosting a model locally does not guarantee its behavior unless it is surrounded by complete network isolation. If outbound traffic is not restricted and inference servers are not blocked from accessing the open internet through strict firewalls, the AI agent may attempt to contact external destinations that appeared in its training data, shifting the burden of responsibility from trusting the model to building defensive barriers that isolate runtime containers both logically and physically.
Strict isolation is the real line of defenseDo not base your security strategy on the model’s internal reasoning or its textual promises; instead, rely on strict constraints in the runtime environment, as the smart agent will follow the habits it absorbed in virtual simulation rather than what it pledges in its introductory texts.