Open model environment expands: NVIDIA and developer community launch local AI agents

Developments in the local AI ecosystem are accelerating with the launch of August activities that highlight NVIDIA's collaboration with open development communities. This approach focuses on enabling developers and interested parties to build, customize and run AI agents on local devices, relying on a new suite of open-weight models, software tools and accelerated computing infrastructure that includes graphics cards, educational libraries and specialized applications.

As part of these launches, NVIDIA announced the Nemotron 3.5 Lightning model, an open model built on a mixture-of-experts architecture with 30 billion parameters, intended for continuously operating AI agents.The model achieves token generation speeds up to four times those of comparable open models, while reducing task completion time by about 30 %.Its open nature lets developers fine-tune it to follow specific writing styles, absorb terminology from domains such as imaging and 3D design, or adapt to enterprise coding standards. NVIDIA also coordinates with runtimes such as vLLM, Ollama, llama.cpp, LM Studio and Unsloth to provide weights in NVFP4 and GGUF formats for a range of local systems, from RTX cards to DGX Spark and DGX Station.

To address rising token-usage costs in enterprise settings without compromising the required level of intelligence, NVIDIA released the open-source routing library NeMo Switchyard. The library automatically directs each step of an agent’s workflow to the most suitable model based on accuracy, speed and cost criteria. Internal benchmark tests indicated that the library helped reduce task-completion cost to roughly one-third of the cost of running the Opus 4.8 model alone, while preserving advanced result accuracy.

In a related development, Meta introduced the Muse Glimmer model, a dense open-weight model with 30 billion parameters and a context window exceeding 120 k tokens, designed specifically for programming tasks and local agents. The model can generate more than 200 tokens per second on desktop machines equipped with a NVIDIA RTX 5090, enabling agents to process documents, private messages and execute multiple tool calls locally without sending confidential data or API keys to the cloud.

NVIDIA supported this expansion with updates to the NVIDIA Sync application via a Cluster Assistant feature that allows two or more DGX Spark systems to be linked through ConnectX 7 ports to form a high-speed compute cluster and automatically distribute workloads. The company also announced that the native Linux ARM64 Chrome browser will be available for one-click installation from the DGX Spark console during August, along with a tool for real-time monitoring of CPU and GPU utilization.

Parallel announcements also introduced other open models and tools, including the Cosmos 3 Edge model with 4 billion parameters for robotics and autonomous vehicles, the MiniMax H3 generative model for synchronized video and audio, the Laguna S 2.1 programming model with 118 billion parameters from Poolside AI, the updated DeepSeek V4 Flash model, the Inkling Small multimodal model, the Unsloth Desktop application that combines local model training and inference, the Wan Animate 2 model for motion transfer from Alibaba, and the LTX 2.5 model for video generation.