MagniTude launches inference engine for AI agents: local hardware alignment doubles speed of open models

The startup MagniTude launched an open-source inference engine designed to run AI agents, in a move that aims to address the computational bottlenecks faced by lightweight models when run locally on developers’ machines. The project, licensed under the Apache 2.0 license, offers a direct alternative to conventional general-purpose engines such as llama.cpp, Olama and LM Studio, relying on a different philosophy for handling the physical hardware of computers.

Common general-purpose engines typically rely on pre-compiled software packages that cover broad categories of processors, leaving a large portion of chip capacity unused.The new engine dispatches processing cores and auto-tunes them to the device’s actual chip before the model is launched., delivering a performance jump of up to double the speed compared with the llama.cpp library. The project’s technical data reveal a 92 percent acceleration of decoding operations on Apple Silicon processors via the Metal API, and a 19 percent acceleration on NVIDIA chips through the CUDA environment, while providing full support for AMD processors and conventional CPUs without imposing a minimum specification threshold.

The issue in agent environments is not limited to token-generation speed; it also extends to memory consumption and the number of concurrent sessions. The platform tackles this challenge by reducing memory usage by 27 percent per agent, freeing space immediately once the agent’s tasks stop. It also provides a system for sharing the cache of prior context across concurrent sessions, preventing system slowdown when several programming agents run simultaneously to complete complex tasks.

The engine is available as a full-desktop application for macOS, Linux and Windows, and includes an integrated command-line interface that enables one-click linking with popular programming agents such as Cloud Code, Codex, Hermes, Open Code, Py and Klein, alongside an API compatible with OpenAI standards. This integration allows tasks to be executed entirely locally and for free, with commands, files and models remaining on the user’s device and no data sent over the internet after the selected model has been downloaded.

This shift represents a watershed for development teams and startups in the Gulf, Egypt and the Arab region at large. Running programming agents and automated task pipelines in the cloud has traditionally imposed a continual drain on inference budgets through token-usage fees, whereas improved local performance shifts daily development workloads to developers’ computers and desktop workstations at zero inference cost. Moreover, the engine gives banks and other sensitive entities in Riyadh, Abu Dhabi and Cairo a secure path to adopt programming-task agents on their internal databases, bypassing compliance and data-governance requirements that previously prohibited sending sensitive code to public cloud servers.