“DreamFly”: a diffusion framework that addresses vision-and-planning context with receding horizon in AI aerial navigation

Researchers Yan Ding and Fai Shui introduced a new framework for vision-and-language-based aerial navigation, called DreamFly, which is primarily based on the Dream-VLA architecture. The study, registered in the arXiv database under the identifier arXiv:2608.12308v1, within the fields of computer vision and artificial intelligence, aims to enable physical agents to perform aerial navigation tasks guided by visual and linguistic instructions under partial environmental observation conditions. These advanced tasks require the system to integrate extended visual evidence over time, plan future steps, and accurately determine the moment when the agent reaches the specified navigation goal.

The research paper, which spans 24 pages and includes 6 illustrative figures and 3 numerical tables, explains that adapting vision, language, and action (VLA) models to aerial navigation faces three major challenges. The first challenge lies in the narrow and limited historical context available to the system, while the second challenge relates to the short planning horizons available for path calculation. The third challenge stems from the unreliability of implicit termination and stopping mechanisms, which are obstacles to achieving safe and accurate aerial navigation when transitioning from environmental perception to actual action generation.

To address the problem of limited historical context, the DreamFly framework incorporates a component known as causally aligned historical memory. This memory enhances the agent's current visual representation by using observations and visual data that preceded the current decision step exclusively. This causally aligned design enables the system to perform temporal inference across a sequence of scenes without any information or data leakage from future steps to the current decision step, ensuring that decisions are made based on what was actually observed in previous steps.

In terms of formulating a movement plan, the framework relies on planning through horizon-retracted spreading. In this strategy, the policy predicts a block of consecutive actions of length K, but the agent only executes the first action, then immediately replans in the next step. This mechanism is known as K-step planning and one-step execution, where future actions are used as auxiliary planning goals, while maintaining closed-loop visual feedback to continuously adjust the path based on changes in vision.

Regarding the termination and stopping mechanism upon arrival, the researchers developed the LiteStop mechanism, which directly estimates the stopping probability from the logit values of actions at the first fully masked state. This mechanism decouples the explicit termination process from action generation, eliminating the uncertainty associated with implicit stopping and ensuring accurate stopping upon reaching the desired navigation goal.

The experimental evaluations conducted on the OpenFly benchmark showed that the DreamFly framework outperformed all compared methods in both visible and invisible environments. The framework achieved a success rate (SR) of 32.04% in previously visible test groups and 29.46% in invisible test groups. In the path-length-weighted success metric (SPL), the system achieved 28.22% on visible environments and 23.54% on invisible environments. These results demonstrated the effectiveness of jointly modeling historical context, future action structure, and explicit termination.lowest aerial navigation error rateamong all tested models.