Skip to content

SWE Prime framework demonstrates data curation superiority as training coding agents on 10% of trajectories outperforms full datasets

Share
SWE Prime framework demonstrates data curation superiority as training coding agents on 10% of trajectories outperforms full datasets

Listen to this article

Read by Anchor

In the race to develop large language model-based software engineering agents, technical practice has largely focused on gathering the largest possible volume of successful solution trajectories and training models on them through supervised fine-tuning. However, a new research paper reveals that a trajectory's success in resolving an issue does not guarantee the quality of its training signal. Trajectories of agents in real-world software environments often contain redundant, inefficient, or code-risky steps, meaning that training models directly on them introduces behavioral noise that leads models to mimic flawed workflows during problem-solving.

To bridge this gap between successful outcomes and trajectory integrity, a research team proposed SWE-Prime, a hierarchical data selection framework operating across two complementary filtering tiers.The first stage conducts a comprehensive screening at the full trajectory levelbased on execution quality, outcome quality, and how representative the trajectory is of software problems, selecting a high-quality, representative subset of successful solutions.The second stage moves to the semantic segment levelby grouping sequential steps into functional blocks and evaluating each segment based on its direct contribution to the final solution, its learnability, and its inherent risks.

The framework employs a selective loss calculation method during fine-tuning, keeping all segments in sequence to maintain contextual coherence while restricting loss computation exclusively to the chosen, beneficial segments. Empirical evaluations on the SWE-Bench Pro and SWE-Bench Verified benchmarks demonstrated the effectiveness of this approach: training on a subset comprising just 10% of selected trajectories outperformed training on the full corpus of successful trajectories, achieving relative performance gains of up to 12.2% on the first benchmark and 24.2% on the second.

This shift carries direct practical implications for development teams and technical innovation centers in the Gulf, Egypt, and the Levant working to train and fine-tune local coding agents within enterprise environments. Relying on a curated 10% subset of top-quality trajectories lowers compute costs and training time to a fraction of standard budgets, while protecting teams from wasting resources on collecting and storing noisy, disorganized trajectory data. Instead of focusing on amassing bulk raw data, regional skill demands are shifting toward engineering filtering criteria and semantic evaluation of model steps, ensuring the deployment of coding agents that follow clean, safe solution paths within sensitive software architectures.

Don't miss the next story

Subscribe for updates