MCR-Bench evaluates multi-round code review as language model accuracy drops sharply across successive software revisions

A new research paper accepted at the ISSTA 2026 conference reveals a fundamental deficiency in the ability of large language models to simulate real-world code review processes, as benchmark results show a sharp decline in model performance when moving from single-round code inspection to tracking the defect lifecycle across consecutive interactive rounds between developers and reviewers.

Conducted by a research team comprising Diwu Zheng, Yanlin Wang, Shiyun Wang, Kefeng Duan, Hongyu Zhang, Shilin Liu, Yuchi Ma, and Zibin Zheng, the study introduced an evaluation benchmark named MCR-Bench as the first defect-state-aware benchmark specifically designed to evaluate multi-round code reviews. The benchmark was developed to address an existing gap in prior research, which treated code review as a static, single-round decision while overlooking the cumulative and interactive dynamics that govern software development in real production environments.

Covering five common programming languages, the benchmark contains 2,269 real-world multi-round review tasks equipped with precise metadata and defect state annotations across the various rounds.This data includes descriptions of each defect, its type, and its severity, alongside chronological tracking of its progression and state changes with each revision submitted by the developer in response to review comments.

Extensive evaluations of leading language models demonstrated clear limitations in their overall performance, both in identifying defect locations and in tracking defect lifecycles, with a notable drop in accuracy as the number of review and interaction rounds increased. The findings also highlighted the models' sensitivity to defect type and complexity, with the likelihood of missing semantically complex defects or those with low visibility within the codebase doubling compared to explicit, superficial errors.

The researchers' in-depth analysis examined the causes of failure, false alarms, and unintended oversights, pinpointing the issue to two primary vulnerabilities: temporal misalignment in connecting context across consecutive rounds, and the models' long-term memory deficits in accurately tracking cumulative changes throughout the review process.

These findings prompt software engineering leaders and development teams across the Gulf, Egypt, and the Levant to reassess their reliance on AI agents in audit pipelines and pull request reviews. Relying on language models as autonomous auditing tools introduces risks of subtle semantic defects and vulnerabilities slipping into banking systems and government platforms that require multiple rounds of refinement. This suggests restricting the role of AI agents to initial static checks while retaining human engineers for cumulative rounds, or allocating additional investment toward dedicated memory engineering to preserve contextual consistency and prevent temporal failures across software supply chains.