> ## Documentation Index
> Fetch the complete documentation index at: https://agent-compass.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# SWE-Bench Pro Verified

This paper studies the reliability of software engineering agent evaluation and introduces a verified version of SWE-Bench Pro.

| Field           | Details                                                                                                          |
| --------------- | ---------------------------------------------------------------------------------------------------------------- |
| Paper           | [SWE-Bench Pro Verified: A Reliable Benchmark for Software Engineering Agents](https://arxiv.org/abs/2609.08149) |
| First submitted | 2026-09-08                                                                                                       |
| Topics          | Software engineering, evaluation reliability, reward hacking                                                     |

## Overview

The study identifies two problems that distort evaluation: agents can access leaked solutions or hidden evaluation information, and some tasks contain misleading instructions or unsuitable tests. SWE-Bench Pro Verified combines controls against information leakage with targeted corrections to task descriptions and tests. The paper evaluates the effects of these changes and examines agent trajectories to understand changes in scores.

## Relation to AgentCompass

The paper releases its code in AgentCompass. You can run the benchmark through the `swebench_pro_verified` integration; the [benchmark guide](/en/user_guide/modules/benchmarks/swebench_pro_verified) documents its controls, supported environments, parameters, and evaluation outputs.

## Resources

* [Paper and citation export on arXiv](https://arxiv.org/abs/2609.08149)
* [AgentCompass code](https://github.com/open-compass/AgentCompass)
* [Verified dataset](https://huggingface.co/datasets/opencompass/SWEBench-Pro-Verified)
* [SWE-Bench Pro Verified benchmark guide](/en/user_guide/modules/benchmarks/swebench_pro_verified)

[All AgentCompass research papers](/en/research/overview)