Skip to main content
Results are the durable record of an AgentCompass run. They are designed for both human inspection and later automation: summary regeneration, post-analysis, badcase mining, and result reuse all start from the run directory.

​
Directory Layout

results/
  <benchmark>/
    <model>/
      <timestamp>/
        details/
        logs/
        params.json
        progress.json
        progress.jsonl
        summary.md
        analysis_summary.json
        analysis_summary.md
details/*.json files are the source of truth. Summary files are derived views.

​
Artifact Roles

ArtifactPurpose
details/*.jsonPer-task attempts, status, metrics, prediction, error, trajectory, and analyzer output.
summary.mdHuman-readable run-level metrics and status.
params.jsonRequest and output metadata for reproducibility.
progress.jsonlAppend-only structured progress stream.
progress.jsonLatest progress snapshot.
logs/*.logRuntime logs for setup, execution, release, and errors.
analysis_summary.*Aggregated analyzer output when analysis is enabled or re-run.

​
Local Result Browser

tools/result-browser provides a local web UI for inspecting completed or in-progress run directories. It is served from a source checkout and requires Node.js and npm on the machine running the UI service. From the repository root: 
cd tools/result-browser
npm install
npm run dev
Open the Vite URL printed by npm run dev, usually http://localhost:5173, then enter the absolute run directory path that contains summary.md and details/, for example /path/to/AgentCompass/results/swebench_verified/your-model/20260703_120000. The entered path is resolved on the machine running npm run dev. If you access the UI through SSH port forwarding or a remote forwarded URL, still enter the server-side absolute path to the run directory. To check the production build locally: 
npm run build
npm run preview

​
Summary vs Analysis

summary is lightweight. It reads existing details and recomputes aggregate benchmark metrics. 
agentcompass summary \
  results/swebench_verified/your-model/20260703_120000
analysis is heavier. It re-runs analyzers over task attempts, writes analysis_result.<AnalyzerId> into details, and regenerates analysis summaries. 
agentcompass analysis \
  --input results/swebench_verified/your-model/20260703_120000 \
  --analysis-params '{"analyzers":["ExceptionAnalyzer","TruncationAnalyzer"]}'
By default, analysis copies the input run into a new timestamped sibling. Use --override only when you intentionally want to mutate the original run directory.

​
Reading Details

Per-task detail files should contain enough context to debug without rerunning:
  • task id, category, and benchmark metadata;
  • attempt status and correctness;
  • benchmark metric fields;
  • harness prediction and trajectory;
  • environment or evaluation errors;
  • analyzer output under analysis_result.

​
Reuse Behavior

Runtime can materialize and load existing details before executing pending tasks. This is useful when a run was interrupted or when only a subset needs to be recomputed. Use stable output paths, stable task_id values, and avoid deleting details/ if you want reuse and summary regeneration to work.

​
Data And Output Roots

SettingDefaultMeaning
runtime.data_dir / --data-dirdataBenchmark datasets and prepared data cache.
runtime.results_dir / --results-dirresultsRoot directory for run outputs.
--run-nameemptyOptional namespace under the result root.
--run-idtimestampExplicit run id for the new result directory.
runtime.data_dir and runtime.results_dir are YAML keys in config/defaults.yaml with matching CLI flags. --run-name and --run-id are per-run CLI flags (fields of the RunRequest output spec, not configuration file keys).