Results Overview

Once an evaluation request starts writing output, it stores task results, run records, aggregate metrics, and logs in one run directory. This page introduces that directory and helps you find the right file for what you want to inspect. The following pages document the fields and usage of each artifact type.

If an evaluation fails preflight before the run directory is created, or if you use launch --dry-run, no result directory is generated.

Directory Layout

A complete run typically generates the following directories and files:

results/
  [<run-name>/]
    <benchmark>/
      <model>/
        <run-id>/
          details/
          retry_details/
          logs/
          run_info.json
          params.json
          progress.json
          progress.jsonl
          .summary_counts.json
          summary.md
          analysis_summary.json
          analysis_summary.md

If run-name is not set, that path segment is omitted. retry_details/ appears only after a runtime retry is actually triggered. Analysis summaries appear only when there are analysis results to aggregate. If a run stops during preflight, task execution, or summary generation, its directory may contain only the artifacts written up to that point.

Where to Start

What you need	Page	Main artifacts
Inspect an individual task’s answer, score, error, trajectory, or retry records	Task Results	details/*.json, retry_details/*.json
Confirm the request, final state, and progress of a run, or troubleshoot it with logs	Run Records and Diagnostics	run_info.json, params.json, progress.json, progress.jsonl, logs/*.log
Review aggregate metrics or post-evaluation analysis for the complete run	Summary and Analysis Results	summary.md, .summary_counts.json, analysis_summary.json, analysis_summary.md

details/*.json stores the per-task results that were written to disk, while summary.md presents run-level aggregate metrics. The first summary at the end of an evaluation uses the results collected during that run; a later agentcompass summary invocation rereads the detail files instead. When analysis is enabled, output for each evaluation attempt is stored under analysis_result in the detail file and then aggregated into run-level analysis summaries. Progress files, logs, and retry_details/ are primarily for monitoring and troubleshooting; they do not directly contribute to Benchmark metrics.

Data, Cache, and Output Directories

Benchmark data and evaluation results are stored in different directories. Use this table to choose the appropriate setting:

Setting	Default	Purpose
runtime.data_dir / --data-dir	data	Stores downloaded datasets, caches, and prepared Benchmark data. These files are not run results.
runtime.results_dir / --results-dir	results	Sets the common root directory for all run results.
--run-name	empty	Adds an optional namespace below the result root so you can group related runs.
--run-id	generated automatically	Sets the directory ID for this run. The same path cannot contain an existing run with that ID.

In a configuration file, use runtime.data_dir and runtime.results_dir to set the root directories. For a single evaluation request, you can also pass the corresponding CLI options. Because run-name and run-id are output settings for an individual request, place them under that request’s output in a multi-evaluation orchestration file. See agentcompass run and agentcompass launch.

Related Pages

• Run Controls
• agentcompass summary
• agentcompass analysis