> ## Documentation Index
> Fetch the complete documentation index at: https://agent-compass.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# agentcompass run

`agentcompass run` creates and executes one evaluation request with the selected benchmark, harness, model, and environment. `BENCHMARK HARNESS MODEL` are positional arguments in a fixed order; select the environment with `--env`.

```bash theme={"system"}
agentcompass run [OPTIONS] BENCHMARK HARNESS MODEL
```

## Run a Minimal Evaluation

The example uses [`sample_ids`](/en/user_guide/modules/benchmarks/overview#shared-benchmark-fields) in `--benchmark-params` to select one benchmark task by its stable task ID so you can quickly verify the component and endpoint configuration:

```bash theme={"system"}
export MODEL_NAME=""
export MODEL_BASE_URL=""
export MODEL_API_KEY=""

agentcompass run \
  swebench_verified \
  mini_swe_agent \
  "$MODEL_NAME" \
  --env docker \
  --benchmark-params '{"sample_ids":["astropy__astropy-12907"]}' \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY"
```

One `run` command corresponds to one evaluation request. To coordinate multiple explicit requests, use [`agentcompass launch`](/en/user_guide/using_agentcompass/cli/launch).

## Parameter Reference

The table below lists all `agentcompass run` parameters, their defaults, and the settings they control. See [Run Controls](/en/user_guide/using_agentcompass/run_controls) for guidance on concurrency, timeouts, retries, output, and debugging. “Built-in default” means the value used before user-level, project-level, or explicit configuration files override it. A conditional parameter is required only when the selected component or endpoint needs it.

### Component Selection and Parameters

| Parameter                                                                                                | Required?   | Built-in default                         | What it controls                                                                                                           |
| -------------------------------------------------------------------------------------------------------- | ----------- | ---------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| [`BENCHMARK`](/en/user_guide/modules/benchmarks/overview#benchmark-list)                                 | Required    | None                                     | Registered benchmark ID. Determines dataset loading, task preparation, verification, and metrics.                          |
| [`HARNESS`](/en/user_guide/modules/harnesses/overview#harness-list)                                      | Required    | None                                     | Registered harness ID. Determines the agent loop or framework used to attempt each task.                                   |
| [`MODEL`](/en/user_guide/modules/models/overview#configure-the-model-spec)                               | Required    | None                                     | Primary model ID and the model-name segment used in the result path. Prefer `"$MODEL_NAME"` in shell commands.             |
| [`--benchmark-params <json>`](/en/user_guide/modules/benchmarks/overview#configure-benchmark-parameters) | Conditional | Selected benchmark defaults              | Overrides the shared and benchmark-specific fields defined by the selected benchmark config.                               |
| [`--harness-params <json>`](/en/user_guide/modules/harnesses/overview#configure-harness-parameters)      | Conditional | Selected harness defaults                | Overrides the complete parameter schema defined by the selected harness.                                                   |
| [`--model-base-url <url>`](/en/user_guide/modules/models/overview#configure-the-model-spec)              | Conditional | `""`                                     | Base URL of the model API that receives inference requests.                                                                |
| [`--model-api-key <key>`](/en/user_guide/modules/models/overview#configure-the-model-spec)               | Conditional | `""`                                     | Credential used to authenticate with the model API.                                                                        |
| [`--model-api-protocol <protocol>`](/en/user_guide/modules/models/overview#model-api-protocol-list)      | Optional    | Unspecified                              | Sets the model API protocol. See the linked page for accepted values and selection rules.                                  |
| [`--model-params <json>`](/en/user_guide/modules/models/overview#configure-the-params-field)             | Optional    | `{}`                                     | Passes fields supported by the selected harness, API protocol, endpoint, and model deployment.                             |
| [`--env <id>`](/en/user_guide/modules/environments/overview#choose-a-provider)                           | Optional    | `host_process`                           | Selects where task commands execute. Common choices are `docker`, `daytona`, and `modal`.                                  |
| [`--env-params <json>`](/en/user_guide/modules/environments/configuration/overview)                      | Conditional | Selected environment and recipe defaults | Overrides shared network policies and the selected provider's image, workspace, resource, credential, or lifecycle fields. |

### Configuration and Recipes

| Parameter                                                                               | Required? | Built-in default         | What it controls                                                                                 |
| --------------------------------------------------------------------------------------- | --------- | ------------------------ | ------------------------------------------------------------------------------------------------ |
| [`--config <path>`](/en/user_guide/using_agentcompass/cli/config)                       | Optional  | None                     | Loads an additional YAML or JSON override file. Repeat to layer multiple files in command order. |
| [`--recipe <id>`](/en/user_guide/other_features/recipes#using-recipes)                  | Optional  | All compatible built-ins | Restricts automatic recipe matching to the specified recipe IDs. Repeat to allow more than one.  |
| [`--recipe-dir <path>`](/en/user_guide/other_features/recipes#trusted-external-recipes) | Optional  | `[]`                     | Loads a trusted external recipe package directory. Repeat for multiple packages.                 |

### Execution Controls

| Parameter                                                                                                     | Required? | Built-in default | What it controls                                                                         |
| ------------------------------------------------------------------------------------------------------------- | --------- | ---------------- | ---------------------------------------------------------------------------------------- |
| [`--task-concurrency <int>`](/en/user_guide/using_agentcompass/run_controls#scale-concurrency-safely)         | Optional  | `32`             | Limits the number of benchmark tasks running concurrently within this process.           |
| [`--max-retries <int>`](/en/user_guide/using_agentcompass/run_controls#retry-only-transient-failures)         | Optional  | `0`              | Retries matching task or scoring failures up to this many additional attempts.           |
| [`--retry-pattern-list <json>`](/en/user_guide/using_agentcompass/run_controls#retry-only-transient-failures) | Optional  | `null`           | Restricts retries to errors matching at least one regex in a JSON string array.          |
| [`--keep-environment`](/en/user_guide/using_agentcompass/run_controls#keep-environments-for-debugging)        | Optional  | Disabled         | Skips environment cleanup so task and verifier sandboxes remain available for debugging. |

### Output and Reuse

| Parameter                                                                                      | Required? | Built-in default  | What it controls                                                                                                                                                              |
| ---------------------------------------------------------------------------------------------- | --------- | ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`--run-name <name>`](/en/user_guide/using_agentcompass/run_controls#name-a-new-run)           | Optional  | `""`              | Adds an optional namespace between `results_dir` and the benchmark directory.                                                                                                 |
| [`--run-id <id>`](/en/user_guide/using_agentcompass/run_controls#name-a-new-run)               | Optional  | Current timestamp | Sets the final run-directory name instead of generating `YYYYMMDD_HHMMSS`.                                                                                                    |
| [`--reuse [run-id]`](/en/user_guide/using_agentcompass/run_controls#resume-an-interrupted-run) | Optional  | Disabled          | Reuses normal task details from the latest run under the same benchmark/model result hierarchy or from the specified run ID. The user must keep measured settings compatible. |

### Process Settings

| Parameter                                                                                                      | Required? | Built-in default            | What it controls                                                                                                                                                                                                    |
| -------------------------------------------------------------------------------------------------------------- | --------- | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`--results-dir <path>`](/en/user_guide/using_agentcompass/run_controls#name-a-new-run)                        | Optional  | `results`                   | Sets the root directory for run outputs.                                                                                                                                                                            |
| [`--data-dir <path>`](/en/user_guide/other_features/results#data-cache-and-output-directories)                 | Optional  | `data`                      | Sets the root directory for downloaded datasets, caches, and prepared benchmark data.                                                                                                                               |
| [`--timeout-seconds <int>`](/en/user_guide/using_agentcompass/run_controls#set-an-appropriate-timeout)         | Optional  | `360000`                    | Sets the overall timeout in seconds for the evaluation execution phase after component preflight. Explicitly set `0` to disable this outer limit. Component-specific command and verifier timeouts remain separate. |
| [`--env-open-qps <provider=qps>`](/en/user_guide/using_agentcompass/run_controls#scale-concurrency-safely)     | Optional  | Local: `0`; remote: `10`    | Limits environment creation rate per provider. Repeat for multiple providers; `0` disables pacing.                                                                                                                  |
| [`--provider-limit <provider=count>`](/en/user_guide/using_agentcompass/run_controls#scale-concurrency-safely) | Optional  | `128` per built-in provider | Sets a process-wide limit on concurrent task executions for the provider, including retries. Repeat per provider; `0` disables the limit.                                                                           |
| [`--progress <mode>`](/en/user_guide/using_agentcompass/run_controls#logs-and-progress)                        | Optional  | `auto`                      | Selects terminal progress output: `auto`, `plain`, or `none`.                                                                                                                                                       |
| [`--log-level <level>`](/en/user_guide/using_agentcompass/run_controls#logs-and-progress)                      | Optional  | `INFO`                      | Sets console logging to `DEBUG`, `INFO`, `WARNING`, `ERROR`, or `CRITICAL`.                                                                                                                                         |
| [`--file-log-level <level>`](/en/user_guide/using_agentcompass/run_controls#logs-and-progress)                 | Optional  | `DEBUG`                     | Sets the run log-file level independently from console logging.                                                                                                                                                     |
| [`--auto-install-dependencies`](/en/user_guide/using_agentcompass/dependencies#automatic-installation)         | Optional  | Disabled                    | Allows AgentCompass to install missing trusted component extras in the active Python environment.                                                                                                                   |

### Analysis

| Parameter                                                                                        | Required? | Built-in default           | What it controls                                                                                    |
| ------------------------------------------------------------------------------------------------ | --------- | -------------------------- | --------------------------------------------------------------------------------------------------- |
| [`--enable-analysis`](/en/user_guide/using_agentcompass/cli/analysis#run-with-evaluation)        | Optional  | Enabled                    | Enables registered post-evaluation analyzers. Use `--no-enable-analysis` to skip them.              |
| [`--analysis-params <json>`](/en/user_guide/using_agentcompass/cli/analysis#run-with-evaluation) | Optional  | Built-in analyzer defaults | Selects analyzers and overrides their concurrency, model, rendering, or analyzer-specific settings. |

### Component-Specific JSON Parameters

The four JSON parameter flags do not share one schema. Their available fields and defaults depend on the selected
component:

| Flag                 | Owner                               | Where to find its fields                                                                                                                              |
| -------------------- | ----------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--model-params`     | Harness, API protocol, and endpoint | [Model parameter schema](/en/user_guide/modules/models/overview#configure-the-params-field) and the endpoint provider's request schema                |
| `--benchmark-params` | Selected benchmark                  | [Benchmark parameter schema](/en/user_guide/modules/benchmarks/overview#configure-benchmark-parameters) and `agentcompass config docs benchmark <id>` |
| `--harness-params`   | Selected harness                    | [Harness parameter schema](/en/user_guide/modules/harnesses/overview#configure-harness-parameters) and `agentcompass config docs harness <id>`        |
| `--env-params`       | Selected environment                | [Environment parameter schema](/en/user_guide/modules/environments/configuration/overview) and `agentcompass config docs env <id>`                    |

[`sample_ids`, `k`, and `avgk`](/en/user_guide/modules/benchmarks/overview#shared-benchmark-fields) all belong in `--benchmark-params`, but they have different roles: `sample_ids` selects tasks, `k` sets the number of independent attempts per task, and `avgk` controls the corresponding mean-metric aggregation. Provider CPU, memory, image, and network settings belong in `--env-params`. See
[Configure an Evaluation](/en/user_guide/using_agentcompass/overview#evaluation-structure) for the conceptual ownership map.