> ## Documentation Index
> Fetch the complete documentation index at: https://agent-compass.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Runtime

> The orchestration layer that turns one request into a bounded evaluation run.

Runtime is the center of AgentCompass. It takes CLI flags or Python arguments, normalizes them into a `RunRequest`, loads registered components, executes tasks with bounded concurrency, and writes durable artifacts.

Use this page when you need to understand what happens after `agentcompass run` starts.

## What Runtime Owns

<CardGroup cols={2}>
  <Card title="Request construction" icon="file-code">
    Merge `config/defaults.yaml`, component defaults, CLI overrides, and Python kwargs into `RunRequest`.
  </Card>

  <Card title="Execution planning" icon="workflow">
    Build an `ExecutionPlan` for each task and let recipes rewrite it before the environment starts.
  </Card>

  <Card title="Bounded concurrency" icon="gauge">
    Enforce per-run `task_concurrency` and process-global provider limits.
  </Card>

  <Card title="Artifact persistence" icon="database">
    Write run info, task details, summary, progress events, logs, and optional analysis output.
  </Card>
</CardGroup>

Runtime does not implement benchmark scoring, agent behavior, provider SDK calls, or badcase diagnosis. Those belong to benchmarks, harnesses, environments, and analyzers.

## Core Objects

| Object                     | Purpose                                                                                        |
| -------------------------- | ---------------------------------------------------------------------------------------------- |
| `RunRequest`               | The full request: benchmark, harness, environment, model, execution controls, output controls. |
| `ExecutionSpec`            | Task concurrency, enabled recipes, retry count, and analysis settings.                         |
| `ExecutionPlan`            | Per-task plan after benchmark and harness planning, before and after recipe rewrites.          |
| `UnifiedEvaluationRuntime` | The in-process executor used by CLI and Python API.                                            |
| `ProgressEvent`            | Structured event emitted during task loading, setup, execution, summary, and release.          |

## Execution Flow

```text theme={null}
run_evaluation(...) / agentcompass run
  -> build RunRequest
  -> load built-in registries
  -> benchmark.load_tasks()
  -> benchmark.select_tasks()
  -> load reusable details
  -> run pending tasks with task_concurrency
  -> process and save summary
  -> optionally save analysis summary
```

For each pending task:

```text theme={null}
planner builds ExecutionPlan
  -> matching recipes apply
  -> environment.open()
  -> benchmark.prepare_task()
  -> harness.start_session()
  -> harness.run_task()
  -> benchmark.evaluate()
  -> analyzers run if enabled
  -> environment.close()
```

## CLI Usage

```bash theme={null}
agentcompass run \
  screenspot \
  qwen3vl_gui \
  qwen3-vl \
  --env <env-provider> \
  --benchmark-params '{"category":"desktop"}' \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY" \
  --model-api-protocol openai-chat \
  --task-concurrency 8 \
  --results-dir results \
  --data-dir data
```

The three positional arguments select `benchmark`, `harness`, and `model`. The environment, model endpoint, concurrency, and output directories become fields on the same `RunRequest`.

## Python Usage

```python theme={null}
from agentcompass import run_evaluation

result = run_evaluation(
    benchmark="screenspot",
    harness="qwen3vl_gui",
    model="qwen3-vl",
    environment="host_process",
    benchmark_params={"category": "desktop"},
    model_base_url="https://your-endpoint/v1",
    model_api_key="sk-...",
    model_api_protocol="openai-chat",
    task_concurrency=8,
    results_dir="results",
    data_dir="data",
)
```

The return value contains aggregate metadata, metrics, summary fields, output paths, and applied recipes. Per-task details remain on disk so large evaluations do not need to keep every artifact in memory.

## Configuration Precedence

| Source                    | Role                                                                          |
| ------------------------- | ----------------------------------------------------------------------------- |
| `config/defaults.yaml`    | Project defaults for runtime, execution, components, providers, and analysis. |
| Component defaults        | `benchmarks.<id>`, `harnesses.<id>`, and `environments.<id>` blocks.          |
| CLI flags / Python kwargs | Highest-priority run-specific overrides.                                      |
| Recipes                   | Final provider-aware plan rewrites before environment startup.                |

## Common Patterns

| Pattern                | How to express it                                                                                     |
| ---------------------- | ----------------------------------------------------------------------------------------------------- |
| Run one sample         | `--benchmark-params '{"sample_ids":["task-id"]}'`                                                     |
| Run with a provider    | `--env <env-provider>`; recipes may infer image and workspace for supported benchmark/provider pairs. |
| Increase throughput    | `--task-concurrency 32`, then confirm model endpoint and provider limits.                             |
| Reuse existing results | Use output reuse options from the CLI or config; runtime skips already materialized details.          |
| Post-analyze results   | Add `--enable-analysis` during a run or use `agentcompass analysis` later.                            |

## Developer Notes

Most extensions do not need to modify runtime. Add or change a benchmark, harness, environment, recipe, or analyzer first. Touch runtime only when you need a new cross-cutting execution concern such as a new progress event, new output persistence behavior, or a new request-level control.

## Related Pages

* [Python API](/reference/python_api)
* [CLI reference](/reference/cli)
* [Configuration module](/key_modules/configuration)
* [Configuration](/reference/configuration)
* [Architecture](/developer/architecture)