Choose a Harness

Select a registered harness and configure its complete harness-parameter schema.

A harness defines how an agent attempts a prepared task. It may call a model directly, launch a coding or terminal CLI, or integrate an external agent framework. The benchmark still owns task preparation and scoring.

Use the recommended harness on the selected benchmark page when reproducing an official result. Choose another harness only when its page confirms that the model protocol, environment, workspace, and prepared-task contract are compatible.

Find a Harness

Use the live registry to see the harnesses available in your installed AgentCompass revision:

agentcompass list harness

The sidebar links to harnesses with dedicated protocol, installation, parameter, execution, and troubleshooting documentation. For any registry entry, including a newly integrated harness without a dedicated page, inspect its current configuration schema with:

agentcompass config docs harness <harness-id>

Configure Harness Parameters

The General Run Parameter Reference introduces --harness-params <json>. The <json> value contains fields owned by the selected harness:

agentcompass run <benchmark> <harness> "$MODEL_NAME" \
  --harness-params '{
    "<harness-field>": "<value>"
  }'

Unlike benchmark parameters, harness parameters have no shared user-facing fields. RuntimeHarnessConfig provides the configuration contract, but each harness defines its own complete schema according to the agent it integrates.

Harness Field Families

The selected schema commonly contains fields from these families:

Field family	Examples	What it controls
Executable and version	binary, version, framework version fields	Selects the agent executable or pinned framework release.
Installation	install_strategy, install_command, upload_src, upload_dst	Decides whether the task environment must contain the agent, may install it during setup, or receives an uploaded executable.
Agent loop	max_turns, max_iterations, step_limit, iterations	Bounds framework-specific turns, steps, or evolution iterations. These are not AgentCompass task retries.
Timeouts	timeout, command_timeout, request_timeout, conversation or terminal timeouts	Bounds different harness lifecycle layers. Use the selected page to distinguish wall-clock, command, and request timeouts.
Cost and context	cost_limit, cost_tracking, context_window, max_tokens	Controls harness-specific budget and model-context behavior.
Tools and services	tools, tool limits, search keys, service credentials	Enables harness tools and external services required by the agent.
Environment and artifacts	env, artifact_dir, skill_dirs	Passes harness runtime variables or controls harness-owned outputs.

These are field families, not a universal list. A field accepted by one harness may be invalid or have different semantics in another.

Inspect the Complete Schema

Print every accepted field, type, default, and description from the selected harness implementation:

agentcompass config docs harness <harness-id>

The harness page explains valid values, supported protocols and environments, installation behavior, and interactions that cannot be expressed by type and default alone.

Build the JSON Object

For example, mini_swe_agent exposes executable, version, installation, step, cost, timeout, environment, and artifact settings in one object:

{
  "version": "2.4.5",
  "install_strategy": "install_if_missing",
  "step_limit": 250,
  "cost_limit": 3.0,
  "cost_tracking": "ignore_errors",
  "timeout": null,
  "command_timeout": 2400
}

This expanded object demonstrates the schema; it is not a recommendation to repeat defaults. Pass only the fields that must differ from the effective harness configuration.

--harness-params must be valid JSON. CLI values deep-merge over matching keys in harness.params from configuration files. Inspect the merged built-in and configuration-file values before adding the final CLI override:

agentcompass config show \
  --harness <harness-id> \
  --config <config-file>

Keep Parameter Ownership Clear

Agent turn limits and installation settings belong in --harness-params. Dataset filters and verifier behavior belong in --benchmark-params; model inference requests belong in --model-params; sandbox images, resources, and network policies belong in --env-params.