ResearchHarness

The researchharness harness runs ResearchHarness as a research agent inside a prepared environment. It runs ResearchClawBench and other long-form research benchmarks such as SGI Deep Research.

How it works

• Prepare ResearchHarness. According to install_strategy, the harness uses an existing package or runs install_command. The installed package must be ResearchHarness 0.0.49 or later.
• Run the research loop. AgentCompass injects a runner into the task environment, passes the prepared prompt and workspace, and launches the ResearchHarness agent. max_rounds limits agent rounds, while timeout limits the maximum runtime of one task.
• Configure retrieval services. Serper powers web and scholar search, Jina fetches web content, and MinerU parses PDF documents. Their credentials are supplied by dedicated harness parameters or same-named environment variables.
• Collect results. The harness converts ResearchHarness events into the standard trajectory, final answer, execution status, and requested output files in RunResult.

Parameters

Pass harness configuration with --harness-params '{...}', or place it under harness.params in the YAML supplied to --config; command-line values take precedence.

Parameter reference

Parameter	Type	Default	Choices / values	Description
install_strategy	string	install_if_missing	preinstalled / install_if_missing / install_always	How to prepare the ResearchHarness package.
install_command	string	python3 -m pip install researchharness	shell command	Command used when the selected strategy installs the package.
install_timeout	int / null	900	integer ≥ 1 / null	Install-command timeout in seconds.
timeout	int	10800	integer ≥ 1	Maximum runtime of one task in seconds.
max_rounds	int	500	integer ≥ 1	Maximum agent rounds for one task.
llm_request_timeout_seconds	int	1200	integer ≥ 1	Timeout for one model request in seconds.
webfetch_tool_timeout_seconds	int	300	integer ≥ 1	Timeout for one WebFetch call in seconds.
readpdf_tool_timeout_seconds	int	300	integer ≥ 1	Timeout for one ReadPDF call in seconds.
max_output_tokens	int	16384	integer ≥ 1	Maximum output tokens for one model request.
max_input_tokens	int	131072	integer ≥ 1	Maximum input tokens for one model request.
compact_trigger_tokens	string	96k	positive integer or k suffix	Token threshold that triggers context compaction.
serper_api_key	string	API key or environment reference	Credential used by web and scholar search.
jina_api_key	string	API key or environment reference	Credential used to fetch web content.
mineru_token	string	token or environment reference	Credential used to parse PDF documents.
extra_tools	list	[]	tool-name list	Additional ResearchHarness tools, for example str_replace_editor.

Retrieval and parsing API keys

serper_api_key, jina_api_key, and mineru_token default to ${SERPER_API_KEY}, ${JINA_API_KEY}, and ${MINERU_TOKEN}. Set those variables in the process that starts AgentCompass, or pass explicit values through --harness-params. Dedicated parameters take precedence over generic environment overrides.

Run examples

Default configuration

Set SERPER_API_KEY, JINA_API_KEY, and MINERU_TOKEN in the shell, then run ResearchClawBench with the ResearchHarness defaults.

agentcompass run \
  researchclawbench \
  researchharness \
  "$MODEL_NAME" \
  --env host_process \
  --benchmark-params '{
    "judge_model": {"id": "your-judge-model", "base_url": "https://your-judge-endpoint/v1", "api_key": "sk-…", "api_protocol": "openai-chat"}
  }' \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY"

Custom parameters

Set explicit research limits and pass the external-service credentials through harness parameters.

agentcompass run \
  researchclawbench \
  researchharness \
  "$MODEL_NAME" \
  --env host_process \
  --benchmark-params '{
    "judge_model": {"id": "your-judge-model", "base_url": "https://your-judge-endpoint/v1", "api_key": "sk-…", "api_protocol": "openai-chat"}
  }' \
  --harness-params '{
    "max_rounds": 600,
    "timeout": 14400,
    "llm_request_timeout_seconds": 1800,
    "serper_api_key": "${SERPER_API_KEY}",
    "jina_api_key": "${JINA_API_KEY}",
    "mineru_token": "${MINERU_TOKEN}"
  }' \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY"

Output

For each task, ResearchHarness returns a RunResult containing the execution status, final answer, normalized trajectory, requested output files, and raw ResearchHarness events and logs. The selected benchmark writes aggregate metrics and per-task details under results/<benchmark>/<model>/<run>/; see Results.