BrowseComp

Browse-heavy research benchmark.

BrowseComp measures long-form web research on hard, browse-heavy questions and uses an LLM judge for answer grading.

Runtime Status

Field	Value
Benchmark id	browsecomp
Tags	Deep Research, Web Browsing, Judge-Scored
Execution type	local with judge
Typical harness	researchharness or naive_search_agent
Typical environment	host_process
Current status	registered in the direct runtime

When to Use

Use BrowseComp when you need to measure deep research behavior with the task assumptions described by this benchmark. For large or remote benchmarks, prefer benchmark recipes so images, workspaces, and provider-specific defaults come from task metadata instead of manual CLI flags.

Parameters

Common parameters for this benchmark include:

• category
• answer_type
• judge_model
• sample_ids
• max_concurrency

Shared runtime controls such as k, avgk, sample_ids, resume, and category follow the conventions in Benchmark Parameters.

Run Example

agentcompass run \
  browsecomp \
  researchharness \
  your-model \
  --env <env-provider> \
  --benchmark-params '{"sample_ids":["<task-id>"]}' \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY" \
  --model-api-protocol openai-chat

Adjust the harness and environment to the supported combination for your branch and deployment.

Outputs

Per-task details are written to results/browsecomp/<model>/<run>/details/. Aggregate metrics are written to summary.md in the same run directory.

Notes

The benchmark requires a complete judge_model spec unless the selected config already supplies one.