Benchmarks Overview

Benchmark ids, tags, and execution shapes.

AgentCompass lists benchmark reference pages alphabetically. Capability labels are metadata tags rather than sidebar categories because many benchmarks touch more than one evaluation dimension.

Registered Benchmarks

The public reference currently covers 21 benchmark entries.

Benchmark	Tags	Typical execution
browsecomp	Deep Research, Web Browsing, Judge-Scored	local with judge
browsecomp_zh	Deep Research, Web Browsing, Chinese, Judge-Scored	local with judge
deepsearchqa	Deep Research, Search QA, Judge-Scored	local with judge
frontierscience	Deep Research, Science, Judge-Scored	local with judge
gaia	Deep Research, Tool Use, File Reasoning, General Assistant	local with judge
gdpval_ac	Productivity, Long-Horizon, Judge-Scored	remote environment
hle	Deep Research, Reasoning, Judge-Scored	local with judge
hle_verified	Deep Research, Reasoning, Verified, Judge-Scored	local with judge
pinchbench	Productivity, OpenClaw, Agentic Coding	remote environment
researchclawbench	Deep Research, Scientific Research, Workspace, Agentic Coding	local or remote workspace
scicode	Agentic Coding, Scientific Coding, Tool Use	local
screenspot	GUI Grounding, Vision	local
sgi_deep_research	Deep Research, Scientific Research, Judge-Scored	local with judge
skillsbench	Terminal, Skills, Productivity, Agentic Coding	remote environment
swebench_multilingual	Agentic Coding, Repository Repair, Multilingual	remote environment
swebench_pro	Agentic Coding, Repository Repair, Long-Horizon	remote environment
swebench_verified	Agentic Coding, Repository Repair, Verified	remote environment
terminal_bench_2	Terminal, Tool Use	remote environment
terminal_bench_2_1	Terminal, Tool Use	remote environment
terminal_bench_2_verified	Terminal, Tool Use, Verified	remote environment
wildclawbench	Productivity, OpenClaw, Long-Horizon, Tool Use	container workspace

Runtime Controls

Shared benchmark params include sample_ids, category, k, avgk, and resume where supported. Use recipes for heavyweight benchmarks so task images and workspace roots come from task metadata instead of CLI overrides.