Skip to main content
Create or update docs/en/user_guide/modules/benchmarks/<benchmark-id>.mdx and its localized counterpart. The page should let a user run the benchmark without reading its source.

​
Required Content

Document:
  • Purpose and official source links.
  • Supported dataset and evaluator versions with pinned revisions.
  • Task count, categories or splits, task browser, license and access requirements.
  • Prerequisites, optional dependencies, task images and credentials.
  • Recommended official harness and other compatible harnesses.
  • Supported environments and provider-specific behavior inferred by recipes.
  • Benchmark-specific parameters, defaults, valid values and selection guidance.
  • Benchmark-specific output and metric semantics.
  • One real smoke command and one complete evaluation command.
  • Known compatibility constraints and official-alignment notes.

​
Keep the Page Benchmark-Specific

  • Link generic k, avgk, sample_ids, and aggregation controls to the shared benchmark parameter page.
  • Link harness installation, step limits, cost tracking, command timeouts, and model settings to harness pages.
  • Do not describe the positional model id as a benchmark parameter.
  • Do not repeat generic pass@k or avg@k outputs unless the benchmark defines different semantics.
  • Omit command parameters whose defaults already produce the intended run.
  • Label the upstream alignment path Recommended harness and alternatives Other optional harnesses.
  • Give every alternative harness a complete evaluation command, not a command fragment.
Commands must be copyable and use environment-variable references for credentials and model configuration. Explain which image, workspace, resource, and network fields recipes infer so users do not copy redundant parameters.

​
Preview and Validate

Add the page to docs/docs.json, keep locale paths symmetric unless maintainers approve staged localization, preview it, and validate it: