Skip to main content
Runtime extensions are new benchmark, harness, environment, recipe, or analyzer components registered with AgentCompass.

Extension Rules

  • Register every component with a stable id.
  • Keep benchmark logic in benchmarks, execution logic in harnesses, provider primitives in environments, and compatibility rewrites in recipes.
  • Add user-facing docs under docs/mintlify; keep English as the primary source and mirror important content under docs/mintlify/zh.
  • Prefer config classes for public parameters so CLI, Python, and YAML defaults stay aligned.
  1. Add or update the component contract.
  2. Register the component.
  3. Add a minimal smoke run.
  4. Document parameters, supported environments, and output shape.
  5. Add a recipe when provider-specific image or workspace selection is needed.