Codex

The codex harness runs the OpenAI Codex CLI non-interactively in a benchmark-prepared workspace.

How it works

• Prepare Codex CLI. If codex is absent from the image, the harness applies install_strategy and install_command.
• Configure and run. The harness writes a session-scoped Codex provider configuration and invokes the CLI non-interactively in the task workspace. Recipes may place this configuration outside the repository so it cannot enter a generated patch.
• Collect results. Codex JSON events are normalized into the trajectory, final answer, and execution status in a RunResult.

Parameters

Pass harness options with --harness-params '{...}'. The table exposes the installation, provider, and execution behavior that normally needs tuning; path and upload details use built-in defaults.

Parameter reference

Parameter	Type	Default	Choices / values	Description
install_strategy	string	install_if_missing	preinstalled / install_if_missing / upload	How to prepare Codex.
install_command	string	npm install -g @openai/codex	shell command	Command used when Codex is missing.
sandbox	string	workspace-write	read-only / workspace-write / danger-full-access	Codex sandbox policy.
reasoning_effort	string	unset	provider-supported effort	Value written to Codex as model_reasoning_effort.
timeout	int / null	unset	integer ≥ 1 or null	Wall-clock timeout for the complete Codex task.
dangerously_bypass_approvals_and_sandbox	bool	true	true / false	Whether to pass —dangerously-bypass-approvals-and-sandbox. Set it to false only when the Codex approval and sandbox flow must be retained.

Model API

Codex requires an OpenAI-compatible --model-base-url and --model-api-key. --model-api-protocol openai-responses selects the Responses API and openai-chat selects Chat Completions; the CLI protocol takes precedence over the harness’s internal wire_api fallback. The harness appends /v1 when it is absent from the base URL.

Run examples

Default

Run with the default installation and workspace-write policy.

agentcompass run \
  swebench_verified \
  codex \
  "$MODEL_NAME" \
  --env docker \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY" \
  --model-api-protocol openai-responses

Custom params

Use the Chat Completions endpoint and adjust the sandbox.

agentcompass run \
  swebench_verified \
  codex \
  "$MODEL_NAME" \
  --env docker \
  --harness-params '{"sandbox":"workspace-write"}' \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY" \
  --model-api-protocol openai-chat

Output

The harness returns a RunResult per task: trajectory, final answer, and execution status. Per-task details and aggregate metrics are written by the benchmark under results/<benchmark>/<model>/<run>/ (see Results).