OpenAI Chat Completions

Connect AgentCompass to OpenAI-compatible Chat Completions endpoints.

Use the openai-chat protocol for endpoints compatible with the OpenAI Chat Completions request shape, commonly exposed at /v1/chat/completions.

Configure the Protocol

export MODEL_NAME=""
export MODEL_BASE_URL="https://your-endpoint.example/v1"
export MODEL_API_KEY="sk-..."

agentcompass run <benchmark> <harness> "$MODEL_NAME" \
  --env <environment> \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY" \
  --model-api-protocol openai-chat \
  --model-params '{"temperature":0}'

The model id is the third positional argument. AgentCompass sends messages through the configured harness and forwards supported --model-params values to the endpoint.

Common Parameters

Parameter	Purpose
temperature	Control sampling randomness when supported by the endpoint.
max_tokens	Limit generated completion tokens.
timeout	Set the request timeout used by AgentCompass direct protocol clients.
max_retries	Set SDK-level retries for AgentCompass direct protocol clients.

Harnesses may add, transform, or restrict model parameters. Check the selected Harness page before copying a configuration between agent implementations.

When to Use It

Choose openai-chat when:

• the provider documents Chat Completions compatibility;
• the selected harness supports openai-chat;
• the evaluation does not require Responses-specific state or request fields.

For stateful Responses endpoints, use OpenAI Responses. For Anthropic-compatible endpoints, use Anthropic Messages.