Sandbox Resource Limits

Configure per-task CPU, memory, storage, and GPU resources without confusing sandbox limits with run concurrency.

Environment resource parameters control the sandbox created for one task attempt. Use them to reproduce benchmark requirements, prevent one task from exhausting a local machine, or request an appropriate remote sandbox size.

Resource limits do not constrain the AgentCompass driver, model endpoint, judge endpoint, or services running outside the selected environment. They are also separate from runtime concurrency and provider scheduling controls.

Understand the Scope

Control	Scope	Configuration
Environment resources	CPU, memory, storage, and GPU assigned to one task or fresh verifier sandbox	Provider fields in --env-params
Task concurrency	Maximum benchmark tasks in flight for the run or orchestration	--task-concurrency
Provider concurrency	Maximum concurrent task attempts assigned to one provider in the AgentCompass process	--provider-limit <provider>=<count>
Environment startup rate	Maximum new environments opened per second	--env-open-qps <provider>=<qps>
Model capacity	Request, token, and concurrency capacity of the external endpoint	Configure and monitor at the model provider

For example, --task-concurrency 8 with cpus: 2 may request up to approximately 16 task CPUs when eight Docker containers are active. Benchmarks that create a fresh verifier sandbox can temporarily keep both task and verifier environments alive within one attempt, so leave headroom rather than sizing the host to the exact product.

Provider Resource Fields

Resource schemas intentionally follow each provider API instead of forcing different providers into one lossy common format.

Provider	CPU	Memory	Storage	GPU	Enforcement notes
host_process	Not configurable	Not configurable	Not configurable	Not configurable	Commands share the AgentCompass host account and operating-system resources.
docker	cpus, positive core count	memory, Docker size such as 6g; optional memory_swap	storage_opt, driver-dependent	gpus, Docker --gpus value	CPU and memory are passed directly to docker run; disk quotas depend on the daemon storage driver.
daytona	resources.cpu, integer cores	resources.memory, integer GiB	resources.disk, integer GiB	resources.gpu, integer count	Availability and accepted sizes depend on the selected Daytona target and account quota.
modal	cpu, number or request/limit pair	memory, MiB, size string, or request/limit pair	Provider/image storage behavior	gpu, Modal GPU request string	Top-level fields are preferred; the alternative resources mapping is normalized before creating the sandbox.

Use agentcompass config docs env <provider> against the installed revision to inspect the exact accepted fields and defaults. Provider pages explain mutually exclusive options and operational constraints that the generated schema cannot capture.

Configure Docker

Set a two-core, 6 GiB container and disable swap by making the combined memory-plus-swap limit equal to the memory limit:

agentcompass run <benchmark> <harness> "$MODEL_NAME" \
  --env docker \
  --env-params '{"cpus":2,"memory":"6g","memory_swap":"6g"}'

memory_swap requires memory. Use -1 only when unlimited swap is intentional. A storage limit such as {"storage_opt":{"size":"20g"}} works only with Docker storage drivers that support per-container size options.

Configure Daytona

Daytona uses one resources object. CPU and GPU values are counts; memory and disk values are GiB:

agentcompass run <benchmark> <harness> "$MODEL_NAME" \
  --env daytona \
  --env-params '{"resources":{"cpu":2,"memory":6,"disk":20}}'

AgentCompass also accepts cpus, storage, and gpus as aliases inside the object, but the canonical singular keys above make the provider units clearest. A request can still fail when the selected Daytona target does not offer that shape or the account has insufficient quota.

Configure Modal

For common fixed requests, use the top-level fields:

agentcompass run <benchmark> <harness> "$MODEL_NAME" \
  --env modal \
  --env-params '{"cpu":2,"memory":"6g"}'

AgentCompass converts memory size strings to the MiB value expected by Modal. Modal also accepts a two-value CPU or memory pair when separate requested and maximum values are needed. The resources mapping supports aliases such as cpu, cpus, memory, memory_mb, memory_gb, gpu, and gpus; explicit top-level cpu, memory, and gpu values take precedence over their mapping equivalents.

Follow Recipe and Override Precedence

Compatible recipes can translate benchmark task metadata into provider resource fields. For example, a task may define CPU and memory in its benchmark metadata while the Docker, Daytona, and Modal recipes convert those values into their provider-specific units.

Explicit compatible --env-params values retain priority over recipe defaults. This gives two useful modes:

• omit resource fields to reproduce the benchmark or recipe-provided task shape; or
• pass deliberate overrides when testing another resource profile.

Inspect run_info.json or the resolved execution plan in each task detail to confirm the values used after recipe adaptation. Record overrides in an alignment report because a lower memory limit, different GPU, or CPU throttling can change completion rates and scores.

Choose Safe Values

1. Start with the benchmark’s official resource metadata or its AgentCompass recipe.
2. Run one representative task and observe peak memory, CPU saturation, disk growth, startup time, and verifier use.
3. Add reasonable headroom for package installation, compilation, caches, and transient verifier processes.
4. Estimate aggregate capacity as per-sandbox resources multiplied by active sandboxes, then set task and provider concurrency below the endpoint and host/provider limits.
5. Increase concurrency gradually; reduce it when startup failures, OOM kills, throttling, or queueing appear.

A resource failure is part of the measured result when the configured limit matches the official evaluation. Do not silently rerun failed tasks with larger resources and combine them with the original run; document the changed setting and start a comparable run or a clearly identified recovery run.

Diagnose Resource Failures

Symptom	Likely cause	Next step
Docker exit code 137, OOMKilled, or abrupt process loss	Container memory limit was exceeded	Inspect container state and peak usage; compare the limit with benchmark metadata before increasing it.
Host becomes unresponsive while containers are healthy	Aggregate concurrency exceeds host capacity	Lower --task-concurrency or --provider-limit docker=<n>.
Daytona or Modal rejects sandbox creation	Unsupported resource shape, account quota, or regional capacity	Verify units, target/region availability, and provider quota; retry one sandbox before scaling.
Task times out with low CPU utilization	Model, network, or harness wait rather than CPU shortage	Inspect phase logs before increasing CPU.
Container filesystem fills	Task artifacts exceed writable-layer capacity	Use a supported storage option, larger provider disk, or a benchmark-compatible image/layout.
GPU is not visible	Runtime, image, driver, or provider request is incompatible	Verify the provider independently and use the exact GPU value accepted by its API.

See agentcompass run for concurrency and retries, and Troubleshooting Runs for phase-based diagnosis.