Skip to main content
Environments are execution providers. They give harnesses a uniform way to run commands, move files, read and write text, expose endpoints, and clean up resources. Benchmarks and harnesses should depend on the EnvironmentSession surface, not on provider SDKs.

What Environments Own

Command execution

Run shell or argv-style commands with cwd, env, timeout, detach, and return-code capture.

File movement

Upload and download individual files or directories between local disk and the environment.

Workspace access

Provide text read/write and default workspace root behavior for prepared tasks.

Provider lifecycle

Open sessions, return optional endpoints, and release local or remote resources.

Session Interface

MethodPurpose
exec(command, shell=False, cwd=None, env=None, timeout=None, detach=False)Run a command and return ExecResult.
upload(src, dst) / download(src, dst)Copy a file across the provider boundary.
upload_dir(src, dst) / download_dir(src, dst)Recursively copy directories.
write_text(path, content) / read_text(path)Modify text files in the environment.
endpoint()Return a service URL for providers that expose one.
close() through providerRelease sandbox, container, or local session state.

Provider Choices

ProviderBest fit
host_processLightweight local smoke tests and benchmarks that do not need isolation.
dockerReproducible local containers when Docker is available.
modalRemote registry-image sandboxes, especially SWE-bench and Terminal-Bench recipes.
daytonaRemote developer-style workspaces and task images.
Select a provider with --env; defaults live under environments.<provider> in config/defaults.yaml. Start with host_process when the task only needs local files or a direct model call, and move to a container or remote sandbox when tasks need isolation or benchmark-specific images. For supported benchmark/provider combinations, recipes infer the image and workspace root from task metadata, so you usually do not need to pass --env-params '{"image":"..."}'.
Per-provider credentials, setup steps, and runnable examples live in the Environments section. For picking a provider on a first run, see the Environment Selection cookbook.

Concurrency And Limits

ControlScope
--task-concurrencyMaximum tasks in flight for one run.
execution.task_concurrencyYAML default for per-run task concurrency.
runtime.provider_limits.<provider>Process-global cap for open provider sessions.
Increase task concurrency only after confirming the model endpoint and provider account can handle the load.

Developer Notes

Add an environment when execution happens in a new provider or runtime substrate. Do not add a new environment just to select a different task image; that belongs in environment.params or a recipe.