Code Implementation

Implement provider config, session primitives, lifecycle, resources, network enforcement, and precedence.

Implement one provider through the shared environment contract and keep provider-specific behavior out of benchmarks and harnesses.

1. Establish the Provider Contract

Use the provider’s official SDK and API documentation as the source of truth. Record:

• Authentication, region, project, pool, app, or organization scope.
• Image, snapshot, named-image, Dockerfile, or template selectors and mutual exclusions.
• Workspace and filesystem persistence behavior.
• CPU, memory, disk, GPU, placement, and quota fields.
• Sandbox startup, operation, idle, maximum-lifetime, and deletion semantics.
• Public, restricted, and allowlisted network capabilities.
• Command, file transfer, endpoint, cancellation, and error behavior.
• SDK async support and thread-safety guarantees.

Align provider calls with the official API instead of inventing benchmark-specific environment fields.

2. Define a Typed Provider Config

Create the provider under src/agentcompass/environments/. Define a RuntimeEnvironmentConfig subclass for every public field and a BaseEnvironment implementation registered with ENVIRONMENTS.

Separate these concepts in the schema:

Concept	Examples
Authentication	API key, token id/secret, endpoint, organization
Sandbox source	Image, snapshot, named image, template
Lifecycle	Startup timeout, operation timeout, idle timeout, maximum lifetime
Resources	CPU, memory, disk, GPU, placement
Workspace	Default root, mounts, volumes, persistence
Provider metadata	Name prefix, labels, tags, region, pool

Do not use one generic timeout or resource dictionary when the provider exposes materially different semantics. Give fields clear units, defaults, validation, and mutual-exclusion errors. Keep credentials out of logs and persisted plans.

3. Implement the Session Primitives

The provider’s session class implements EnvironmentSession:

Method	Required behavior
exec()	Run list-form commands without a shell or explicit string commands with shell=True; preserve return code, stdout, stderr and timeout
upload() / download()	Transfer one file to or from the active environment
write_text() / read_text()	Perform deterministic UTF-8 text I/O with actionable missing-path errors
upload_dir() / download_dir()	Transfer directory trees without silently changing the requested root
endpoint()	Return the externally reachable endpoint when the provider supports services, otherwise None
set_network_policy()	Apply a new enforceable policy only when dynamic switching is supported

Normalize provider-specific responses into ExecResult. Do not raise merely because the executed command returns a nonzero status; return the command result unless transport or provider execution itself failed. Preserve timeout versus provider-error meaning.

Use async provider APIs inside async methods. If the official SDK only exposes blocking calls, isolate them explicitly instead of blocking the event loop under high task concurrency.

4. Implement Open and Close as One Lifecycle

BaseEnvironment.open() creates one task environment from the recipe-adjusted ExecutionPlan; close() releases it. The environment provider must not load benchmark data or rewrite benchmark results.

During open():

1. Build and validate the provider config from the resolved plan.
2. Resolve mutually exclusive image or provider-native selectors.
3. Apply resources, workspace, labels, and setup network policy.
4. Create the sandbox and wait only up to the documented startup timeout.
5. Construct a session after the provider reports a usable state.
6. Clean up partially created resources before propagating any failure.

During close(), stop or delete the exact resource owned by the session. Make cleanup safe after partial startup and idempotent enough for cancellation or repeated error handling. Never discover cleanup targets through broad names or unvalidated global searches.

5. Declare and Enforce Network Capabilities

Set supported_network_modes, supported_allowlist_entry_types, and supports_dynamic_network_policy to the provider’s real enforcement capabilities.

Mode	Required guarantee
public	Provider-default outbound access
no-network	Outbound requests fail at the transport layer
allowlist	Only normalized supported hosts or networks are reachable

Fail closed when a policy or allowlist entry type cannot be enforced. Do not advertise a mode implemented only through a prompt, environment variable, or best-effort agent instruction.

Dynamic providers must support the runtime sequence from setup policy to agent-run policy, back to setup, and then to verifier policy when evaluation reuses the environment. Fresh verifier environments must start with the verifier policy.

Protect and redact proxy credentials, internal gateways, policy tokens, and generated URLs. Remove temporary networks, proxy containers, and provider policies after normal close and startup failure.

6. Preserve User-First Provider Precedence

Environment code consumes the final plan; recipes determine benchmark-specific defaults. Both layers must preserve:

explicit provider-native selector
  > explicit environment image
  > task metadata
  > recipe fallback

Resolve the winner before removing incompatible fields. Apply resource values field by field so explicit user values win and unspecified fields can inherit task hints. Preserve explicit workspaces, commands, timeouts, credentials, network policies, tags, and labels.

Adding a new provider often requires recipes for benchmarks with prebaked task images. Keep each recipe narrow to a benchmark/provider pair, copy the plan before rewriting it, and do not call the provider SDK from the recipe.

7. Respect Runtime Limits and Observability

BaseEnvironment applies the process-global provider-open limiter. Provider code must still respect its own SDK request limits, task concurrency, account quotas, and resource capacity without adding unbounded internal fan-out.

Log stable sandbox ids, lifecycle phases, elapsed time, selected non-secret image or snapshot, and actionable provider errors. Do not log tokens, signed URLs, internal proxy credentials, or full environment dictionaries that may contain secrets.

8. Register and Inspect the Component

Export the provider from src/agentcompass/environments/__init__.py, then verify registry discovery and its live config schema:

uv run agentcompass list env
uv run agentcompass config docs env <environment-id>