EnvironmentSession contract. It owns
sandbox creation, command and file primitives, endpoints, resources, enforceable network policy, and cleanup while
remaining independent of benchmark scoring and harness behavior.
Integration Boundary
Provider SDK calls, container commands, filesystem transfer, lifecycle timeouts, resource translation, and network enforcement belong in the environment. Dataset parsing belongs in benchmarks; agent installation and execution belong in harnesses; benchmark/provider image and workspace adaptation belongs in recipes.Integration Workflow
- Implement the environment contract, including provider config, session primitives, lifecycle, resources, network enforcement, and precedence.
- Update public documentation with setup, parameters, supported capabilities, examples, and troubleshooting.
- Validate the integration against a real provider, including primitive, override, resource, network, cleanup, and concurrency behavior.
Completion Gate
- The provider implements every required session primitive with consistent semantics.
- Startup failure, cancellation, and normal close leave no owned sandbox resources behind.
- Advertised resources and network modes have enforcement-level evidence.
- Explicit selectors and resource values survive recipe and provider resolution.
- Real single-task and concurrent runs pass for every claimed workflow.
- Public docs match the live config schema and provider API.