Validation and Alignment

Validate environment primitives, precedence, resources, network enforcement, cleanup, and concurrency.

Primitive Contract Tests

Against a real provider environment, verify:

• List-form and shell command execution, stdout, stderr, nonzero return code, timeout, cwd and environment variables.
• File and nested-directory upload/download, text read/write, missing paths and path quoting.
• Workspace creation and the documented default root.
• Endpoint discovery when supported.
• Session close after success, command error, timeout, cancellation, and partial startup.

Do not claim compatibility from a mocked SDK alone.

Source, Resource, and Override Matrix

Verify automatic and explicit paths:

Case	Expected behavior
Provider-native selector supplied	It wins over image and task metadata
Explicit image supplied	It wins when no native selector is active
Task image only	Compatible recipe maps it to the provider
Resource defaults plus explicit override	Explicit fields win; other task defaults remain
Explicit workspace	Recipe and provider defaults do not replace it
Invalid mutually exclusive fields	Fails before sandbox startup with an actionable message

Confirm the resolved plan and the real sandbox configuration. Test CPU and memory enforcement where supported, plus disk or GPU behavior when claimed in documentation.

Network Enforcement Matrix

For every advertised mode:

1. Prove public can reach an approved test destination.
2. Prove no-network denies a real outbound request at the transport layer.
3. For allowlist, prove an allowed destination succeeds and a denied destination fails.
4. Test every documented hostname, wildcard, IPv4, IPv6, or CIDR entry type.
5. Test setup → run → setup → verifier transitions when dynamic switching is advertised.
6. Confirm policy and proxy resources are removed after close and failed startup.

Do not use sensitive production endpoints as test destinations. Inspect logs and result metadata for credential leakage.

End-to-End and Concurrency Validation

Run one real benchmark/harness task with concurrency 1, then a small concurrent batch. Include a benchmark whose normal task image and workspace are supplied through a recipe when the provider claims that workflow.

The concurrent run should exercise provider-open rate limiting, unique resource names, independent sessions, cleanup, SDK async behavior, quota errors, and cancellation. Verify that one failed environment does not close or corrupt another task’s session.

For a new provider, compare task outcome and collected artifacts with an already supported provider using the same benchmark, harness, model, image, resources, and network policy where possible. Explain provider-specific differences.

Pull Request Evidence

Include:

• Official provider API or SDK version and links.
• Supported feature and network matrix.
• Redacted resolved config and smoke commands.
• Primitive, resource, network, cleanup, and concurrency outcomes.
• Automatic recipe and explicit-override evidence.
• Updated environment and affected benchmark documentation.

Repository Checks

uvx pre-commit run --all-files --show-diff-on-failure
cd docs
mint broken-links
mint validate