> ## Documentation Index
> Fetch the complete documentation index at: https://agent-compass.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Modal

> Run benchmark tasks in Modal cloud sandboxes.

Modal is a remote environment provider for task images that can run in cloud sandboxes. It is a good fit for SWE-bench and Terminal-Bench style workloads where the task metadata already identifies a compatible registry image.

## Official Setup Links

| Need                          | Link                                                                              |
| ----------------------------- | --------------------------------------------------------------------------------- |
| Create a Modal account        | [Modal signup](https://modal.com/signup)                                          |
| User account setup docs       | [Modal user account setup](https://modal.com/docs/guide/modal-user-account-setup) |
| Create automation tokens      | [Modal service users](https://modal.com/docs/guide/service-users)                 |
| Token configuration reference | [Modal token config](https://modal.com/docs/sdk/py/latest/modal.config)           |
| Token settings page           | [Workspace token settings](https://modal.com/settings/tokens/service-users)       |

## First-Time Setup

1. Create a Modal account from the official signup page.
2. Configure Modal credentials for the machine that runs AgentCompass.
3. Export `MODAL_TOKEN_ID` and `MODAL_TOKEN_SECRET`, or let the Modal CLI write credentials to `~/.modal.toml`.
4. Run a single AgentCompass task before increasing concurrency.

For local development, the Modal CLI setup flow is usually enough:

```bash theme={"system"}
modal setup
```

For automated runners or shared workspaces, create a service user token in Modal and export it in the shell or CI secret store:

```bash theme={"system"}
export MODAL_TOKEN_ID="..."
export MODAL_TOKEN_SECRET="..."
```

AgentCompass authentication precedence is:

| Source                                           | Use when                                                                   |
| ------------------------------------------------ | -------------------------------------------------------------------------- |
| `environments.modal.token_id` and `token_secret` | You intentionally keep credentials in a private `--config` file.           |
| `MODAL_TOKEN_ID` and `MODAL_TOKEN_SECRET`        | Recommended for shell sessions, CI, and shared scripts.                    |
| `~/.modal.toml`                                  | Convenient for local development after `modal setup` or `modal token set`. |

<Note>
  Do not commit Modal tokens to `config/defaults.yaml`. Use environment variables or a private config file passed with `--config`.
</Note>

## Verify Credentials

Use the Modal CLI to verify the active credentials:

```bash theme={"system"}
modal token info
```

Then run one recipe-backed task with AgentCompass:

## SWE-bench Verified

```bash theme={"system"}
agentcompass run \
  swebench_verified \
  mini_swe_agent \
  "$MODEL_NAME" \
  --env modal \
  --benchmark-params '{"sample_ids":["astropy__astropy-12907"]}' \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY"
```

The SWE-bench Modal recipe selects the task image from SWE-bench metadata and sets the workspace root to `/testbed`. You normally do not need to pass `image` or `memory`.

## Full Benchmark With Concurrency

```bash theme={"system"}
agentcompass run \
  swebench_verified \
  mini_swe_agent \
  "$MODEL_NAME" \
  --env modal \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY" \
  --task-concurrency 32
```

Use lower concurrency when your model endpoint, Modal account limits, or benchmark image startup time becomes the bottleneck.

## Provider Params

Common Modal overrides live under `environments.modal` or `--env-params`:

| Field                       | Default                                 | How to use it                                                                                                              |
| --------------------------- | --------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| `token_id` / `token_secret` | `MODAL_TOKEN_ID` / `MODAL_TOKEN_SECRET` | Prefer environment variables; both values must be provided together.                                                       |
| `app_name`                  | `agentcompass-sandboxes`                | Group task sandboxes under a dedicated Modal App when separating teams or evaluations.                                     |
| `environment_name`          | Unset                                   | Select a Modal Environment only when the workspace uses named environments.                                                |
| `name`                      | Generated                               | Set a stable sandbox name only for focused debugging; concurrent tasks require unique names.                               |
| `image`                     | Unset                                   | Let a benchmark recipe select a registry image. Use it for an intentional custom-image run.                                |
| `named_image`               | Unset                                   | Use a published Modal named image instead of `image`; the two fields are mutually exclusive.                               |
| `add_python`                | Unset                                   | Request a Python series only when adapting a compatible registry image that does not already contain the required runtime. |
| `timeout`                   | `43200`                                 | Maximum sandbox lifetime. Keep it above the longest legitimate task, including setup and verification.                     |
| `idle_timeout`              | Unset                                   | Set only when automatic idle termination is safe for the harness's activity pattern.                                       |
| `workdir`                   | Image default                           | Must be absolute. Prefer a recipe-provided benchmark workspace.                                                            |
| `cpu`                       | Provider default                        | Accepts a positive value or a two-value range; start from benchmark metadata and endpoint concurrency.                     |
| `memory`                    | Provider default                        | Accepts MiB values, size strings such as `6g`, or a two-value range. Set it above observed peak task use.                  |
| `gpu`                       | Unset                                   | Request a Modal GPU type only for a benchmark or harness that uses it.                                                     |
| `cloud` / `region`          | Provider placement                      | Restrict placement only for data locality, availability, or measured latency requirements.                                 |
| `default_workspace_root`    | `/workspace/`                           | Fallback root when the benchmark does not provide one.                                                                     |
| `env_variables`             | `{}`                                    | Inject sandbox variables. Keep credentials in environment references or a private config.                                  |
| `tags`                      | `{}`                                    | Add ownership, cost, or cleanup metadata visible to provider operations.                                                   |
| `resources`                 | `{}`                                    | Alternative mapping for `cpu`, `memory`, and `gpu`; direct top-level fields take precedence.                               |
| `sandbox_start_timeout`     | `300`                                   | Increase when a valid image consistently takes longer to create.                                                           |
| `operation_timeout`         | `1800`                                  | Increase for legitimate long provider operations; harness command limits remain separate.                                  |

Modal-native `block_network`, outbound allowlists, and inbound CIDR fields are also available at sandbox creation.
Prefer [provider-neutral phase policies](/en/user_guide/modules/environments/network) for evaluation isolation and dynamic transitions.

## Image Overrides

Use `image` for a registry image or `named_image` for a Modal named image only when intentionally overriding recipe behavior.

```bash theme={"system"}
export MODEL_NAME=""

agentcompass run <benchmark> <harness> "$MODEL_NAME" \
  --env modal \
  --env-params '{"image":"python:3.13-slim"}'
```

Use `agentcompass config docs env modal` to inspect the live field types and defaults in the installed revision.

## Related Pages

* [Environments Overview](/en/user_guide/modules/environments/overview)
* [Network Policy](/en/user_guide/modules/environments/network)
* [Sandbox Resource Limits](/en/user_guide/modules/environments/resource_limits)
* [agentcompass run](/en/user_guide/cli/run#control-task-execution)
* [Configuration Reference](/en/user_guide/overview#configuration-files-and-precedence)