Installation

Prepare a supported host, install AgentCompass from source, and verify local or remote execution.

AgentCompass is currently installed from a source checkout. Start with the package installation below; the operating-system matrix and environment prerequisites later on this page explain which task execution paths are supported on your device.

Install AgentCompass

Use one isolated virtual environment. Do not mix uv, pip, and conda package installation in the same environment unless you fully understand their resolution behavior.

Clone the repository

Clone the repository first from GitHub:

git clone https://github.com/open-compass/AgentCompass.git
cd AgentCompass

Recommended: uv

Install uv using the official installation guide.

Linux, WSL, or macOS

curl -LsSf https://astral.sh/uv/install.sh | sh
uv --version

uv python install 3.12
uv venv --python 3.12
source .venv/bin/activate

uv pip install -e .

Windows PowerShell

Install uv with WinGet or scoop or the official standalone installer. Here we use WinGet as an example:

winget install --id=astral-sh.uv -e
uv --version

uv python install 3.12
uv venv --python 3.12
.venv\Scripts\Activate.ps1

uv pip install -e .

uv can install a managed Python 3.12 runtime when the host does not already provide one. See the official uv Python installation guide.

Alternative: pip and venv

Install Python 3.12 using the official Python downloads, then create a virtual environment:

Linux, WSL, or macOS

python3.12 -m venv .venv
source .venv/bin/activate

python -m pip install --upgrade pip
python -m pip install -e .

Windows PowerShell

py -3.12 -m venv .venv
.venv\Scripts\Activate.ps1

python -m pip install --upgrade pip
python -m pip install -e .

Alternative: Conda

After installing and initializing Conda:

conda create -n agentcompass python=3.12
conda activate agentcompass

python -m pip install --upgrade pip
python -m pip install -e .

The commands above install only AgentCompass and its required core dependencies. Optional benchmark and harness extras are checked only when the selected execution path needs them, and automatic installation is disabled by default. See Dependencies for the available extras, host-versus-sandbox rules, manual installation, and opt-in automatic installation.

Verify the installation

With the environment activated, confirm that Python 3.12 or newer is active and that the AgentCompass CLI opens successfully:

python --version
agentcompass

Supported Operating Systems

AgentCompass itself is installed on your computer, but the agent does not always execute tasks directly on that computer. A task can run in one of three places:

• Directly on your computer with host_process.
• Inside a Docker container on your computer with docker.
• Inside a cloud sandbox with daytona or modal.

host_process vs. docker

host_process is a direct execution adapter, not a sandbox. Commands run as normal subprocesses on your computer and file operations use the real host filesystem. The process inherits the host’s installed tools, permissions, network access, and operating-system state.

docker creates a container for the task. Commands and file operations run inside the container, and the task receives the dependencies and filesystem layout defined by its image.

Behavior	host_process	docker
Isolation	None; commands run directly on your computer	Container-level process and filesystem isolation
Files	Uses the configured host workspace	Uses the container filesystem and configured mounts
Dependencies	Must already exist on your computer	Come from the task’s container image
Reproducibility	Can vary with host state and installed packages	More consistent across machines using the same image
Startup	Fast; no image pull or container startup	May need to pull a large image and start a container
Recommended use	Trusted, lightweight, or service-backed benchmarks	Coding, terminal, and other command-heavy agent tasks

Do not use host_process for an untrusted or shell-capable agent. It can read, modify, or delete files available to your user account and can start processes directly on your computer. Prefer Docker or a remote sandbox for agentic coding and terminal benchmarks.

Docker substantially reduces task access to the host, but a container is not an absolute security boundary. Avoid unnecessary host mounts, protect Docker daemon access, and apply an appropriate network policy for untrusted evaluations.

Operating System Matrix

The supported options depend on your operating system:

Operating system	Install and use AgentCompass	Run tasks directly on this computer	Run tasks with local Docker	Run tasks in Remote Sandbox
Linux	Yes	Yes	Yes	Yes
WSL 2	Yes	Yes	Yes	Yes
Windows	Yes	No	No	Yes
macOS	Yes	Limited	No	Yes

Docker Desktop may be able to start Linux containers on native Windows or macOS, but AgentCompass does not currently treat those paths as supported local benchmark environments. Coding and terminal benchmark images, workspace paths, and evaluators assume a Linux environment. Use WSL 2, Daytona, or Modal for those workloads.

WSL 2: AgentCompass treats WSL 2 as a Linux host. Run AgentCompass, keep the repository, and invoke Docker from inside the WSL distribution.

Why native Windows cannot run tasks directly: The current host_process implementation sends shell commands to /bin/sh. Benchmark and harness workflows can also rely on POSIX paths, executable permissions, signals, and command-line tools that are not available in native Windows. WSL provides the required Linux behavior.

What “Limited” means on macOS: macOS provides /bin/sh and POSIX filesystem behavior, so lightweight or service-backed benchmarks can work with host_process. A benchmark may still depend on Linux-only packages, utilities, paths, or evaluation scripts. Use host_process only when the benchmark documentation explicitly supports it; use Daytona or Modal for coding, terminal, and other command-heavy benchmarks.

Prerequisites

All installation paths require:

Requirement	Purpose
Python >=3.12	Required by pyproject.toml and current runtime dependencies
Git	Clone the repository and receive updates
HTTPS access	Download source, Python packages, benchmark data, and task images
Model endpoint credentials	Call the model selected for an evaluation
One execution environment	Linux/WSL local execution, Daytona, or Modal

Disk, memory, and network requirements depend on the benchmark. Coding and terminal benchmarks can pull multi-gigabyte task images, so confirm available storage before starting a large run.

Common System Tools

On Ubuntu, Debian, and Ubuntu-based WSL distributions, install the common command-line tools before creating the Python environment:

sudo apt-get update
sudo apt-get install -y \
  build-essential \
  ca-certificates \
  curl \
  git \
  unzip \
  wget

Tool	Why it is needed
git	Source checkout and Git-backed benchmark assets
curl / wget	Installer, dataset, and artifact downloads
unzip	Archive-based benchmark preparation
build-essential	Fallback for optional Python packages without a compatible wheel
ca-certificates	TLS verification for package indexes and external services

Verify the tools:

git --version
curl --version
wget --version
unzip -v

For another Linux distribution, use its package manager and follow the official Git installation links.

macOS

On macOS, lightweight supported tasks can use host_process; Linux-specific tasks should run in Daytona or Modal. Homebrew is the recommended way to install and maintain common command-line tools:

brew install git wget
git --version
curl --version
wget --version
unzip -v

macOS already includes curl and unzip. If Homebrew or an optional Python package needs native compilation tools, install the Xcode Command Line Tools with xcode-select --install. Local Docker benchmark execution is not part of the supported macOS path.

Windows

On Windows without WSL, AgentCompass runs locally while benchmark tasks run in Daytona or Modal. Use either WinGet or Scoop to install common tools from PowerShell.

WinGet is included with current Windows 10 and Windows 11 installations:

winget install --id Git.Git --exact --source winget

Alternatively, install Scoop by following its official setup instructions, then use it to manage Git and optional Unix-style download tools:

scoop install git wget

Verify the available tools:

git --version
curl.exe --version

Windows includes curl.exe, and PowerShell provides archive extraction through Expand-Archive. Git for Windows is also available as a standalone installer. If a benchmark requires a complete POSIX environment such as /bin/sh, use WSL 2 instead of assembling that environment from individual Windows packages.

Linux Local Docker Setup

Use Docker Engine from Docker’s official repository for your distribution:

• Docker Engine installation
• Docker Engine on Ubuntu
• Linux post-installation steps

After installation, start the daemon if necessary and verify the complete client-to-daemon path:

sudo systemctl start docker
docker version
docker info
docker run --rm hello-world

If Docker only works with sudo, follow Docker’s post-installation guide to configure non-root access:

sudo groupadd docker
sudo usermod -aG docker "$USER"
newgrp docker
docker run --rm hello-world

Membership in the docker group grants root-level privileges on the host. Review Docker’s security warning before enabling non-root daemon access.

WSL 2 and Docker Setup

Install WSL from an elevated PowerShell session by following Microsoft’s WSL installation guide:

wsl --install -d Ubuntu

Restart Windows when requested, then confirm that the distribution uses WSL 2:

wsl --version
wsl --list --verbose

Choose one Docker topology:

1. Install Docker Engine inside the WSL Linux distribution by following the matching Linux instructions.
2. Install Docker Desktop and enable its WSL 2 integration by following the Docker Desktop WSL guide.

Do not maintain separate Docker Engine and Docker Desktop daemons inside the same WSL distribution. After setup, open the WSL shell and verify:

docker version
docker info
docker run --rm hello-world

Store the AgentCompass checkout in the WSL Linux filesystem, for example ~/code/AgentCompass, rather than under /mnt/c/. Docker recommends the Linux filesystem for better bind-mount performance and correct Linux file semantics.

Remote Sandbox Setup

Daytona and Modal run benchmark tasks in cloud sandboxes. They are the supported execution environments for Windows without WSL and the reliable path for Linux-specific workloads on macOS. They can also be used from Linux or WSL.

Provider	Prepare before installation	Official links
Daytona	Create an account and an API key with sandbox access	Documentation, API keys, Dashboard
Modal	Create an account and a user or service token	User setup, Service users, Signup

You need only one provider. Keep its credentials in environment variables or a secret manager; never commit them to the repository.

Linux, WSL, or macOS

# Daytona
export DAYTONA_API_KEY="..."

# Modal
export MODAL_TOKEN_ID="..."
export MODAL_TOKEN_SECRET="..."

Windows PowerShell

# Daytona
$env:DAYTONA_API_KEY = "..."

# Modal
$env:MODAL_TOKEN_ID = "..."
$env:MODAL_TOKEN_SECRET = "..."

Provider-specific endpoints and organization settings are optional. Continue with the Daytona guide or Modal guide after the base installation.

Optional Component Dependencies

The base installation contains the AgentCompass runtime and common integrations. Benchmark- or harness-specific packages are installed lazily on first use when automatic dependency installation is enabled.

Install known extras ahead of time when your computer will later run without package-index access:

uv pip install -e ".[swebench,mini-swe-agent]"

A no-network task sandbox cannot install a missing harness CLI or package at task time. Use a task image that already contains the required runtime, or prepare the dependency before enabling the sandbox network restriction.

See Dependency Management for available extras, lazy-install behavior, and offline preparation.

Update to the latest version

Commit or stash local changes before rebasing:

git pull --rebase

Reinstall the editable package when pyproject.toml or dependency metadata changes:

uv

uv pip install -e .

pip or Conda

python -m pip install -e .

Troubleshooting

Symptom	Check
Python version mismatch	Run python --version; recreate the environment with Python >=3.12.
agentcompass: command not found	Activate the environment, reinstall editable mode, or use uv run agentcompass.
uv, pip, or dataset downloads fail	Check DNS, proxy, CA certificates, and HTTPS access to the package index.
Cannot connect to the Docker daemon	Start Docker and run docker info from the same Linux or WSL shell.
Docker works in Windows but not WSL	Enable Docker Desktop integration for that WSL 2 distribution.
Repository operations are slow in WSL	Move the checkout from /mnt/c/ to the WSL Linux filesystem.
Daytona startup fails	Verify the API key and optional API endpoint or target.
Modal authentication fails	Run modal token info and verify the active workspace credentials.
Optional dependency installation fails in a restricted sandbox	Preinstall it in the task image or prepare it before network access is disabled.