> ## Documentation Index
> Fetch the complete documentation index at: https://agent-compass.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Validation and Alignment

> Validate a harness across protocols, environments, lifecycle failures, concurrency, and official settings.

## Focused Validation

Verify import, registry discovery, config parsing, protocol validation, launch command generation, output parsing,
redaction, and cleanup behavior before launching a full task.

Test controlled failures for:

* Missing executable or optional dependency.
* Unsupported model protocol or environment capability.
* Invalid credentials or model endpoint response.
* Installation failure.
* Command timeout, step limit, malformed output, and cancellation.
* Cleanup after partial session startup.

## End-to-End Matrix

Start with one real benchmark task, task concurrency `1`, and retries disabled. Then cover:

| Dimension    | Minimum evidence                                                   |
| ------------ | ------------------------------------------------------------------ |
| Benchmark    | One intended benchmark using a normal `PreparedTask` and evaluator |
| Protocol     | Every newly claimed model API protocol                             |
| Environment  | Every provider claimed in the harness documentation                |
| Installation | Every claimed launch/install strategy                              |
| Outcome      | Successful run plus controlled failure and timeout paths           |
| Concurrency  | Small concurrent batch after the single-task path succeeds         |

The small concurrent batch should expose shared config files, fixed process names, session-id collisions, global mutable
state, log mixing, client reuse, and cleanup races.

When the harness supports setup/run network transitions, prove installation or startup under the setup policy and issue
a real denied request during restricted execution. Verify that `close_session()` still runs when policy switching or
agent execution fails.

## Official Alignment

If the harness claims parity with an official agent or leaderboard configuration, run the complete official benchmark
split using the pinned harness version and match its prompt, model protocol, inference settings, step/cost limits,
timeouts, resources, network policy, retry policy, and attempts per task.

Report:

* Supported benchmark, protocol, environment, and installation matrix.
* Exact redacted smoke and full-evaluation commands.
* A representative normalized trajectory and termination record.
* Task coverage, classified failures, score, and comparison with the official setting.
* Material prompt, provider, model deployment, or resource differences.

Follow the report structure in [Benchmark Validation and Alignment](/en/developer_guide/benchmark_integration/validation_and_alignment)
when publishing a score alignment claim.

## Repository Checks

```bash theme={"system"}
uvx pre-commit run --all-files --show-diff-on-failure
cd docs
mint broken-links
mint validate
```