Source Map

Follow these call chains to move from a user-visible behavior to the function that owns it.

All paths below are relative to the repository root. Public extension code should still import shared types from agentcompass.runtime, not from the implementation paths shown here for navigation.

Entry and runtime call chains

The run command and the high-level Python helper converge before orchestration:

src/agentcompass/cli/main.py::run_command
  -> src/agentcompass/launcher.py::run_evaluation
  -> src/agentcompass/launcher.py::async_run_evaluation
  -> src/agentcompass/launcher.py::_build_run_request_from_config
  -> src/agentcompass/launcher.py::async_run_evaluation_request
  -> src/agentcompass/runtime/models/orchestration.py::Orchestration.from_requests
  -> src/agentcompass/runtime/orchestration.py::Orchestrator.execute

build_run_request is the public SDK constructor for callers that want to inspect or modify a request before execution. It is not an intermediate call made by run_command:

src/agentcompass/launcher.py::build_run_request
  -> RunRequest
  -> src/agentcompass/launcher.py::run_evaluation_request
  -> src/agentcompass/launcher.py::async_run_evaluation_request

async_run_evaluation_request merges any selected run config into the prepared request, wraps it as a one-request Orchestration, and uses the same Orchestrator as a multi-request launch.

Multi-request entry. The launch command and SDK reach the same Orchestrator through this call chain:

src/agentcompass/cli/main.py::launch_command
  -> src/agentcompass/runtime/orchestration.py::load_orchestration_spec
  -> src/agentcompass/launcher.py::async_launch
  -> src/agentcompass/runtime/orchestration.py::resolve_orchestration
  -> src/agentcompass/runtime/orchestration.py::Orchestrator.execute

The synchronous SDK function agentcompass.launch is implemented by launch() in src/agentcompass/launcher.py; it wraps async_launch with asyncio.run.

Inside the runtime. Once the Orchestrator owns a resolved request, the concrete per-task path is:

Orchestrator._preflight
  -> UnifiedEvaluationRuntime.preflight

Orchestrator._prepare_in_order
  -> UnifiedEvaluationRuntime.prepare
  -> BaseBenchmark.load_tasks
  -> BaseBenchmark.select_tasks
  -> RunStore.materialize_reused_details
  -> RunStore.load_partial_results

Orchestrator._worker
  -> UnifiedEvaluationRuntime.execute_task
  -> UnifiedEvaluationRuntime._run_attempts
  -> Planner.plan                         # once per k-attempt
  -> UnifiedEvaluationRuntime._run_single_attempt
  -> BaseEnvironment.open
  -> BaseBenchmark.prepare_task
  -> BaseHarness.start_session / run_task / close_session
  -> BaseBenchmark.collect_artifacts
  -> BaseBenchmark.evaluate
  -> analyze_task
  -> RunStore.save_partial_result

Orchestrator._finalize_one
  -> UnifiedEvaluationRuntime.finalize
  -> summarize_results
  -> BaseBenchmark.aggregate_metrics
  -> RunStore.save_results

For a HarnessFreeBenchmark, the runtime calls its run_task() method instead of the three Harness methods. The remaining plan, Environment, artifact, evaluation, analysis, and persistence stages stay shared.

Directory and symbol map

Area	Concrete symbols	Repository path
Public Python API	exported launch and request helpers	src/agentcompass/__init__.py, src/agentcompass/launcher.py
CLI	run_command, launch_command, inspection commands	src/agentcompass/cli/main.py
Configuration	load_run_config, deep_merge, runtime settings	src/agentcompass/runtime/config/
Orchestration models	OrchestrationSpec, Orchestration, RequestOutcome	orchestration.py under src/agentcompass/runtime/models/
Request models	RunRequest and its eight sections	src/agentcompass/runtime/models/request.py
Task and plan models	TaskSpec, PreparedTask, ExecutionPlan	src/agentcompass/runtime/models/task.py, src/agentcompass/runtime/models/plan.py
Component interfaces	BaseBenchmark, BaseHarness, BaseEnvironment, BaseRecipe, BaseAnalyzer	src/agentcompass/runtime/base.py
Discovery	component registries and built-in imports	src/agentcompass/runtime/registry.py
Planning	Planner.plan, run-local Recipe loading	src/agentcompass/runtime/planner.py, src/agentcompass/runtime/recipes.py
Scheduling	Orchestrator, TaskExecutor, provider limiters	src/agentcompass/runtime/orchestration.py, src/agentcompass/runtime/tasks.py, src/agentcompass/runtime/limits.py
Attempt execution	UnifiedEvaluationRuntime	src/agentcompass/runtime/runner.py
Result pipeline	detail shaping, aggregation, rendering, storage	src/agentcompass/runtime/results/, src/agentcompass/runtime/metrics/
Implementations	built-in components	src/agentcompass/benchmarks/, src/agentcompass/harnesses/, src/agentcompass/environments/, src/agentcompass/recipes/, src/agentcompass/analyzers/

A Model is stored directly in the request rather than selected through a component registry. See Runtime Contracts and Planning for the type, configuration limits, and change boundaries.

Find a change by symptom

Symptom or change	Start here	Then inspect
CLI and SDK resolve different values	src/agentcompass/launcher.py	request models and config loader
Wrong task set or unstable task ID	selected Benchmark’s load_tasks and select_tasks	BaseBenchmark.filter_tasks_by_sample_ids
Wrong image, workspace, or evaluation mode	Planner.plan and matching Recipe	Benchmark build_plan, Environment config
Agent output is malformed	selected Harness run_task	RunResult, trajectory models
Score is wrong	selected Benchmark evaluate	its aggregate_metrics and shared metrics protocol
A task runs twice or is starved	Orchestrator._worker and _select_state	PreparedRun.pending_tasks and provider limiters
A sandbox survives failure	_run_single_attempt and _run_fresh_evaluate	provider close implementation and cancellation path
Existing result cannot be reused	RunStore._get_reuse_source_directory	materialize_reused_details, detail naming
Summary denominator is wrong	summarize_results	Benchmark aggregate_metrics, MetricResult.counts

Continue with Execution, Scheduling, and Cleanup for the phase ordering behind these calls.