> ## Documentation Index > Fetch the complete documentation index at: https://agent-compass.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Task Results Each JSON file under `details/` records the result of one Benchmark task; this page calls it a task detail file. It contains the final answer, score, trajectory, errors, and the task's evaluation attempts. If a runtime retry is triggered, AgentCompass also writes the discarded execution to `retry_details/` so you can identify why it was retried. Before reading these files, distinguish two concepts: * An `attempt` is an independent evaluation attempt controlled by `k` and included in the final task result. * A `retry` reruns recoverable work within the same evaluation attempt. It does not add another `attempt` or directly contribute to Benchmark metrics. See [Shared Benchmark Fields](/en/user_guide/modules/benchmarks/overview#shared-benchmark-fields) for how to configure and aggregate `k` and `avgk`. | File | When it is generated | What it stores | | -------------------------------------------- | ------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | | `details/[_].json` | The task produced a detail without an execution or evaluation error | The final answer, score, trajectory, and evaluation attempts. An incorrect answer or a skipped task can still use this filename. | | `details/_error_[_].json` | At least one recorded evaluation attempt has an execution or evaluation error | The same task information as a normal detail; the `_error_` prefix indicates that the file contains an error. | | `retry_details/*.json` | The runtime determines that the current error can be retried and retry budget remains | The discarded result and the error that triggered the retry. The filename also records the evaluation-attempt number, retry number, and failure stage. | The `category` segment appears only when the task has a category. `/` and `:` in task IDs, categories, and stage names are replaced with `_`. A normal run stores multiple evaluation attempts in the `attempts` object of one detail file. Only the replacements described above are applied; this is not complete path sanitization. Custom components should produce trusted, stable task IDs, categories, and stage names without backslashes, control characters, or directory segments. The combination of `task_id` and `category` must also remain unique after `/` and `:` are replaced, or different tasks can write to the same path. ## Task Detail Files Normal details and `_error_` details use the same JSON structure. Top-level fields describe the complete task, while `attempts` stores the result of each evaluation attempt. Field contents depend on the selected Benchmark, Harness, and analyzers, so some values can be `null` and optional fields may be absent. If a task fails before a result can be formed and saved, it may have no corresponding task detail file. However, the first summary at the end of the evaluation uses the results collected during that run and can still count the task as an error. See [Summary and Analysis Results](/en/user_guide/other_features/results/summary_analysis#summarymd) for the difference between the initial and regenerated summaries. ```json theme={"system"} { "task_id": "", "category": "", "correct": true, "solved_at": 1, "attempts_tried": 1, "k": 1, "retry_count": 2, "retry_counts": { "1": 2 }, "attempts": { "1": { "correct": true, "final_answer": "", "ground_truth": "", "trajectory": {}, "status": "completed", "score": 1.0, "error": "", "artifacts": {}, "extra": {}, "analysis_result": {}, "meta": { "resolved_execution_plan": {} } } } } ``` ### Task-Level Fields | Field | Meaning | | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `task_id` | The task identifier provided by the Benchmark. AgentCompass uses it to identify tasks during aggregation and reuse. | | `category` | An optional task category provided by the Benchmark. The normal runtime writes an empty string for an uncategorized task; compatible external or older results may omit it or use `null`. | | `correct` | Whether at least one recorded evaluation attempt passed scoring or verification. Omitted when a non-null `avgk_value` is used. | | `solved_at` | The first evaluation attempt that passed scoring or verification, numbered from `1`; `null` if none passed. Omitted when a non-null `avgk_value` is used. | | `attempts_tried` | The number of evaluation attempts actually recorded in `attempts`. When `avgk` is disabled, execution can stop after the first success, so this value can be lower than `k`. | | `k` | The maximum number of evaluation attempts allowed for this task. | | `max_score` | An optional task-level maximum score supplied by an adapter. It is absent when the producer does not provide one. | | `avgk_value` | Optional precomputed task-level `avg@k`, primarily for compatibility with externally generated results. When it is non-null, the top level no longer uses `correct` or `solved_at`, and aggregation reads this value first. Normal runs omit this field and calculate `avg@k` from `attempts`. | | `retry_count` | The total number of runtime retries actually triggered across all evaluation attempts. | | `retry_counts` | The number of retries triggered by each evaluation attempt. Keys are string-form attempt numbers. This map is sparse: an attempt with no retry has no key. | | `attempts` | A map of evaluation attempts keyed by string numbers such as `"1"` and `"2"`. Each value uses the attempt-level structure below. | A task detail has no top-level `status` or `score`; each evaluation attempt records its own status and score. The first summary generated at the end of an evaluation uses the results collected by that run. A later, separate `agentcompass summary` command reads the saved detail files and recalculates the summary. ### Attempt-Level Fields To inspect one evaluation attempt, first check `status` and `error` to determine whether execution was valid, then use `correct` and `score` to review the evaluation outcome. `trajectory`, `artifacts`, `extra`, and `meta` provide further execution and diagnostic context. | Field | Meaning | | ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `correct` | Whether this evaluation attempt passed the Benchmark's scoring or verification. | | `final_answer` | The final answer produced by the model or agent. It can be text, a patch, or structured JSON defined by the Benchmark. | | `ground_truth` | The reference answer provided by the Benchmark. It can be `null` for tasks that use a hidden verifier. | | `trajectory` | A record normalized by the Harness to the standard AgentCompass trajectory structure; `null` when no trajectory is available. See [Trajectory Fields](#trajectory-fields) for its structure. | | `status` | The execution status of this evaluation attempt. See [Status Values](#status-values). | | `score` | The Benchmark score for this evaluation attempt; it can be `null` when only a pass/fail result is available. | | `max_score` | An optional maximum score for this evaluation attempt. It is absent when not provided. | | `error` | An error produced during execution or evaluation. It is normally an empty string or `null`; on failure it can include a stack trace. | | `artifacts` | Additional artifact content or indexes collected by the Benchmark or Harness. Its shape is integration-specific. | | `extra` | Additional structured data written by the Benchmark or Harness. Its fields are not consistent across Benchmarks. | | `analysis_result` | Analyzer output generated during evaluation, keyed by analyzer family. See [Analysis Results](#analysis-results) for its structure. | | `meta` | Supplementary information written by the runtime or an integration. Besides `resolved_execution_plan`, component-specific fields can include `plan`, `extra`, `harness_metrics`, `status`, and `scoring`. | Do not rely on internal Harness `metrics` as a stable attempt-level field. A Benchmark or Harness that needs to retain integration-specific metrics normally writes them under `meta.harness_metrics`, `extra`, or `artifacts`. `meta.resolved_execution_plan` is only a compact summary; other component-defined fields under `meta` can contain more complete configuration or diagnostic information. ### Status Values | `status` | Meaning | | ------------------------- | ------------------------------------------------------------------------------------------------ | | `completed` | Execution and evaluation produced a valid result. This does not mean that the answer is correct. | | `run_error` | The task execution phase failed. | | `eval_error` | The scoring or verification phase failed. | | `run_error_or_eval_error` | Both execution and evaluation failed, or the failure cannot be assigned to only one of them. | | `skipped` | This evaluation attempt was skipped. | ### Trajectory Fields `ACTF_v1.0` is a trajectory schema version defined by AgentCompass. It gives different Harness implementations a common representation for agent execution records; it is not a protocol defined by a model provider or third-party agent framework. `trajectory` uses this structure to record model input and output, tool calls, Environment observations, timing, and token metrics in execution order. Which fields contain values depends on the Harness; when a Harness does not produce a trajectory, `trajectory` is `null`. | Field | Meaning | | ---------------- | ------------------------------------------------------------------------------- | | `schema_version` | The AgentCompass trajectory schema version. The current default is `ACTF_v1.0`. | | `steps` | An array of interaction steps in execution order. | | `started_at` | The start time of the complete trajectory. | | `finished_at` | The finish time of the complete trajectory. | Each element of `steps[]` contains: | Field | Meaning | | ------------------------------------- | ---------------------------------------------------------------------------------- | | `step_id` | The step number within the trajectory. | | `system_prompt` | The system prompt used for this step. | | `user_content` | User content or subsequent input sent to the model. | | `tools` | Tool information recorded for this step; the exact contents depend on the Harness. | | `assistant_content.content` | The assistant's visible content for this step. | | `assistant_content.reasoning_content` | Optional reasoning content supplied by the Harness. | | `assistant_content.tool_calls` | Tool calls requested by the assistant during this step. | | `observation` | Observations returned by tool or Environment actions. | | `metric.prompt_tokens_len` | The number of input tokens for this step; `null` when unavailable. | | `metric.completion_tokens_len` | The number of output tokens for this step; `null` when unavailable. | | `metric.llm_infer_ms` | Model inference time in milliseconds. | | `metric.env_action_ms` | Environment action time in milliseconds. | | `metric.stop_reason` | The reason the model response stopped. | | `started_at` | The start time of this step. | | `finished_at` | The finish time of this step. | ### Resolved Execution Plan `attempts..meta.resolved_execution_plan` records the Environments, network policies, and [Recipes](/en/user_guide/other_features/recipes) resolved for this evaluation attempt. This summary is created before the Environment is opened. It therefore shows that the plan was resolved, but does not prove that the Environment was created successfully or include its complete configuration. | Field | Meaning | | ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `environment` | The Environment planned for task execution. It contains its `id` and the `network_policy` used for Environment startup, Benchmark preparation, and Harness setup. | | `evaluation_environment` | The separate Environment planned for scoring. It contains its `id` and startup `network_policy`, and can be `null` when none is configured. | | `run_network_policy` | The network policy used while the Harness or Benchmark performs model and tool operations. | | `verifier_network_policy` | The network policy used during Benchmark scoring or verification. | | `applied_recipes` | The Recipe IDs actually applied to this task. | Each `network_policy` object above contains `network_mode` and `allowed_hosts`. `network_mode` identifies the network mode, while `allowed_hosts` lists the hosts that can be accessed. See [Network Policies](/en/user_guide/modules/environments/configuration/network) for the meaning of each setting. ### Analysis Results When [`agentcompass analysis`](/en/user_guide/using_agentcompass/cli/analysis#run-with-evaluation) runs with an evaluation, `analysis_result` stores each evaluation attempt's output by analyzer family. A successful analysis can contain the fields below; a failed analysis may contain only a subset: | Field | Meaning | | ------------ | ----------------------------------------------------------------------------------------------------------------- | | `is_badcase` | Whether the analyzer classified this result as a bad case. Statistics-only analyzers can return `null`. | | `details` | A structured explanation object produced by the analyzer; normally an empty object when no details are available. | | `score` | An optional score produced by the analyzer. | | `error` | An error from the analyzer itself. It is normally absent when no error occurred. | | `extra` | Optional additional data produced by the analyzer. | If a selected analyzer's `analysis()` call raises an exception, AgentCompass normally writes `is_badcase: false` and the exception under that family's `error`, but omits `details`. The error does not change the Benchmark's existing `status`, `correct`, or `score`. If the failure occurs while creating or matching the analyzer, or while checking its requirements, that family may not appear in `analysis_result`; consult the logs to identify the cause. ## Error Detail Files The `_error_` prefix marks a task detail that contains an execution or evaluation error. It is used when any recorded evaluation attempt meets either condition: * `status` is `run_error`, `eval_error`, or `run_error_or_eval_error`; * `error` is non-empty. To support result structures produced by different integrations, `meta.status: "error"` also causes this prefix to be used. `_error_` does not mean that the answer was merely incorrect. It means the detail contains an execution or evaluation error and therefore cannot be reused. If multiple evaluation attempts include both `completed` and error states, one attempt that meets a condition above is enough to give the complete task detail this prefix. A task with `status: "completed"` and `correct: false` uses a normal detail filename. With [`--reuse`](/en/user_guide/using_agentcompass/run_controls#resume-an-interrupted-run), AgentCompass reuses only normal details. Tasks that have only an `_error_` detail are run again in the new run, and the source run is not modified. If a normal detail for that task is later written in the target directory, its stale error counterpart is removed. ## Retry Detail Files The runtime reruns work and writes a retry detail only when an error matches the retry rules and retry budget remains. Therefore, the absence of a retry detail does not mean that the task did not fail. A final failure that is not retried normally remains in an `_error_` task detail; if no result could be formed and saved when the failure occurred, there may be no detail file. See [Retry Only Transient Failures](/en/user_guide/using_agentcompass/run_controls#retry-only-transient-failures) for rules and budgets. ```json theme={"system"} { "schema_version": "agentcompass.retry.v1", "task_id": "", "category": "", "attempt": 1, "retry": 1, "max_retries": 2, "stage": "evaluate", "scope": "evaluate", "matched_pattern": "", "error": "", "discarded_result": {} } ``` | Field | Meaning | | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `schema_version` | The retry-detail schema version. The current value is `agentcompass.retry.v1`. | | `task_id` | The Benchmark task ID whose work was retried. | | `category` | The optional task category. | | `attempt` | The evaluation attempt to which this retry belongs, numbered from `1`. | | `retry` | The retry number within the current evaluation attempt, numbered from `1`. It resets for the next evaluation attempt. | | `max_retries` | The maximum number of runtime retries available to each evaluation attempt. | | `stage` | The lifecycle stage in which the retry was triggered. Common values are listed below. | | `scope` | The amount of work restarted by the retry: `attempt` or `evaluate`. | | `matched_pattern` | The first regular expression that matched the error text. If no retry patterns were configured, any non-empty error matches and this field is ``. | | `error` | The error text that triggered the retry. For exceptions, it normally includes a stack trace. | | `discarded_result` | The discarded result snapshot, including `meta.resolved_execution_plan`. If no result existed yet, the runtime constructs an error result. | `discarded_result` is for diagnosis only and preserves as much of the discarded result as possible, so it can have more fields than an evaluation attempt in `details/*.json`. It normally contains the `status`, `correct`, `score`, `final_answer`, `ground_truth`, `trajectory`, `error`, `artifacts`, `extra`, and `meta` fields described above. It can also contain: | Field | Meaning | | ------------ | ---------------------------------------------------------------------------------------------------------------------------------- | | `task_id` | The task ID associated with the discarded result. | | `category` | The optional task category associated with the discarded result. | | `metrics` | Raw metrics returned by the Harness, for diagnosis only. This is not a stable field in normal task details. | | Other fields | A Benchmark or Harness that returns a dictionary result can retain its own additional fields, whose shape is integration-specific. | Use `scope` to determine which work the retry repeats: | `scope` | Behavior | | ---------- | ------------------------------------------------------------------------------- | | `attempt` | Restart the complete current evaluation attempt. | | `evaluate` | Rerun only scoring or verification without creating another evaluation attempt. | Use `stage` to identify the earliest phase that failed: | `stage` | Phase | | ---------------------- | ----------------------------------------------------------------- | | `plan` | The retry occurred before a more specific task stage was entered. | | `open_environment` | Create the task execution Environment. | | `prepare_task` | Prepare Benchmark input and the workspace. | | `run_task` | Run a Benchmark task that does not use a Harness. | | `start_harness` | Start the Harness session. | | `run_harness` | Execute the task through the Harness. | | `collect_artifacts` | Collect task artifacts. | | `evaluate_environment` | Create a separate scoring Environment. | | `evaluate` | Perform scoring or verification. | | `attempt` | Fallback when no more specific stage is available. | ## Handle Sensitive Content Before writing task and retry details, AgentCompass recursively redacts credential fields that it recognizes. Answers, prompts, observations, stack traces, and integration-specific data can still contain task content or other sensitive text. Protect these files like logs, and review their contents before publishing a run directory. `details/*.json` feeds aggregation, and normal detail files can also be reused. `retry_details/*.json` is for diagnostics only. To correct evaluation configuration or results, rerun the task instead of editing these files directly. ## Related Pages * [Results Overview](/en/user_guide/other_features/results) * [Run Records and Diagnostics](/en/user_guide/other_features/results/run_records) * [Summary and Analysis Results](/en/user_guide/other_features/results/summary_analysis) * [Run Controls](/en/user_guide/using_agentcompass/run_controls) * [`agentcompass analysis`](/en/user_guide/using_agentcompass/cli/analysis) * [Network Policies](/en/user_guide/modules/environments/configuration/network)