NaiveSearchAgent

The naive_search_agent harness runs the AgentCompass built-in deep-search agent, having the model under test complete research benchmarks such as GAIA, DeepSearchQA, and FrontierScience one task at a time.

You only need to provide the model’s access credentials; the harness automatically sets up the search agent in the run environment, drives the model through multi-turn retrieval with tools such as search / visit to produce an answer, and finally collects the model’s final answer and full search trajectory. Credentials are passed via the CLI --model-base-url / --model-api-key, and both the openai-chat and openai-responses --model-api-protocol protocols are supported.

How it works

• Tools and loop. tools selects the enabled tools (search / browse / visit); the engine interacts with the model over multiple turns using the function-calling protocol. max_iterations caps per-task iterations, max_tool_calls_per_turn caps tool calls in a single assistant message, and max_tool_response_length truncates an over-long single tool response (keeping the head and tail). When the model stops issuing tool calls, the answer is considered complete and the content of the last assistant message is taken as the final answer.
• External services. search depends on Serper, and browse / visit depend on Jina Reader; keys are supplied via serper_api_key / jina_api_key (defaulting to the same-named environment variables). tool_model_name can set a dedicated web-summary model for visit, falling back to the model under test when left empty.

Built-in tools

The agent can call the following three tools during the search loop. Use the tools parameter to choose which to enable (default ["search", "visit"]); they can be combined as needed.

Tool	Inputs	Purpose	Dependencies
search	query (search terms)	Runs a single Google search and returns a result list (titles, snippets, links, etc.). Used to discover pages relevant to the question — the entry point of retrieval.	Serper
visit	url (a single link or an array of links), goal (what this visit aims to obtain)	Fetches one or more pages and returns a summary of the content focused on goal (rather than the full text). The summary is generated by the model set in tool_model_name, falling back to the model under test. Suited for targeted extraction from long pages.	Jina Reader + summary model
browse	url (a single link)	Fetches the full content of a single page (title, summary, body) and returns it verbatim, without LLM summarization. Suited for cases that need to preserve the page’s original detail.	Jina Reader

The default combination search + visit matches the typical deep-search flow: use search to find candidate pages, then use visit with an explicit goal to read closely and extract information. When you need the page’s original text rather than a summary (for example, comparing tables, code, or clauses verbatim), switch to or add browse. The difference between visit and browse is that the former returns a goal-oriented summary while the latter returns the full text.

Parameters

Pass a JSON object via --harness-params '{...}', or a harness.params block in the YAML given to --config; the CLI wins on shared keys (deep-merge).

Parameter reference

Parameter	Type	Default	Choices / values	Description
install_root	string	/opt/agentcompass/naive_search_agent	—	Engine package unpack directory inside the sandbox (unused in host_process local mode).
tools	list	[“search”, “visit”]	search / browse / visit	Enabled tool list.
max_iterations	int	50	≥ 1	Maximum agent iterations per task.
max_retry	int	10	≥ 1	Application-level retry limit for one LLM or tool call.
retry_interval	int	5	≥ 1	Seconds to wait between retries.
max_tool_calls_per_turn	int	5	≥ 1	Maximum tool calls allowed in one assistant message.
max_tool_response_length	int	8192	≥ 1	Maximum printable units retained from a tool response (truncated beyond this, keeping head and tail).
request_timeout	int	2000	≥ 1	Read timeout for one LLM HTTP request in seconds.
tool_model_name	string	""	—	Dedicated web-summary model for the visit tool; falls back to the model under test when empty.
serper_api_key	string	—	Serper search API key.
jina_api_key	string	—	Jina Reader API key.
env	dict	{}	—	Additional environment variables injected into the engine process.
timeout	int / null	9000	≥ 1	Task wall-clock timeout in seconds; aborts on timeout. null = no limit.

Search and parsing API keys

serper_api_key / jina_api_key default to environment-variable references (${SERPER_API_KEY} / ${JINA_API_KEY}): set the same-named variables in your shell and they are injected automatically, or pass the keys inline in --harness-params. When only search is enabled you can omit the Jina key; when only visit / browse are enabled you can omit the Serper key — just provide the key matching the tools actually enabled.

Run examples

naive_search_agent is passed as the second positional argument to agentcompass run <benchmark> naive_search_agent <model>; harness configuration is passed via --harness-params. GAIA, DeepSearchQA, and similar benchmarks are all judge-scored and require a judge model judge_model via --benchmark-params, otherwise tasks cannot be scored (see the respective benchmark docs).

Default

Pass the Serper / Jina keys directly via --harness-params, defaults for the rest.

agentcompass run \
  gaia \
  naive_search_agent \
  "$MODEL_NAME" \
  --env host_process \
  --benchmark-params '{
    "judge_model": {"id": "Qwen3.6-35B-A3B", "base_url": "https://your-judge-endpoint/v1", "api_key": "sk-…"}
  }' \
  --harness-params '{
    "serper_api_key": "your-serper-key",
    "jina_api_key": "your-jina-key"
  }' \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY" \
  --model-api-protocol openai-chat

Custom params

Narrow the toolset and iterations, pass keys inline, and set a task wall-clock timeout.

agentcompass run \
  gaia \
  naive_search_agent \
  "$MODEL_NAME" \
  --env host_process \
  --benchmark-params '{
    "judge_model": {"id": "Qwen3.6-35B-A3B", "base_url": "https://your-judge-endpoint/v1", "api_key": "sk-…"}
  }' \
  --harness-params '{
    "tools": ["search", "visit"],
    "max_iterations": 40,
    "timeout": 9600,
    "serper_api_key": "your-serper-key",
    "jina_api_key": "your-jina-key"
  }' \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY" \
  --model-api-protocol openai-chat \
  --task-concurrency 16

Output

The harness returns a RunResult per task: the final answer (final_answer), the trajectory, the execution status, and diagnostic metrics (iteration count, engine exit code, config / prompt / result paths, etc.). When the engine exits abnormally, errors out, or does not finish cleanly, the status is recorded as RUN_ERROR with the error message attached. Per-task details and aggregate metrics are written by the benchmark under results/<benchmark>/<model>/<run>/ (see Results).