ScreenSpot

GUI grounding benchmark.

ScreenSpot evaluates GUI grounding by asking a VLM agent to identify target regions in screenshots.

Runtime Status

Field	Value
Benchmark id	screenspot
Tags	GUI Grounding, Vision
Execution type	local
Typical harness	qwen3vl_gui
Typical environment	host_process
Current status	registered in the direct runtime

When to Use

Use ScreenSpot when you need to measure GUI grounding behavior with the task assumptions described by this benchmark. For large or remote benchmarks, prefer benchmark recipes so images, workspaces, and provider-specific defaults come from task metadata instead of manual CLI flags.

Parameters

Common parameters for this benchmark include:

• category
• sample_ids
• agent_type
• max_concurrency

Shared Benchmark fields such as sample_ids follow Benchmark Parameters. Pass the ScreenSpot-specific category field in the same --benchmark-params object. Configure repeated attempts with --k and --attempt-strategy; see Metrics and Aggregation.

Run Example

agentcompass run \
  screenspot \
  qwen3vl_gui \
  "$MODEL_NAME" \
  --env <env-provider> \
  --benchmark-params '{"sample_ids":["<task-id>"]}' \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY" \
  --model-api-protocol openai-chat

Adjust the harness and environment to the supported combination for your branch and deployment.

Outputs

Per-task details are written to the run directory’s details/ subdirectory. The same run directory contains the aggregate views summary.md and metrics.json.

Notes

Set category to desktop, mobile, web, or all.