Python API
Embed evaluations in Python workflows.
CLI
Run evaluations, summaries, analysis, and component listing from a shell.
Supported Components
Browse registered benchmarks, harnesses, and model API protocols.
Configuration
Understand defaults, overrides, providers, and analysis params.
Results
Inspect details, summaries, logs, progress, and analysis artifacts.
Analyzers
Use post-execution badcase detection, statistics, and qualitative reports.
Benchmarks
Browse registered benchmark ids and execution shapes.