Description

Adds SDK and CLI support for the public Workflow Evals API that shipped in roboflow/roboflow#16426 (https://api.roboflow.com/workspaces/{workspace}/workflow-evals). All 71 public routes are covered.

SDK

• roboflow/adapters/workflowevalsapi.py: shared transport. Handles bearer auth, Idempotency-Key / If-Match / Deletion-Key headers, and parses the Workflow Evals error envelope into a typed WorkflowEvalError (code, category, retryable, requestId, details). The platform OAuth error envelope is parsed too. Also includes SSE parsing for AI drafting and signed-URL PUTs.
• roboflow/core/workflow_evals.py: WorkflowEvals client with one method per endpoint, reached through rf.workspace().workflow_evals(). On top of that it adds a few helpers:
  • upload_case_asset: intent → signed PUT → complete
  • ai_draft / iter_ai_draft: SSE drafting
  • delete_confirmed / delete_run_confirmed: the two-step deletion-key protocol
  • wait_for_run / wait_for_export: poll until done
• When no idempotency key is passed, commands generate a UUID v4. Updates require the current revision.

CLI: roboflow workflow-eval …

• Top-level commands: list, get (--by-url), create, update, delete, capabilities, openapi, operation, compare
• Groups: spec, dataset, case, binding, preparation, run, execution, export, embedding, evaluator, schema, agent
• --body accepts inline JSON, @file.json, a file path, or - for stdin. Explicit flags are merged on top of it.
• Shortcut flags for the common cases: case import --source/--from-dataset id:split/--artifact, binding suggest --workflow [--workflow-version], preparation start --case, compare -x a -x b, run start --wait, export start --wait.
• delete shows the deletion impact and asks for confirmation (--yes skips it). Without a TTY, it refuses unless --yes is passed.
• Errors are structured JSON on stderr in --json mode. Exit codes: 2 for 401/403, 3 for 404, 1 otherwise. Revision, scope, feature-flag and validation errors come with hints.

Also updates the CLI-COMMANDS.md quickstart. The full reference in roboflow-product-docs still needs a follow-up.

Type of change

• New feature (non-breaking change which adds functionality)

How has this change been tested, please provide a testcase or example of how you tested the change?

• tests/test_workflow_evals.py: a table-driven test sends a request through every JSON route and checks the method, URL, bearer auth, the UUID v4 idempotency key on the routes that require it, and If-Match. The 71st route (SSE ai-draft) has its own tests. Also covers error envelope parsing, the deletion-key flow, SSE events, the three-step upload, and polling.
• tests/cli/test_workflow_eval_handler.py: --help for every subcommand, plus body loading from file/stdin, flag merging, exit codes, delete confirmation, and payload building for import, binding, preparation, run, export and compare.
• Full suite: 1161 tests pass. ruff and mypy are clean.
• Smoke-tested against production with read-only commands: capabilities, evaluator list, agent skills, spec list, schema get spec, and a 404 on get (exit code 3 with a structured error).

Any specific deployment considerations

None. The new endpoints still require the Feature.WorkflowEvals entitlement and the workflow-evals:* key scopes.

Docs

• Docs updated? What were the changes: CLI-COMMANDS.md quickstart section and command-group table.