[cli-consistency] CLI Consistency Issues - 2026-09-28

Summary

A comprehensive inspection of gh aw CLI help output (/tmp/gh-aw/agent/all-help.txt, 483 per-command help files) against docs/src/content/docs/setup/cli.md found 3 confirmed issues: 2 high-severity help-text/behavior mismatches and 1 medium-severity documentation gap. No typos, capitalization issues, or broken examples were found elsewhere in the ~480 help texts reviewed.

Severity breakdown: 🔴 High: 2 · 🟡 Medium: 1 · ⚪ Low: 0

Category: Help text references non-existent flags/tools

2 findings — click to expand

1. `gh aw edit` help references a `--no-merge` flag that does not exist on that command (High)

File: pkg/cli/edit_command.go (Long description, line ~29)

Exact quoted CLI output (from gh aw edit --help, file edit.txt):

Workflows managed by a source: declaration can be edited
locally; by default, future updates will merge in those local changes (use --no-merge to override).

Actual flags on gh aw edit (from the same help output):

Flags:
      --add stringArray             Append a value to a list (path=value)
      --add-import stringArray      Append a workflow import path
      --add-skill stringArray       Append a workflow skill
  -d, --dir string                  Workflow directory (default: $GH_AW_WORKFLOWS_DIR or .github/workflows)
      --dry-run                     Validate changes without writing or compiling
  -h, --help                        Show help for gh aw edit
      --remove stringArray          Remove a value from a list (path=value)
      --remove-import stringArray   Remove a workflow import path
      --remove-skill stringArray    Remove a workflow skill
      --schedule string             Set a schedule using a fuzzy schedule or cron expression; use off to remove it
      --set stringArray             Set a frontmatter path (path=value)
      --unset stringArray           Remove a frontmatter path

There is no --no-merge flag defined anywhere in edit_command.go — a repo-wide search confirms --no-merge is only implemented for gh aw update (pkg/cli/update_command.go / equivalent), not edit.

Expected vs actual: The help text tells users to run gh aw edit <workflow> --no-merge to opt out of future merges, but that flag is silently rejected by cobra (unknown flag error) because it was never registered on the edit command. Users following the documented workaround will hit an error.

Suggested fix: Either (a) implement a --no-merge flag on gh aw edit that sets the same "do not merge future upstream changes" behavior as update's flag, or (b) if that capability doesn't exist for edit, remove/rewrite the sentence in edit_command.go's Long description to not reference a flag that isn't implemented on this command (e.g., point users to gh aw update --no-merge instead, if that's the actual mechanism, or describe the real opt-out path).

Priority: High — actively misleads users into running a command that will fail.

2. `gh aw mcp-server` help text omits the `audit-diff` tool from its tool list (High)

File: pkg/cli/mcp_server_command.go (Long description, lines 28–37)

Exact quoted CLI output (from gh aw mcp-server --help, file mcp-server.txt):

The server provides the following tools:
  - status      - Show status of agentic workflow files
  - compile     - Compile Markdown workflows to GitHub Actions YAML
  - logs        - Download and analyze workflow logs (requires write access or higher)
  - audit       - Investigate a workflow run, job, or step and generate a report (requires write access or higher)
  - checks      - Classify CI check state for a pull request
  - mcp-inspect - Inspect MCP servers in workflows and list available tools
  - add         - Add workflows from remote repositories to .github/workflows
  - update      - Update workflows from their source repositories
  - fix         - Apply automatic codemod-style fixes to workflow files

Only 9 tools are listed. However, the actual registered tool set includes a 10th tool, audit-diff, confirmed in multiple places in the codebase:

pkg/cli/mcp_server_tools_test.go:45: expectedTools := []string{"status", "compile", "logs", "audit", "audit-diff", "checks", "mcp-inspect", "add", "update", "fix"}
pkg/cli/mcp_server_unit_test.go:32: same list including audit-diff
pkg/cli/mcp_argument_validation_test.go:387: same list including audit-diff
pkg/cli/mcp_tools_privileged.go:721: registerAuditDiffTool registers a tool named "audit-diff"
pkg/cli/mcp_config_file.go:49: Tools: []string{"compile", "audit", "logs", "inspect", "status", "audit-diff"}

Notably, docs/src/content/docs/setup/cli.md (line ~951) correctly lists all 10 tools including audit-diff — so the documentation is actually more accurate than the CLI's own --help output here, meaning the bug is purely in the Go source's Long description string, not in the docs.

Expected vs actual: Running gh aw mcp-server --help under-reports the available tools; a user relying on --help output alone would not know audit-diff exists.

Suggested fix: Add a line for audit-diff to the tool list in pkg/cli/mcp_server_command.go's Long field, e.g.:

  - audit-diff  - Compare two workflow runs and generate a diff report (requires write access or higher)

(adjust wording to match actual behavior/permission requirements).

Priority: High — incomplete tool discovery via --help, and it's an easy one-line fix.

Category: Documentation gap (CLI flag missing from docs)

3. `gh aw env update --visibility` flag is undocumented in `cli.md` (Medium)

File: docs/src/content/docs/setup/cli.md, #### \env update [file]`` section (~line 897)

Exact quoted CLI output (from gh aw env update --help, file env-update.txt):

Scope and flag behavior:
...
- --visibility controls access when creating org or ent variables (all|private|selected). Existing variables keep their visibility.
...
Flags:
      --dry-run             Preview updates without applying any changes
      --enterprise string   Target enterprise slug (required for --scope ent)
  -h, --help                Show help for gh aw env update
      --org string          Target organization (required for --scope org unless inferable from --repo/current repo)
  -r, --repo string         Target repository (owner/repo format only; GHES host prefixes are not supported). Defaults to current repository
      --scope string        Variable scope (repo|org|ent)
      --visibility string   Visibility for newly created org or ent variables (all|private|selected) (default "all")
  -y, --yes                 Skip confirmation prompt

Exact quoted docs excerpt (cli.md, env update section):

**Options:** `--scope` (required), `--repo/-r`, `--org`, `--enterprise`, `--yes/-y`, `--dry-run`

--visibility is entirely absent from the docs' Options list and from the surrounding prose. A repo-wide search of docs/src/content/docs/setup/cli.md for the string "visibility" returns zero matches.

Expected vs actual: Users creating new org/enterprise-scoped compiler-default variables via gh aw env update --scope org --org my-org have no documented way to learn that --visibility exists to control whether the variable is created as all, private, or selected visibility — an important detail for org-scope variable creation that's only discoverable via --help.

Suggested fix: Add --visibility to the Options list for env update in cli.md, and add a short explanatory sentence (mirroring the CLI help): "--visibility controls access when creating new org or ent variables (all|private|selected, default all); existing variables keep their current visibility."

Priority: Medium — a real usability gap for org/enterprise variable management, not a correctness bug.

Inspection metadata

Commands inspected: All top-level and nested subcommands from the pre-collected help dataset (init, add, add-wizard, new, compile, validate, lint, edit, format, fix, trial, run, list, status, logs, audit, graders, outcomes, models, health, checks, forecast, experiments, enable, disable, remove, update, deploy, upgrade, env (+get/update), mcp (+list/list-tools/inspect/add), pr transfer, mcp-server, domains, version, completion, project (+new), hash-frontmatter, json-schema, secrets (+set/bootstrap), doctor) — 483 help-output files and the 24,922-line consolidated all-help.txt dump.
Method: Cross-referenced each command's actual --help output (verified against Go source in pkg/cli/*.go where flags/behavior were in question) with the corresponding section of docs/src/content/docs/setup/cli.md; searched for typos, terminology drift, inconsistent flag naming/negation patterns, mismatched short-flag reuse, and missing/extra documented options. Verified suspicious findings directly against pkg/cli/edit_command.go, pkg/cli/mcp_server_command.go, pkg/cli/mcp_server_tools_test.go, pkg/cli/mcp_argument_validation_test.go, and pkg/cli/mcp_tools_privileged.go.
Date: 2026-09-28
Run: github/gh-aw run 36428537516

Generated by ✅ CLI Consistency Checker · copilot · auto · 123.4 AIC · ⌖ 6.19 AIC · ⊞ 8.3K · ◷

expires on Sep 30, 2026, 5:38 AM UTC-08:00

🍪 Issue Monster selected this for Copilot

I identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 9.24 AIC · ⌖ 13.7 AIC · ⊞ 13.3K · ◷

🍪 Issue Monster selected this for Copilot

I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 7.01 AIC · ⌖ 12.4 AIC · ⊞ 13.3K · ◷

🍪 Issue Monster selected this for Copilot

I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 6.14 AIC · ⌖ 10.5 AIC · ⊞ 13.3K · ◷

🍪 Issue Monster selected this for Copilot

I have identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 9.51 AIC · ⌖ 10.2 AIC · ⊞ 13.3K · ◷

🍪 Issue Monster selected this for Copilot

I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 9.5 AIC · ⌖ 10.2 AIC · ⊞ 13.3K · ◷

🍪 Issue Monster selected this for Copilot

I have identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 8.96 AIC · ⌖ 10.4 AIC · ⊞ 13.4K · ◷

🍪 Issue Monster selected this for Copilot

I have identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 10.9 AIC · ⌖ 12.7 AIC · ⊞ 13.4K · ◷

🍪 Issue Monster selected this for Copilot

Ive identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 10.9 AIC · ⌖ 10.3 AIC · ⊞ 13.4K · ◷

🍪 Issue Monster selected this for Copilot

I’ve identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 9.34 AIC · ⌖ 10.3 AIC · ⊞ 13.5K · ◷

🍪 Issue Monster selected this for Copilot

I’ve identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 10.7 AIC · ⌖ 12.6 AIC · ⊞ 13.6K · ◷

🍪 Issue Monster selected this for Copilot

I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 8.69 AIC · ⌖ 12.6 AIC · ⊞ 14.1K · ◷

🍪 Issue Monster selected this for Copilot

I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 11 AIC · ⌖ 10.4 AIC · ⊞ 14.1K · ◷

🍪 Issue Monster selected this for Copilot

I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 7.83 AIC · ⌖ 10.5 AIC · ⊞ 13.9K · ◷

🍪 Issue Monster selected this for Copilot

I identified this as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

Why this cookie: the issue contains specific help-text and docs mismatches with low conflict risk and clear files to touch.

If assignment succeeds, Copilot will analyze the issue and open a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 24.7 AIC · ⌖ 13.9 AIC · ⊞ 13.9K · ◷

🍪 Issue Monster selected this for Copilot

Ive identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 9.5 AIC · ⌖ 10.4 AIC · ⊞ 13.9K · ◷

🍪 Issue Monster selected this for Copilot

I identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 6.83 AIC · ⌖ 14 AIC · ⊞ 13.9K · ◷

🍪 Issue Monster selected this for Copilot

I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 8.82 AIC · ⌖ 11.9 AIC · ⊞ 13.9K · ◷

🍪 Issue Monster selected this for Copilot

I have identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 9.82 AIC · ⌖ 12.7 AIC · ⊞ 13.9K · ◷

🍪 Issue Monster selected this for Copilot

I have identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 8.22 AIC · ⌖ 10.5 AIC · ⊞ 14.1K · ◷

🍪 Issue Monster selected this for Copilot

I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 11.7 AIC · ⌖ 12.8 AIC · ⊞ 14.1K · ◷

🍪 Issue Monster selected this for Copilot

I have identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 6.11 AIC · ⌖ 10.9 AIC · ⊞ 14.1K · ◷

This issue was automatically closed because it expired on 2026-09-30T13:38:38.095Z.

Closed by Workflow

🍪 Issue Monster selected this for Copilot

I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · pi · gpt54 · 9.57 AIC · ⌖ 12.6 AIC · ⊞ 14.1K · ◷

docs: document missing CLI flags and json-schema command in cli.md

Navigation Menu

Repository navigation

[cli-consistency] CLI Consistency Issues - 2026-09-28 #64018

Summary

Category: Help text references non-existent flags/tools

1. `gh aw edit` help references a `--no-merge` flag that does not exist on that command (High)

2. `gh aw mcp-server` help text omits the `audit-diff` tool from its tool list (High)

Category: Documentation gap (CLI flag missing from docs)

3. `gh aw env update --visibility` flag is undocumented in `cli.md` (Medium)

Inspection metadata

66 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[cli-consistency] CLI Consistency Issues - 2026-09-28 #64018

Description

Summary

Category: Help text references non-existent flags/tools

1. gh aw edit help references a --no-merge flag that does not exist on that command (High)

2. gh aw mcp-server help text omits the audit-diff tool from its tool list (High)

Category: Documentation gap (CLI flag missing from docs)

3. gh aw env update --visibility flag is undocumented in cli.md (Medium)

Inspection metadata

Activity

github-actions commented on Sep 28, 2026

github-actions commented on Sep 28, 2026

github-actions commented on Sep 28, 2026

github-actions commented on Sep 28, 2026

github-actions commented on Sep 28, 2026

github-actions commented on Sep 28, 2026

github-actions commented on Sep 28, 2026

github-actions commented on Sep 28, 2026

github-actions commented on Sep 28, 2026

github-actions commented on Sep 28, 2026

66 remaining items

github-actions commented on Sep 30, 2026

github-actions commented on Sep 30, 2026

github-actions commented on Sep 30, 2026

github-actions commented on Sep 30, 2026

github-actions commented on Sep 30, 2026

github-actions commented on Sep 30, 2026

github-actions commented on Sep 30, 2026

github-actions commented on Sep 30, 2026

github-actions commented on Sep 30, 2026

github-actions commented on Sep 30, 2026

github-actions commented on Sep 30, 2026

github-actions commented on Sep 30, 2026

github-actions commented on Sep 30, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

1. `gh aw edit` help references a `--no-merge` flag that does not exist on that command (High)

2. `gh aw mcp-server` help text omits the `audit-diff` tool from its tool list (High)

3. `gh aw env update --visibility` flag is undocumented in `cli.md` (Medium)