Repository navigation
[cli-tools-test] MCP compile embeds multi-megabyte debug stderr in configuration errors #66455
Description
Problem Description
The MCP compile tool returns multi-megabyte config_error.message values when compilation exits without JSON output while debug logging is active. Three targeted compilation attempts produced 4.2–12.8 MB response payloads, forcing response externalization. The inline previews showed debug initialization messages instead of the actionable dependency error at the end.
Command/Tool
- Tool:
compilethrough the agenticworkflows MCP CLI - Severity: Medium — failure diagnostics are difficult and expensive for callers to consume.
- Frequency: Observed in all three selected compilation attempts in this session.
Steps to Reproduce
- Use an MCP server with compiler debug logging active and a required compilation dependency unavailable.
- Call
compilewith{"workflows":["windows"],"dry_run":true,"max_tokens":2000}. - Inspect the returned payload size and the generated validation error message. In this interface,
dry_runenables strict validation and all required checks; it is not a no-write compilation preview.
The same symptom occurred with agent-job-health and with archie plus strict:true. No checks were disabled and no dependency workaround was attempted.
Expected Behavior
Return a bounded, actionable diagnostic identifying the missing dependency. Keep verbose debug output separate from structured validation errors, while preserving the failed validation result and mandatory checks.
Actual Behavior
| Workflow | Response payload bytes | Error message characters | MCP elapsed time |
|---|---|---|---|
| windows | 12,757,794 | 12,274,368 | 56.8 s |
| archie | 11,054,878 | 10,641,456 | 48.9 s |
| agent-job-health | 4,213,641 | 4,052,249 | 39.7 s |
Timings are observational; the latter two requests overlapped with other tests. They are not isolated performance benchmarks. max_tokens is deprecated and intentionally ignored, so this finding concerns the unbounded error construction rather than a broken token-limit contract.
Logs/Diagnostics
Bounded diagnostics and source evidence
Each result had valid:false and errors[0].type:"config_error". Error messages began with debug lines such as:
workflow:domains Loading domain sets from embedded JSON
workflow:domains Loaded 42 ecosystem categories and 7 engine default domain sets
workflow:script_registry Creating new script registry
The actionable final diagnostic was shellcheck not available. The complete multi-megabyte messages are deliberately omitted.
The fallback in pkg/cli/mcp_tools_readonly.go, in registerCompileTool, takes the full captured stderr, trims whitespace, and passes it directly to buildCompileErrorResults when the subprocess fails with empty stdout. This is consistent with the observed debug output being embedded wholesale in config_error.message.
Environment and Scope
- Repository: github/gh-aw
- Date: 2026-10-07
- **MCP server version reported in response meta(redacted)
8c52586 - Testing run: §37576173978
- Compilation success and generated YAML correctness remain unverified because the required dependency was unavailable.
- Duplicate search was attempted through GitHub MCP, but all returned candidates were integrity-filtered; deduplication is inconclusive.
Suggested Next Action
Bound the fallback diagnostic, retain the actionable terminal error, and keep verbose debug material separate. Add a focused regression test with large debug stderr followed by an actionable error. Preserve strict validation failure behavior.
References: Testing run §37576173978
Generated by 🧪 Daily Cli Tools Tester · codex · gpt60 · 138.3 AIC · ⌖ 36.7 AIC · ⊞ 19.6K · ◷
- expires on Oct 13, 2026, 9:32 PM UTC-08:00
Activity
pelikhan commentedon Oct 10, 2026
Resolved by merged PR #67424. Compile failure diagnostics now preserve actionable causes and remediation while bounding debug stderr and fallback JSON; regression coverage includes debug interleaving, additional logger namespaces, and scanner-diagnostic isolation.