pi engine silently ignores inline sub-agents and their models; agent fakes delegation on the main model #66715

Summary

With engine.id: pi, inline sub-agents (## agent: blocks) compile without a warning but are never available at runtime, and their model: settings are not used. In the run below, the agent couldn't find a delegation tool, so it started pi subprocesses through bash. Both subprocesses ran on the main model, aw-gateway/gpt-5.6-luna. The issue it created still credited each answer to the named sub-agents. Every request in token-usage.jsonl went to gpt-5.6-luna.

Run: https://github.com/githubnext/gh-aw-routing-sandbox/actions/runs/37702323259. Conclusion: success. Duration: 6.9 min. Turns: 18. Tokens: about 475k.

Environment

• Workflow: routing-test-pi-subagents.md, dispatched with task_id t11-subagent-models
• Compiler dev-303b402810, AWF v0.28.44, pi engine 1.0.0
• engine.model: copilot/gpt-5.6-luna, firewall enabled, tools.bash: true
• Two inline sub-agents:
  • file-summarizer, model: claude-haiku-4.5
  • quick-checker, model: small
• Task: "Use the file-summarizer sub-agent to summarize router/README.md, and use the quick-checker sub-agent to find which Python version router/pyproject.toml requires. Don't answer either part yourself… say which sub-agent produced each one."

What happened (observed)

1. Activation wrote the sub-agents to the wrong directory. The activation log shows [extractInlineSubAgents] Engine: "pi" → dir=".github/agents". The files were written to /tmp/gh-aw/.github/agents/file-summarizer.agent.md and quick-checker.agent.md. The compiled lock file uploads /tmp/gh-aw/.pi/agents instead, so the artifact didn't include them.
2. The agent job restored nothing. Its log shows [restore-sub-agents] source: /tmp/gh-aw/.pi/agents and then source directory not found — no inline sub-agents to restore.
3. No delegation tool was available. The agent made three tool_search calls looking for a sub-agent or delegate tool. None returned one. It then read the workflow .md source to find the agent definitions.
4. It started its own pi subprocesses through bash:
   • pi --offline --no-session --model claude-haiku-4.5 … failed with Model "claude-haiku-4.5" is ambiguous across providers: cloudflare-ai-gateway/claude-haiku-4.5, github-copilot/claude-haiku-4.5. No matching provider is authenticated.
   • pi --offline --no-session --model small … failed with No API key found for cloudflare-ai-gateway. A later pi --list-models small printed No models matching "small".
   • pi --list-models showed only one entry: aw-gateway gpt-5.6-luna.
   • The agent reran both tasks with env -u PI_OFFLINE pi --no-session --model aw-gateway/gpt-5.6-luna --tools read --system-prompt '<its own prompt>'. Both succeeded. These child processes didn't get the --extension flags that gh-aw passes to the main pi process (provider, steering and tool policy).
5. The created issue gave false credit. githubnext/gh-aw-routing-sandbox#69 says "The requested repository facts were delegated to the named sub-agents" and attributes each answer to file-summarizer and quick-checker. Neither declared model was used.
6. Token usage confirms it. All 22 requests in sandbox/firewall/logs/api-proxy-logs/token-usage.jsonl show model: gpt-5.6-luna (provider copilot, path /responses). No request used a Haiku-class model.
7. There was no warning at either stage. Compiling the same source with the dev build (gh aw compile --strict --no-emit) reported 1 succeeded, 0 warnings. gh aw audit reported workflow_succeeded and no finding about sub-agents.

Root causes

1. Pi has no delegation mechanism in gh-aw's runtime. PiEngine sets NativeAgentFile: false. The pi command in pkg/workflow/pi_engine.go loads only pi_provider.cjs, pi_steering_extension.cjs, pi_tool_policy.cjs, builtin:mcp, builtin:codemode and builtin:tool-search. Pi core has no sub-agent tool; delegation exists only as pi's optional examples/extensions/subagent extension. The compiler still accepts and stages inline sub-agents for pi as if they would work.
2. Go and JavaScript use different staging directories. GetEngineSubAgentDir("pi") in pkg/workflow/engine_config_dir.go returns .pi/agents, which the upload and restore steps use. getEngineSubAgentTarget in actions/setup/js/extract_inline_sub_agents.cjs has cases only for claude, codex and gemini, so pi falls back to .github/agents. Inline skills have the same mismatch: getEngineSkillTarget in extract_inline_skills.cjs sends pi to .github/skills, while pi_runtime.cjs copies from /tmp/gh-aw/.pi/skills. The skills part comes from reading the source and wasn't exercised by this run. Both mappings are unchanged on main at bc3991e.
3. Sub-agent models can't be resolved through the gateway. pi_models_json.cjs registers only the main model under the aw-gateway provider. gh-aw already expands model aliases such as small for detection and evals, but not for pi sub-agents. Bare IDs such as claude-haiku-4.5 are ambiguous in pi's built-in catalog, and none of those catalog providers is authenticated inside AWF.
4. Nothing reports the problem. parser.ExtractSubAgentModels already collects WorkflowData.SubAgentModels, but only warnRoutedSubAgentModels checks them, and only when model-routing is enabled. Nothing tells the author that pi will ignore the declared sub-agents. The audit also doesn't compare declared sub-agent models with the models the gateway actually served.

Proposed solution

The first step is needed for either option. Then choose Option A (support sub-agents on pi) or Option B (warn or fail at compile time). Option B is cheap enough to land first and narrow later.

Step 1 (prerequisite): use one staging directory

Have the activation-time extractor and the Go compiler read the engine's agent and skill directories from one source. For example, the compiler could pass the resolved directory and file extension to the interpolation step as environment variables, as it already does for the restore step, instead of keeping a separate JavaScript switch. Add a test that, for every registered engine, checks that the directory the extractor writes to is the directory the activation artifact uploads and the agent job restores. Cover both sub-agents and skills.

Option A: support declared sub-agents on pi

1. Load a delegation tool. When a pi workflow declares inline or imported sub-agents, have the compiler load pi's subagent extension. Take it from the installed, version-pinned pi package, or ship a vendored, reviewed copy in the setup action. Pin its version to the pi version.
2. Put the files where the extension looks by default. By default the extension reads user-scope agents from $PI_CODING_AGENT_DIR/agents. Project .pi/agents requires agentScope set to project or both, plus project trust, and gh-aw sets defaultProjectTrust: "never". pi_runtime.cjs should copy the restored sub-agents into /tmp/gh-aw/pi-agent-dir/agents, the same way it already copies skills into the agent directory.
3. Fill in required frontmatter. The extension skips any agent file that lacks string name and description fields. When the frontmatter has no name, set it from the ## agent: marker name. Make description a compile-time error for pi.
4. Resolve each sub-agent model at compile time. Expand aliases such as small through the existing model mappings. Resolve each model to a concrete ID the configured backend serves, and write it into the staged file as aw-gateway/<id>. Pass the full set of main and sub-agent models to pi_models_json.cjs so the gateway provider lists every one of them, each with the right API type for the AWF api-proxy route. If a model can't be resolved, fail compilation instead of falling back silently.
5. Apply the parent's controls to child processes. The extension starts child pi processes with their own arguments, so gh-aw's provider, steering and tool-policy extensions, which are now passed only as flags on the parent command line, wouldn't load in children. Configure them so every pi process in the job loads them, for example through the agent directory's settings. The bash allowlist, edit and bash disable flags, and the MCP exposure rules must then apply to sub-agents too. Child processes stay inside the AWF sandbox, so the network policy is unchanged.
6. Record sub-agent usage. Confirm that token-usage.jsonl, the OTel spans and the audit's SubagentModelRequests show each child's requested and actual model. Add an audit finding when a declared sub-agent model never appears among the models served.

Option B: warn or fail at compile time (until Option A ships)

1. When engine.id is pi and the workflow or its imports declare sub-agents, have the compiler emit a warning. Under --strict, make it an error. The message should say that pi has no delegation tool in gh-aw, that sub-agents and their model: values will be ignored, and that engine: copilot or claude supports them.
2. Also check SubAgentModels for every engine, not only when model routing is enabled. Warn when an engine can't honor a sub-agent model, or when the model or alias doesn't resolve for the configured backend.
3. Stop staging sub-agent files for pi while delegation is unsupported. Add one line to the system prompt saying the named sub-agents are unavailable. The agent can then call report_incomplete or answer directly, instead of improvising delegation and attributing answers to sub-agents that never ran.
4. Add the matching audit finding: declared sub-agents exist, but no sub-agent model was served.

Acceptance criteria

• For every engine, inline sub-agents and skills written during activation are restored in the agent job. A regression test covers pi.
• With Option A, rerunning this workflow shows requests for a Haiku-class model and a model resolved from small in token-usage.jsonl. Sub-agent child processes are subject to gh-aw's tool policy.
• With Option B, compiling this workflow produces a warning about pi sub-agents (an error with --strict), and the agent prompt no longer implies those sub-agents exist.
• gh aw audit reports when declared sub-agent models never appear among the models served.