Persona Overview
- Agent: agentic-workflows (custom workflow-authoring agent)
- Personas This Run: Information Worker, Product Manager, QA Tester
- Scenarios Tested: 4 (IW-1, PDM-1, PDM-2, QA-1) of 6 generated
- Average Quality Score: 4.7/5.0
- Run: §38064059069
Key Findings
- All 4 scenarios scored ≥4.6 average — no weak or failed responses this run.
- The agent consistently separates concerns correctly:
schedule+workflow_dispatch for recurring digests (IW-1, PDM-1) vs. pull_request events for PR-reactive flows (PDM-2, QA-1).
- Read-only permissions plus safe-outputs-only writes (
create-issue, add-comment) were applied in every scenario — no scenario suggested a write-scoped permission.
- "Replace previous result" patterns (
close-older-issues for digests, hide-older-comments for PR comments) were suggested proactively in 3 of 4 scenarios, matching repo convention.
- The one recurring soft spot:
prompt_clarity scored 4/5 (never 5) across all 4 scenarios — the agent's recommendation didn't surface a clarifying question for ambiguous scope terms (e.g. "completed" = closed issues vs. merged PRs, coverage threshold source, closing-keyword detection mechanics).
Top Patterns
- Triggers:
schedule + workflow_dispatch for weekly digests; pull_request (closed+merged, or opened+synchronize) for PR-reactive automation.
- Tools:
github tool in gh-proxy mode with toolsets: [default], extended with actions toolset only when CI artifact access was needed (QA-1).
- Security: read-only permissions (
contents: read, issues: read, pull-requests: read, actions: read as needed) in all 4 cases; zero unneeded network access.
- Safe outputs:
create-issue for digests, add-comment for PR comments, noop fallback specified in every scenario for the no-match case.
View High Quality Responses (Top 3)
- IW-1 (Information Worker, stale-docs digest): Offloaded last-modified-date computation to a deterministic
git log step rather than having the agent parse raw history — matches the repo's "deterministic preprocessing" guidance. Scored 5 on trigger, tools, security, and completeness.
- PDM-2 (Product Manager, roadmap PR comments): Correctly mapped "PRs that close issues" to
pull_request: types: [closed] gated on merged == true, and scoped tools/permissions to exactly what's needed for closing-issue/label lookups. Scored 5 on trigger, security, and completeness.
- QA-1 (QA Tester, coverage-change comments): Matched the repo's own
test-coverage.md template almost verbatim — artifacts-first principle (download existing CI coverage artifact instead of recomputing), actions toolset, and hide-older-comments dedup.
View Areas for Improvement (Top 3)
- Ambiguity not surfaced: In 3 of 4 scenarios (PDM-1, PDM-2, QA-1) the agent made a silent assumption on an underspecified term ("completed" scope, closing-keyword detection method, coverage threshold source) instead of flagging it as a clarifying question — consistent with the 4/5
prompt_clarity ceiling.
- Fallback network scoping (QA-1): when no CI coverage artifact exists, the suggested fallback ("recompute coverage") needs an inferred package-ecosystem network allowance (node/go/python) rather than defaulting to
network: defaults alone — the recommendation got this right, but only because the evaluator had to reason it out explicitly; it's an easy place for a less careful pass to under-scope.
- Label-grouping edge case (PDM-1): no explicit fallback behavior was specified for issues carrying both or neither of
customer-facing/internal labels.
Recommendations
- Add explicit guidance (e.g. in
.github/aw/create-agentic-workflow.md) prompting the agent to surface 1-2 targeted clarifying questions when a request contains an ambiguous scope term (e.g. "completed," "closes," "threshold") before finalizing trigger/safe-output recommendations — this was the single most consistent gap across all 4 scenarios.
- Document the "replace previous result" pattern (
close-older-issues for recurring digest issues, hide-older-comments for PR comments) as a named, explicitly recommended default for recurring/PR-reactive workflows in .github/aw/github-agentic-workflows.md — it already appears correctly in 3 of 4 scenarios and is worth codifying rather than leaving implicit.
- Add a short note on fallback network-ecosystem scoping for "artifact-first, recompute-as-fallback" workflows (as seen in QA-1) so that recompute fallbacks don't default to
network: defaults without the inferred package ecosystem.
References:
Generated by 🎭 Agent Persona Explorer · claude · agent · 163.2 AIC · ⌖ 0.722 AIC · ⊞ 6.6K · ◷
Persona Overview
Key Findings
schedule+workflow_dispatchfor recurring digests (IW-1, PDM-1) vs.pull_requestevents for PR-reactive flows (PDM-2, QA-1).create-issue,add-comment) were applied in every scenario — no scenario suggested a write-scoped permission.close-older-issuesfor digests,hide-older-commentsfor PR comments) were suggested proactively in 3 of 4 scenarios, matching repo convention.prompt_clarityscored 4/5 (never 5) across all 4 scenarios — the agent's recommendation didn't surface a clarifying question for ambiguous scope terms (e.g. "completed" = closed issues vs. merged PRs, coverage threshold source, closing-keyword detection mechanics).Top Patterns
schedule+workflow_dispatchfor weekly digests;pull_request(closed+merged, or opened+synchronize) for PR-reactive automation.githubtool ingh-proxymode withtoolsets: [default], extended withactionstoolset only when CI artifact access was needed (QA-1).contents: read,issues: read,pull-requests: read,actions: readas needed) in all 4 cases; zero unneeded network access.create-issuefor digests,add-commentfor PR comments,noopfallback specified in every scenario for the no-match case.View High Quality Responses (Top 3)
git logstep rather than having the agent parse raw history — matches the repo's "deterministic preprocessing" guidance. Scored 5 on trigger, tools, security, and completeness.pull_request: types: [closed]gated onmerged == true, and scoped tools/permissions to exactly what's needed for closing-issue/label lookups. Scored 5 on trigger, security, and completeness.test-coverage.mdtemplate almost verbatim — artifacts-first principle (download existing CI coverage artifact instead of recomputing),actionstoolset, andhide-older-commentsdedup.View Areas for Improvement (Top 3)
prompt_clarityceiling.network: defaultsalone — the recommendation got this right, but only because the evaluator had to reason it out explicitly; it's an easy place for a less careful pass to under-scope.customer-facing/internallabels.Recommendations
.github/aw/create-agentic-workflow.md) prompting the agent to surface 1-2 targeted clarifying questions when a request contains an ambiguous scope term (e.g. "completed," "closes," "threshold") before finalizing trigger/safe-output recommendations — this was the single most consistent gap across all 4 scenarios.close-older-issuesfor recurring digest issues,hide-older-commentsfor PR comments) as a named, explicitly recommended default for recurring/PR-reactive workflows in.github/aw/github-agentic-workflows.md— it already appears correctly in 3 of 4 scenarios and is worth codifying rather than leaving implicit.network: defaultswithout the inferred package ecosystem.References: