Repository navigation
Pydantic AI as an agentic engine #50971
Description
hey guys! @strawgate brought agentic workflows to our attention a couple months ago and we've been
using them since then (https://github.com/pydantic/pydantic-ai/tree/main/.github/workflows, pydantic/pydantic-ai#7211, pydantic/pydantic-ai#7253 (review)).
in parallel we've been running our own handrolled agent that does triage and review in our repo pydantic/pydantic-ai#7235 (comment).
we like agentic workflows and want to keep using them, while also dogfooding our own stack, which
is why we want to build a Pydantic AI based engine pydantic/pydantic-ai-harness#569 following the third-party model in your https://github.github.io/gh-aw/reference/engines/
claude's helping me build it so I'll leave its description below, + some questions we have for you guys so we can make the right choices
thank you for your time and attention!
the shape. An owner-maintained engine definition (engine: {id: pydantic-ai, behaviors: ...})
published in pydantic-ai-harness and imported
pinned to a tag, backed by a headless coding-agent CLI installed via runtimes: {uv: {}} +
pre-agent-steps (the aider.md pattern), with secret-strategy: universal-llm-consumer for
auth. Nothing needs to live in your repo or binary -- users import the definition from ours.
built already (pydantic-ai-harness#569):
the engine definition and a smoke workflow, compile-tested:
- a new
pydantic-aiengine id imported from a shared definition compiles clean with--stricton
v0.85.4. we bisected the releases: v0.84.3 still rejects new ids, the Refactoring opencode integration for shared workflows #50145 fix lands in v0.84.4.
we'll document v0.85.4 as our minimum supported version. - cross-repo pinned imports (
owner/repo/path.md@tag) resolve, SHA-pin, and vendor correctly --
tag, branch, and bare-SHA refs all work. - one docs/UX note from testing: the bare-string form (
engine: pydantic-ai) errors even when the
definition is imported -- only the object form (engine: {id: pydantic-ai}) works, and the error
message doesn't hint at that. cost us a minute; will cost every new user a minute.
being built now: the CLI itself. pydantic-ai-harness ships all the pieces a coding agent needs
(sandboxed filesystem + shell tools, repo context loading, planning, context compaction) but no
console entry point yet -- that's the gap we're closing, designed against your engine contract from
day one: prompt in, one-shot run, exit code out, MCP config consumed from the gateway so safe
outputs flow through the standard safeoutputs server.
three questions where your answers change what we build:
- install.
behaviors.installation.package-managervalues other thannpmcompile clean but
silently emit no install step. we're fine on thepre-agent-stepspath -- but a compile warning
for non-npm values would save the next Python/Rust engine author a confusing afternoon. want an
issue/PR for that? - logs and metrics. behavior-defined engines get no log parser: no conversation rendering in
the step summary, andgh aw logs/gh aw auditreport zero tokens and turns (AI-credit
accounting through the proxy still works). we get why -- GitHub Agentic Workflow Engine Enhancement Proposal #20416 scoped declarative log parsing
out as a non-goal -- but that was before imported engines became the recommended route for third
parties. is there appetite for a hook here, e.g. a documented streaming-JSONL schema plus
parse_custom_log.cjspromoted to a real contract? we'd emit whatever schema you standardize,
and we're happy to contribute the parser. - discoverability. once our definition is live and smoke-tested, would you take a link to it
from the third-party agent guide?
not asking for a row in the engines table or any support commitment from you -- just a pointer
for people searching "how do I run X on gh-aw".
whatever shape this takes, we intend to maintain it long-term -- so if any assumption above looks
wrong, tell us now and we'll adjust before we ship :)
Activity
dsfaccini commentedon Aug 17, 2026
Following up on the above: we rebuilt the engine definition around the coder agent that pydantic-ai-harness now ships, and validated it end to end on a live AWF runner.
The branch is ready at https://github.com/dsfaccini/gh-aw/tree/pydantic-ai-coder-engine (diff: main...dsfaccini:gh-aw:pydantic-ai-coder-engine) — repository settings prevent us from opening the pull request ourselves.
What it does, briefly:
Codercapability (filesystem, shell, planning, repo context, explorer sub-agent — 21 tools) plus one toolset per MCP gateway server, written as a generated Python module becausepai -atakes a single target and its spec format cannot name harness capabilities.-mis always passed (the CLI otherwise falls back toopenai:gpt-5silently), the provider endpoint is discovered from/reflectlike the Crush/DeepSeek definitions, and agent-load failures surface the real Python traceback instead ofpai's one-line error.smoke-pydantic.mdis re-added with its safe-output instructions corrected to the safeoutputs tools and aread_fileassertion that fails unless the coder toolset actually loaded.Evidence from the vendored-definition test repo: green run and the issue it filed through safeoutputs, which quotes the README's first line via a genuine
read_filecall.Could you either open the PR from that branch or enable us to? Happy to walk through any of it.
pelikhan commentedon Aug 18, 2026
Here is a pr - https://github.com/github/gh-aw/pull/53542/files
once you have it merged, I will update engines.json to point to the repo.
This is better as you keep ownership and the implementation.
dsfaccini commentedon Aug 25, 2026
hi @pelikhan
I don't get it, that's your PR which already merged. are you saying we should keep the implementation in our fork, and you'll point the agent definition to that?
pelikhan commentedon Aug 25, 2026
Yes correct... well we could host it here too and guess but you would have to send issue-as-spec to maintain it. Both work.
dsfaccini commentedon Sep 9, 2026
Hi @pelikhan, the definition is merged and lives on
mainof pydantic/pydantic-ai-harness:pydantic/pydantic-ai-harness/gh-aw/pydantic.md@main. User docs: https://github.com/pydantic/pydantic-ai-harness/blob/main/gh-aw/README.md.Two things in the compiler to look at when you repoint
engines.json, from reading currentmain(4c43622):knownEngineImportWithCompilerRefinpkg/workflow/engine_definition.goappends the gh-aw CLI version as@<ref>to any catalog import that has none. That ref does not exist in our repository (our tags arev0.26.0and so on), so the tip in the compile error would name a 404. An import outsidegithub/gh-awneeds@main, or a ref carried by the catalog entry.TestKnownEngineImportsFile_MatchesSharedEngineFilesinpkg/workflow/engine_definition_test.goforbids@in catalog imports and requires every engine under.github/workflows/shared/to be listed there, so the shared copy ofpydantic.mdand the test move together with the repoint.pelikhan commentedon Sep 9, 2026
@dsfaccini can file new issue? The long history bloats the context. Best to link new issue for new work.
dsfaccini commentedon Sep 9, 2026
Done: the repoint is #59755, written as a plan with the files, the compiler-ref change and the verification steps. Nothing further is needed on this thread from our side.