RFC: Agent model_name: declared, inherited, or both? #337

cc @BlazeUp-AI/maintainers — looking for design input.

TL;DR

Today every agent in the registry has a required model_name field set at creation time (e.g., claude-sonnet-4, gemini-2.5-pro). But for most target IDEs that field is silently ignored — the IDE runs whatever model the user has selected in their session. I'd like community input on whether model_name should stay as-is, become advisory metadata only, or be dropped in favor of "inherit from host IDE."

Current behavior

When you observal pull <agent> --ide <ide>, here's whether agent.model_name actually reaches runtime:

So an agent author who picks gemini-2.5-pro and publishes today is, for 5 of 7 supported IDEs, communicating a fact that has no operational effect.

Why this is a design question, not just a bug

It's tempting to say "drop the field." But model_name does pull weight beyond runtime:

1. Documentation / intent — "I authored and tested this agent against Sonnet 4"
2. Telemetry tag — every trace records agent.model_name so you can compare runs across model versions
3. Eval grouping — scorecards group by model so you don't compare apples-to-oranges
4. Registry browsing / filtering — "show me agents tested on Claude Sonnet 4"
5. Cost estimation (planned) — knowing the model helps estimate token cost
6. Actually runtime-honored on Kiro

Dropping it loses these. Keeping it as-is means the field is misleading. Renaming/repurposing might land in the middle.

Sub-question: do agents naturally inherit?

Most published agents I've seen elsewhere (Claude Code subagents, Cursor rules, Codex AGENTS.md, Copilot instructions) are prompt + tool packages without a model declaration — they assume the host IDE's model. The host model is the right abstraction for the user: "I picked Opus for this session, my subagents use Opus too."

Kiro is the outlier — it lets each agent pin its own model, which is occasionally useful (a cheap classification subagent on Haiku while the main loop is on Opus).

Options under consideration

Option A — Make model_name advisory metadata only

• Rename to recommended_model / tested_with
• Make it optional on creation
• Keep it for telemetry, eval grouping, registry filtering
• At install time, warn if the host IDE can't honor it ("This agent was authored for gemini-2.5-pro; Claude Code will use its session model instead")
• Pro: honest about what we control, keeps all the metadata value
• Con: no longer enforces consistency where it could (Kiro)

Option B — Make model_name a per-install override

• Drop from agent registration entirely
• Add --model to observal pull for IDEs that support it (Claude Code, Kiro)
• Telemetry records the runtime model from the trace, not a declared one
• Pro: matches how most IDEs actually work
• Con: lose authored-with-X provenance; eval grouping needs another dimension

Option C — Keep model_name required, enforce per-IDE compatibility at install

• Reject observal pull <gemini-agent> --ide claude-code with a clear error
• Force agent authors to publish per-IDE variants if they want cross-IDE reach
• Pro: never misleading
• Con: high friction for authors, splits the registry, breaks the "one agent, many IDEs" promise

Option D — Two-layer: authored_for (advisory) + runtime_model (per-install, optional)

• authored_for is metadata, locked at registration
• runtime_model is set at install time for IDEs that support it
• Telemetry records both
• Pro: keeps provenance AND respects host IDE realities
• Con: more concepts for users to learn

Specific questions for the community

1. When you publish an agent intended to work across Claude Code + Cursor + Gemini CLI, do you actually care which model it runs on at the user's end? Or is "use whatever you have" the right default?
2. Do you rely on agent.model_name for filtering or comparison today? If we made it optional, would that hurt your workflow?
3. For Kiro users specifically: do you want the agent file to keep pinning a model, or would you prefer it inherit too?
4. If we adopt Option D, would you actually use a per-install --model override, or is that complexity nobody asked for?
5. Any prior art from other agent registries (LangChain Hub, AutoGen, Open WebUI, etc.) we should look at?

Context

• Repo: https://github.com/BlazeUp-AI/Observal
• Relevant code:
  • Agent schema: observal-server/schemas/agent.py
  • Per-IDE config generation: observal-server/services/agent_config_generator.py
  • Install-time options (already supports --model for Claude Code): observal_cli/cmd_pull.py

Looking forward to your thoughts — particularly if you've hit the "authored for model X, but the IDE runs model Y" footgun in practice.

— Apoorv