Describe the issue
In the AI assistant chat (agent), the update_* tools (e.g. update_alert_rule) use a two-phase propose/confirm flow: the first tool call computes a change set and returns an approval interrupt (a pending proposal is persisted to the message extra and to Redis), and a later user reply "confirm" is resolved deterministically by replaying the tool's apply leg (router_ai_interrupt.go tryResumePending), which only triggers when the previous message carries extra.pending.
Problem: the model can produce the confirmation copy ("About to modify ... reply 'confirm' to apply") in plain reply text without actually invoking the tool. In that case no pending proposal is created, so when the user replies "confirm", tryResumePending finds prevMsg.Extra.Pending == nil and silently falls back to the normal agent flow. The change is never persisted, yet the agent is free to fabricate a "confirmed and applied" receipt because the runtime gives it no grounding. The user believes the change succeeded when it did not.
Observed in a production fork with update_alert_rule:
- User asks to change an alert rule threshold.
- The model replies with the exact confirmation copy ("以上改动尚未写入。回复「确认」立即生效…") but never calls
update_alert_rule → no pending proposal (the conversation transcript shows zero tool calls).
- User replies "确认".
- No pending → agent flow resumes; the model claims the change was applied, and may even fabricate the "✅ 已确认并写入" receipt.
- The rule's threshold stays unchanged; the user must repeat the request multiple times until one turn actually calls the tool.
Also observed: when the model does call the tool but passes the full baked expression (e.g. metric{...} < 20000) as the prom_ql argument of update_alert_rule, rebuildBakedPromQL treats it as the base and appends the current operator+threshold, producing an invalid query like metric{...} < 20000 < 40000 that is then shown to the user as the proposed change.
Expected behavior
A user reply "confirm" should either deterministically apply a real pending proposal, or be rejected with a clear message that there is nothing pending — never silently fall into the agent flow where the model can report a success that did not happen. Invalid/self-contradictory tool arguments should be rejected by the tool rather than concatenated into a bad expression.
Steps to reproduce
- Start an agent chat in n9e with the AI assistant feature enabled.
- Ask the assistant to modify an existing alert rule (e.g. change its threshold).
- Observe that the model may print the standard proposal confirmation copy without an actual tool call (verify via the conversation transcript / pending state).
- Reply with the confirmation word.
- The rule is not changed, and depending on the model the assistant may still say it applied.
Environment
- nightingale version: current main (v9.1.1 series)
- AI assistant chat with tool-use enabled;
update_alert_rule two-phase write gate
- LLM: observed with a DeepSeek-class chat model; likely model-agnostic
Describe the issue
In the AI assistant chat (agent), the
update_*tools (e.g.update_alert_rule) use a two-phase propose/confirm flow: the first tool call computes a change set and returns an approval interrupt (a pending proposal is persisted to the message extra and to Redis), and a later user reply "confirm" is resolved deterministically by replaying the tool's apply leg (router_ai_interrupt.gotryResumePending), which only triggers when the previous message carriesextra.pending.Problem: the model can produce the confirmation copy ("About to modify ... reply 'confirm' to apply") in plain reply text without actually invoking the tool. In that case no pending proposal is created, so when the user replies "confirm",
tryResumePendingfindsprevMsg.Extra.Pending == niland silently falls back to the normal agent flow. The change is never persisted, yet the agent is free to fabricate a "confirmed and applied" receipt because the runtime gives it no grounding. The user believes the change succeeded when it did not.Observed in a production fork with
update_alert_rule:update_alert_rule→ no pending proposal (the conversation transcript shows zero tool calls).Also observed: when the model does call the tool but passes the full baked expression (e.g.
metric{...} < 20000) as theprom_qlargument ofupdate_alert_rule,rebuildBakedPromQLtreats it as the base and appends the current operator+threshold, producing an invalid query likemetric{...} < 20000 < 40000that is then shown to the user as the proposed change.Expected behavior
A user reply "confirm" should either deterministically apply a real pending proposal, or be rejected with a clear message that there is nothing pending — never silently fall into the agent flow where the model can report a success that did not happen. Invalid/self-contradictory tool arguments should be rejected by the tool rather than concatenated into a bad expression.
Steps to reproduce
Environment
update_alert_ruletwo-phase write gate