Pi: replacement systemPrompt can defeat structured prompt-section cache preservation #953

Summary

Ponytail's Pi extension currently returns a replacement systemPrompt from
before_agent_start when a Ponytail mode is active.

On recent Pi versions, this can defeat cache-preserving structured system-prompt sections used by other extensions.

A concrete example is pi-subagents 0.73.1. It moved its dynamic agent catalog to:

event.systemPromptOptions.sections.advertised_subagents

specifically so enabling the subagent tool during a session can add a small system section without rewriting the leading system prompt.

pi-subagents documents an important limitation: if another loaded extension returns a replacement systemPrompt from before_agent_start, Pi may need to fold structured section changes back into the leading system prompt for that
request, which can invalidate the provider prompt cache.

I observed a large, prefix-sized prompt-cache miss immediately after enabling subagents in a warmed long Pi session with Ponytail active.

I am intentionally not attaching the original session transcript because it may contain private project context. The reproduction below uses only synthetic data.

Current Ponytail behavior

The current Pi extension follows this pattern:

pi.on("before_agent_start", async (event) => {
  if (!currentMode || currentMode === "off") return;

  return {
    systemPrompt: `${event.systemPrompt}\n\n${getPonytailInstructions(currentMode)}`
  };
});

The important part for this issue is that the handler returns a replacement systemPrompt.

Minimal reproduction

This reproduction requires no real repository, source code, credentials, memory contents, customer data, or other private information.

1. Install Pi 0.99.1.
2. Enable Pi's showCacheMissNotices.
3. Load pi-subagents 0.73.1 and Ponytail's Pi extension.
4. Set Ponytail to an active mode, for example full.
5. Keep the subagent tool inactive initially.
6. Start a fresh session using a provider/model with prompt-prefix caching.
7. Build a reasonably long conversation using synthetic text only.
8. Send another trivial synthetic prompt so the provider prefix cache is warm.
9. Enable the subagent tool with subagents_enable or the equivalent Pi flow.
10. Send one more trivial synthetic prompt.
11. Inspect Pi's cache-miss notice and, where available, provider cache-read usage.

Control

Repeat the same experiment with exactly the same:

• Pi version
• model/provider
• synthetic conversation
• pi-subagents version
• tool activation sequence

but with Ponytail either:

• not loaded, or
• set to off

Compare the cache-read / cache-miss behavior after enabling subagent.

Expected behavior

pi-subagents changes advertised_subagents through Pi's structured prompt sections.

That small section change should not require rewriting the already cached leading system prompt and conversation prefix.

Observed interaction

With Ponytail active, its before_agent_start handler returns a replacement systemPrompt.

Pi can therefore lose the structured-section cache-preservation path when advertised_subagents changes, potentially turning a small dynamic section change into a prefix-sized cache miss.

In my real-world observation the miss was large enough to be clearly prefix-sized, but I am deliberately not attaching that session or its contents because it contains private project context.

I have not yet completed a controlled provider-level A/B proving that Ponytail alone accounts for the entire observed miss. This report therefore describes the documented interaction and a reproducible hypothesis rather than claiming exclusive causation.

Relation to existing issues

#776

#776 is related because it also concerns the value returned from before_agent_start.

However, its proposed fix still returns a replacement systemPrompt. Preserving the multi-part prompt shape therefore appears orthogonal to the structured-section cache-preservation issue described here.

Resident prompt cost

This is also different from issues measuring the recurring cost of keeping Ponytail instructions resident in context.

This report is specifically about invalidating an already cached long prefix when another extension changes a structured system-prompt section.

pi-subagents

pi-subagents addressed the same class of cache-loss problem on its own side by moving its dynamic agent catalog into:

systemPromptOptions.sections.advertised_subagents

Its documentation notes that this optimization cannot preserve the cache for a request if another loaded extension returns a replacement systemPrompt.

Suggested direction

When the host supports structured prompt sections, Ponytail could inject its instructions through its own named section instead of replacing the entire system prompt, conceptually:

pi.on("before_agent_start", async (event) => {
  if (!currentMode || currentMode === "off") {
    delete event.systemPromptOptions?.sections?.ponytail;
    return;
  }

  event.systemPromptOptions.sections.ponytail =
    getPonytailInstructions(currentMode);
});

For compatibility with older Pi versions, the existing replacement behavior could remain as a fallback when systemPromptOptions.sections is unavailable.

Environment

• Pi: 0.99.1
• pi-subagents: 0.73.1
• Ponytail: current Pi extension
• Ponytail mode: active
• Provider: prompt-prefix-caching provider

Happy to help with a fully synthetic A/B harness or a PR once the preferred backward-compatibility approach is confirmed.