Skip to content

Latest commit

 

History

History
261 lines (193 loc) · 16.8 KB

v0.10.0-consolidation.md

File metadata and controls

261 lines (193 loc) · 16.8 KB

v0.10.0 Consolidation Design

Status: Design-locked. Q1-Q4 resolved 2026-05-21. Implementation can proceed. Date: 2026-05-21 (drafted), 2026-05-21 (Q1-Q4 resolved) Target release: v0.10.0 Reviewers: PluginForge maintainers, downstream consumer agents (Bibliogon, AdaptivLearner, pluginforge-app-template) Tracking: Adoption feedback from Bibliogon + AdaptivLearner + the template after v0.9.0 ships.

Context

v0.6.0 through v0.9.0 were four consecutive feature releases (lifecycle, identity, testing helpers, lifecycle visibility). Each grew the public surface. v0.10.0 is the first deliberate pause: a consolidation release.

Drivers:

  • Two independent consumers re-implemented the same hack. Both AdaptivLearner and pluginforge-app-template need to overlay user-supplied config on top of the YAML-loaded app config; both do it by assigning to manager._app_config with a type: ignore[attr-defined] escape hatch. Two independent code paths converging on the same private-attribute write is a real API gap.
  • Two re-exports were promised but never landed. FilterReason and config_diff are documented as public API in the v0.6.0 CHANGELOG and the plugin-author guide, but neither is in pluginforge/__init__.__all__. Consumers either deep-import from pluginforge.state / pluginforge.utils (which works but is poor signalling) or copy-paste the literal values.
  • DiscoveryResult.by_filter_reason() has no DiscoveryDiff equivalent. Bibliogon's reported usage pattern after every rediscover(): iterate diff.states.items() and group manually by state.filter_reason. The helper exists for one type and not the other; this is parity work, not a feature.
  • Prose drift. The [fastapi] extra was removed in v0.6.0 (commit aa9a009) but README.md and the consumer-integration guide still tell users to pip install pluginforge[fastapi]. Both consumers reported having to clean it up during adoption.
  • Release-process small wins. The v0.8.0 release surfaced the "always cd .. before clean-venv smoke" lesson that did not make it into the release workflow doc; adding it now closes the loop.
  • Single-router convention deprecation: the v0.8.0 announcement said v0.10.0 "may" make it an error. Bibliogon's export plugin still ships three routers and the refactor is backlogged P3. "May" never bound; extending the warning through v0.10.0 keeps consumers honest about migration without breaking anyone today.

Theme: API quality over API surface. Nothing new is added that does not have a concrete consumer report behind it. Items with no validated use case (Service-Registry, plugin metrics, hot-reload/ZIP unification, bidirectional compat matrix) stay out.

Scope table

# Item Tier Design needed?
1 DiscoveryDiff.by_filter_reason() parity helper Must No - mirrors DiscoveryResult.by_filter_reason() at state.py:114
2 __all__ audit: re-export FilterReason + config_diff (and any other documented-but-missing symbols) Must No - mechanical
3 Public config-overlay mechanism on PluginManager Must Yes - see "API surface" section below. Two independent consumers re-implemented the same manager._app_config = ... hack.
4 Scrub stale [fastapi] extra references (README + PyPI long_description + consumer-integration guide) Must No - mechanical, grep + edit
5 Add twine to [tool.poetry.group.dev.dependencies] Should No - one-line poetry add --group dev twine
6 Release-workflow doc: cd .. before clean-venv smoke (v0.8.0 methodology catch) Should No - one paragraph in .claude/rules/release-workflow.md
7 ARCHITECTURE.md audit (Issue #13) and update to v0.9.0 state Should No - content audit, not architecture decision
8 Single-router convention: keep as DeprecationWarning. Update timeline wording. Decision No - message-text edit only, two files (fastapi_ext.py:21, docs/guides/plugin-author.md:128) plus consumer-integration.md:318

Explicitly NOT in v0.10.0:

  • P1.2 Service-Registry (no validated use-case)
  • P3.1 Hot-Reload/ZIP unification (no second consumer)
  • P3.2 Plugin metrics (no consumer signal)
  • P3.3 Bidirectional compat matrix (precondition pending)
  • No new FilterReason values
  • No new PluginState fields
  • No new event hooks

API surface - item 3 (config-overlay method)

The hack being replaced

Both pluginforge-app-template and AdaptivLearner currently do this:

# template: backend/app/main.py:305-329
def _sync_manager_with_overlay() -> None:
    from app import config_overlay
    merged = config_overlay.read_app_config_merged()  # project YAML + user-overlay + secrets + env-vars
    try:
        manager._app_config = merged  # type: ignore[attr-defined]
    except Exception:
        logger.warning(
            "Could not patch PluginManager._app_config with overlay view; "
            "Settings-UI changes will not take effect until next restart."
        )

_sync_manager_with_overlay()
# ... later, inside lifespan:
manager.discover_plugins()

A second site in plugin_install.py does the same thing post-discovery (after a Settings-UI plugin install writes to the user-overlay YAML). AdaptivLearner does the same at its own main.py:285 per the v0.10.0 brief.

Two semantic patterns the consumers actually use

  1. Pre-discovery overlay: at startup, BEFORE discover_plugins(). Replaces the snapshot the manager loaded from app.yaml with a caller-computed merged view (project + user-overlay + secrets + env). No plugins are active yet, so on_config_changed would fire for nobody.
  2. Post-discovery overlay: after a Settings-UI write or env-var change, while plugins are active. Replaces the snapshot AND must notify active plugins via on_config_changed.

The existing refresh_config(new_app_config=dict) covers (2) - replace + notify is exactly what it does. It does NOT cleanly cover (1): consumers want the notify path skipped because there's nothing to notify, but refresh_config always notifies.

Two viable shapes

Option A - refresh_config(notify=...) (minimal)

Extend the existing method with a notify flag:

def refresh_config(
    self,
    new_app_config: dict[str, Any] | None = None,
    *,
    notify: bool = True,
) -> list[PluginError]:
    """... existing docstring ...

    Args:
        new_app_config: If provided, replaces the snapshot directly. If
            None, reloads app.yaml from disk (and refreshes i18n).
        notify: When True (default), call on_config_changed on each
            active plugin. When False, replace the snapshot silently.
            Set False during startup BEFORE discover_plugins() when no
            plugins are active to notify. The return value is always [].
    """

Pros: smallest change. One new kwarg. Backward-compatible. Cons: consumers still own the merge logic in their own code (_deep_merge is duplicated across templates). The method name says "refresh," not "merge."

Option B - new merge_app_config(overlay: dict) (richer)

Library owns the merge semantic. Consumers supply only the overlay layer:

def merge_app_config(
    self,
    overlay: dict[str, Any],
    *,
    notify: bool = True,
) -> list[PluginError]:
    """Deep-merge `overlay` on top of the current app config snapshot.

    Merge semantics: dicts merge recursively, lists and scalars replace.
    Matches the well-known YAML-overlay pattern: consumer-supplied
    project YAML + user-overlay + secrets + env-vars layered on top of
    the baseline app config.

    When `notify=True` (default), each active plugin's
    on_config_changed(old, new) is called with pre- and post-merge
    snapshots. When False, the snapshot is replaced silently -
    appropriate during startup before discover_plugins() when no
    plugins are yet active.

    Args:
        overlay: Dict to merge on top of the current config.
        notify: When True (default), notify active plugins. When False,
            return [] without calling on_config_changed.

    Returns:
        List of PluginError entries from plugins whose on_config_changed
        raised. Always [] when notify=False or no plugins are active.
    """

Pros: consumers shed their _deep_merge helper. Library owns the merge contract; consistency across consumers. merge_app_config(overlay) is more semantically accurate than refresh_config(merged_dict). Cons: mid-sized addition. The merge contract (dicts merge, lists replace) becomes a stability surface PluginForge must commit to. If a future consumer wants different merge semantics (e.g. lists concatenate), they're back to pre-computing the merge themselves.

Decision (APPROVED 2026-05-21)

Ship Option B AND the Option A kwarg. Add merge_app_config(overlay, *, notify=True) AND extend refresh_config with a notify: bool = True kwarg. Reasons:

  • The merge method is the canonical path. Both consumer hacks today carry their own _deep_merge implementation; centralising removes duplication risk.
  • The merge contract is the same trivial one both consumers use (dicts merge, lists/scalars replace). Locking that in carries near-zero forward-stability cost.
  • refresh_config(notify=False) covers the escape hatch: consumers who want custom merge semantics pre-compute the merged dict themselves and push it without firing hooks (AdaptivLearner's existing pattern).

Constraints (locked)

  1. The internal _deep_merge implementation MUST NOT be exposed as a public utility. The merge logic is an implementation detail of merge_app_config. Consumers who need custom merge semantics use refresh_config(merged_dict, notify=...) with their own pre-computed dict; they do NOT get a pluginforge.utils.deep_merge(...) to lean on. This keeps the merge contract local to one method, not a sprawl across the public API.

  2. merge_app_config is NOT a replacement for refresh_config. The two methods have distinct semantics:

    • merge_app_config(overlay): MERGE the overlay INTO the current snapshot. Dicts merge recursively, lists/scalars replace. The pre-existing keys not present in overlay are preserved.
    • refresh_config(new_app_config): REPLACE the entire snapshot with the caller-supplied dict. Pre-existing keys not present in new_app_config are LOST.

    The two methods exist as deliberate alternatives, not as aliases. The CHANGELOG and the docstring of each must say this explicitly.

Migration path

For pluginforge-app-template:

# Before (line 305-329 of backend/app/main.py):
def _sync_manager_with_overlay() -> None:
    merged = config_overlay.read_app_config_merged()
    try:
        manager._app_config = merged  # type: ignore[attr-defined]
    except Exception:
        logger.warning(...)

# After (option 1 - minimum diff, keep computing merged externally):
def _sync_manager_with_overlay() -> None:
    merged = config_overlay.read_app_config_merged()
    manager.refresh_config(merged, notify=False)

# After (option 2 - shed _deep_merge, supply only the overlay):
def _sync_manager_with_overlay() -> None:
    overlay = config_overlay.read_overlay_only()  # user_overlay + secrets + env, NOT project YAML
    manager.merge_app_config(overlay, notify=False)

The post-discovery site in plugin_install.py follows the same shape but with the default notify=True.

For AdaptivLearner: identical migration; the brief reports the same main.py:285 hack pattern.

Backward compat: both new entry points are additive. Existing manager._app_config = ... callers keep working (the attribute is not removed); the migration is opt-in. Doc the private-attr access as deprecated, slated for v1.0.0 removal in the v0.10.0 CHANGELOG.

Interaction with existing methods

Method Behavior in v0.10.0
reload_config() Unchanged. Reloads from disk, no notification. The new methods do NOT change this.
refresh_config(None) Unchanged. Reload from disk + notify (the default).
refresh_config(dict) Existing: replace snapshot + notify. v0.10.0 adds notify=False kwarg.
merge_app_config(overlay) New. Deep-merge overlay onto current snapshot + notify (default) or skip notify.
get_app_config() Unchanged. Returns whatever the current snapshot is (after merge if applicable).

Resolved questions (2026-05-21)

Q1. config_diff re-export shape (item 2). RESOLVED.

Add config_diff to pluginforge.__all__. from pluginforge import config_diff becomes the canonical import. from pluginforge.utils import config_diff continues to work; no breaking change. The CHANGELOG promise from v0.6.0 is honored at the package root.

Q2. Which Literal aliases are public (item 2). RESOLVED.

Re-export FilterReason and ErrorPhase (both documented in CHANGELOG v0.6.0). Severity stays private. Consumers comparing severity == "error" / severity == "warning" as strings is the intended pattern; exposing a Literal alias for that adds no value and creates a maintenance surface.

Q3. merge_app_config shape (item 3). RESOLVED.

Ship Option B + Option A kwarg. Specifically:

  • New method: merge_app_config(overlay: dict, *, notify: bool = True) on PluginManager. Recursive dict merge; lists/scalars replace. When notify=True, calls on_config_changed on active plugins via the same codepath as refresh_config.
  • refresh_config gains a notify: bool = True kwarg for consumers who push a pre-computed new config without triggering hooks (edge case; AdaptivLearner's _app_config hack does exactly this).
  • Migration path: consumers replace manager._app_config = merged with manager.merge_app_config(user_overlay). The type: ignore[attr-defined] block disappears.
  • Documented constraint: merge_app_config is NOT a replacement for refresh_config. The two methods have distinct semantics (merge vs replace); the docstring of each spells this out.

Additional constraint locked: the internal _deep_merge implementation is NOT exposed as a public utility. Consumers who need custom merge semantics use refresh_config(merged_dict, notify=...) with their own pre-computed dict.

Q4. Single-router warning timeline (item 8). RESOLVED.

Use "earliest v0.11.0". Updated wording in pluginforge/fastapi_ext.py warning text and the plugin-author + consumer-integration guides. Forward pressure preserved without overcommitting.

Implementation order sketch

Each commit lands individually green (per the project's atomic-commits rule):

  1. feat(diff): add DiscoveryDiff.by_filter_reason() (item 1)
    • One method on the dataclass + one unit test.
  2. feat(api): re-export FilterReason + ErrorPhase + config_diff at package root (item 2)
    • pluginforge/__init__.py __all__ additions + matching from ... import ... lines.
    • One test confirming from pluginforge import FilterReason, ErrorPhase, config_diff works.
  3. feat(manager): add merge_app_config + refresh_config(notify=) (item 3)
    • Two new entry points on PluginManager (the new method + the existing-method kwarg).
    • Unit tests: pre-discovery (notify=False, no plugins fired), post-discovery (notify=True, plugins fired, errors returned), merge semantic (dicts merge, lists replace).
    • Deprecate the _app_config private-attribute write in the v0.10.0 CHANGELOG ("deprecated, slated for removal at v1.0.0").
  4. docs(text): scrub stale [fastapi] extra references (item 4)
    • README.md line 16, docs/guides/consumer-integration.md if it carries the same line, pyproject.toml description if it does, any release-notes mentions.
  5. fix(text): update single-router deprecation timeline (item 8)
    • pluginforge/fastapi_ext.py:21 warning text.
    • docs/guides/plugin-author.md:128 and docs/guides/consumer-integration.md:318 prose.
    • No code-behavior change; deprecation stays as DeprecationWarning, error transition pushed to "v0.11.0 (earliest)".
  6. chore(dev): add twine to dev dependencies (item 5)
    • poetry add --group dev twine + lockfile regen.
  7. docs(release): document cd .. before clean-venv smoke (item 6)
    • One paragraph in .claude/rules/release-workflow.md under the build/publish step.
  8. docs(arch): audit ARCHITECTURE.md against v0.9.0 state (item 7, Issue #13)
    • Content audit pass; updates any stale references to v0.6.0/v0.7.0/v0.8.0 architecture, adds v0.9.0 lifecycle-visibility section, verifies the public-API listing matches __init__.__all__ (which itself just got updated in commit 2).

Total: 8 commits. Items 4-8 are scope-trivial; items 1-3 carry the implementation weight.

Verification

Per the project's numeric-claims-verification rule, every quoted test count, file count, or coverage number in the v0.10.0 CHANGELOG must be verified by running the authoritative command in the release session itself. No memory; no stale audit snapshots.

make test must stay green at every commit boundary. The full pre-commit hook chain (ruff check, ruff format --check, mypy, codespell, pytest) must pass on each commit.