regression(2.0.9): relative cross-references in re-exported and inherited objects expand against private modules #346

Description of the bug

Since 2.0.9, relative cross-references are expanded against the module where an object is defined rather than the path where it's documented. This breaks the very common layout of defining things in private modules and re-exporting them from a public one, which used to work fine up to 2.0.8.

The cause is a9a4ca3 (#342), the fix for #341, which I reported. It anchors the dot-walk on docstring.parent whenever it's set. That's what #341 needed for members inherited from another package, but docstring.parent is also set for every alias, and there it points at the private defining object. I mentioned that risk at the end of #341, but I didn't realize how common the affected case is, sorry about that.

There are three variants, all shown by the reproducer below:

1. A re-exported object: pkg.Base is an alias to pkg._base.Base, so [..Derived] in its docstring now expands to pkg._base.Derived instead of pkg.Derived.
2. A member of a re-exported class: pkg.Base.method gets pkg._base.helper instead of pkg.helper.
3. An inherited member whose base class is defined in a private module: pkg.Derived.method gets pkg._base.helper too.

References only keep working when the target happens to be defined in the same private module, because then the expanded canonical path still matches an anchor. That makes the breakage look random from the outside.

To Reproduce

mkdir -p repro/pkg repro/docs
cd repro

cat > pkg/__init__.py <<'EOF'
"""The package."""

from pkg._base import Base
from pkg._derived import Derived
from pkg._helpers import helper

__all__ = ["Base", "Derived", "helper"]
EOF

cat > pkg/_helpers.py <<'EOF'
"""Private module defining helpers."""


def helper() -> None:
    """A helper."""
EOF

cat > pkg/_base.py <<'EOF'
"""Private module defining the base class."""


class Base:
    """A base class, see also [`Derived`][..Derived]."""

    def method(self) -> None:
        """A method, see also [`helper`][...helper]."""
EOF

cat > pkg/_derived.py <<'EOF'
"""Private module defining the derived class."""

from pkg._base import Base


class Derived(Base):
    """A derived class."""
EOF

printf '# Package\n\n::: pkg\n' > docs/index.md

cat > mkdocs.yml <<'EOF'
site_name: pkg
strict: true
plugins:
- mkdocstrings:
    handlers:
      python:
        options:
          inherited_members: true
          relative_crossrefs: true
EOF

for version in 2.0.8 2.0.9; do
  echo "### mkdocstrings-python $version"
  python -m venv .venv-$version
  .venv-$version/bin/pip install -q --disable-pip-version-check mkdocs mkdocstrings mkdocstrings-python==$version
  PYTHONPATH=. .venv-$version/bin/mkdocs build 2>&1 | grep -v '^INFO'
done

Output:

### mkdocstrings-python 2.0.8
### mkdocstrings-python 2.0.9
WARNING -  mkdocs_autorefs: index.md: from /tmp/repro/pkg/_base.py:5: (pkg.Base) Could not find cross-reference target 'pkg._base.Derived'
WARNING -  mkdocs_autorefs: index.md: from /tmp/repro/pkg/_base.py:8: (pkg.Base.method) Could not find cross-reference target 'pkg._base.helper'
WARNING -  mkdocs_autorefs: index.md: from /tmp/repro/pkg/_base.py:8: (pkg.Derived.method) Could not find cross-reference target 'pkg._base.helper'

Aborted with 3 warnings in strict mode!

Full traceback

There is no traceback, only the warnings above, which abort the build in strict mode.

Expected behavior

The same as in 2.0.8: relative references expand from the path where the object is documented, pkg.Derived and pkg.helper here.

Environment information

• System: Linux-6.12.111+deb13-amd64-x86_64-with-glibc2.42
• Python: cpython 3.12.14
• Installed packages:
  • mkdocs v1.6.1
  • mkdocstrings v1.0.6
  • mkdocstrings-python v2.0.9
  • mkdocs-autorefs v1.4.4
  • griffelib v2.3.0

Additional context

This breaks the docs builds of several projects in the frequenz-floss organization when updating to 2.0.9. For example, frequenz-floss/frequenz-core-python#201 (3 warnings), frequenz-floss/frequenz-channels-python#560 (28), frequenz-floss/frequenz-dispatch-python#306 (6) and frequenz-floss/frequenz-client-common-python#291 (330). All of them build cleanly with 2.0.8.

About a fix, I looked into a narrow one first: keeping the new anchor only for inherited aliases (self.current_object.inherited or not self.current_object.is_alias). That fixes variants 1 and 2, but not variant 3, which is most of the client-common warnings. For inherited members, the defining object is the right tree but the wrong path whenever the base class lives in a private module, and I couldn't find a reliable way to get its public path. Restricting the new anchor to members inherited from another package doesn't work either: Alias.package returns the target's package, and namespace packages like frequenz.* share the same top module.

So I'd suggest reverting a9a4ca3 if we don't find a proper fix soon, which brings back the 2.0.8 behavior and with it the cross-package case of #341. That seems like the smaller problem: it only affects relative references in docstrings inherited from another package, it has the workaround discussed in #344, and a proper fix would need the kind of cross-reference rework that, as you said there, is better left to Zensical. This regression, on the other hand, affects the most common package layout. I can send a PR with the revert plus regression tests for the three variants above, if you agree.