Skip to content

Store database docstrings xz-compressed behind an optional doc feature #8836

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
youknowone merged 2 commits into from
Sep 27, 2026
Merged

Store database docstrings xz-compressed behind an optional doc feature #8836

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c3d62e5
Skip Python-defined classes when collecting builtin type docs
youknowone Sep 27, 2026
bdf7553
Store database docstrings xz-compressed behind an optional doc feature
youknowone Sep 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Skip Python-defined classes when collecting builtin type docs 
generate.py recorded every object.__subclasses__() entry as builtins.<name>
after importing the whole stdlib, so classes written in Python (tkinter,
unittest, tarfile, ...) ended up in the doc database. Classes that are
mutable heap types without C method descriptors and whose module is not a
C extension are now skipped, and their entries are removed from
data.inc.rs.

Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
  • Loading branch information
@youknowone
youknowone committed Sep 27, 2026
commit c3d62e55052fb9038159291e29180e2eab390415
23 changes: 23 additions & 0 deletions crates/doc/generate.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,14 @@

UNICODE_ESCAPE = re.compile(r"\\u([0-9]+)")

HEAPTYPE = 1 << 9
IMMUTABLETYPE = 1 << 8
C_DESCRIPTORS = (
types.MethodDescriptorType,
types.WrapperDescriptorType,
types.ClassMethodDescriptorType,
)

IGNORED_MODULES = {"this", "antigravity"}
IGNORED_ATTRS = {
"__annotations__",
Expand Down Expand Up @@ -177,6 +185,18 @@ def traverse(
yield DocEntry(new_parts, pydoc._getowndoc(attr))


def is_python_class(typ: type) -> bool:
"""Whether a class was defined in Python code rather than in C."""
if not typ.__flags__ & HEAPTYPE or typ.__flags__ & IMMUTABLETYPE:
return False
# C types keep their slot wrappers and method descriptors even when
# their __module__ names a Python module (e.g. ast.AST).
if any(isinstance(attr, C_DESCRIPTORS) for attr in vars(typ).values()):
return False
module = sys.modules.get(typ.__module__)
return module is not None and not is_c_extension(module)


def find_doc_entries() -> "Iterable[DocEntry]":
yield from (
doc_entry
Expand All @@ -196,6 +216,9 @@ def find_doc_entries() -> "Iterable[DocEntry]":
builtin_types.append(obj)

for typ in builtin_types:
# Classes defined in Python get their docs from source.
if is_python_class(typ):
continue
parts = ("builtins", typ.__name__)
yield DocEntry(parts, pydoc._getowndoc(typ))
yield from traverse(typ, __builtins__, parts)
Expand Down
Loading