doctest fails to collect tests from a C function that has been wrapped

Bug report

Bug description:

doctest.DocTestFinder is now failing to collect examples from functions that are defined in C and then wrapped. It still works just fine with functions that are defined in C but that are not wrapped.

This bug was introduced by #115440. It breaks doctests for Numpy ufuncs in pytest-doctestplus (see scientific-python/pytest-doctestplus#248).

I have placed reproducer code in this Gist: https://gist.github.com/lpsinger/65e59728555dc2096af88d394e2d4a6b. To reproduce, retrieve the code and run the following commands:

pip install -e .
python test.py

The script test.py fails with this error message:

$ python test.py 
Traceback (most recent call last):
  File "/Users/lpsinger/src/doctest-func-without-code/test.py", line 14, in <module>
    assert len(finder.find(bar.hello)[0].examples) == 1
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/doctest.py", line 942, in find
    self._find(tests, obj, name, module, source_lines, globs, {})
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/doctest.py", line 1004, in _find
    test = self._get_test(obj, name, module, globs, source_lines)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/doctest.py", line 1072, in _get_test
    lineno = self._find_lineno(obj, source_lines)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/doctest.py", line 1121, in _find_lineno
    obj = inspect.unwrap(obj).__code__
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'builtin_function_or_method' object has no attribute '__code__'. Did you mean: '__call__'?

CPython versions tested on:

3.9

Operating systems tested on:

macOS

Linked PRs

Shorter repro on main:

>>> import binascii, doctest, functools
>>> def dec(func):
...    @functools.wraps(func)
...    def wrapper(*args, **kwargs):
...         return func(*args, **kwargs)
...    return wrapper
... 
>>> b = dec(binascii.b2a_hex)
>>> finder = doctest.DocTestFinder()
>>> finder.find(b)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    finder.find(b)
    ~~~~~~~~~~~^^^
  File "/Users/sobolev/Desktop/cpython2/Lib/doctest.py", line 964, in find
    self._find(tests, obj, name, module, source_lines, globs, {})
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython2/Lib/doctest.py", line 1026, in _find
    test = self._get_test(obj, name, module, globs, source_lines)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython2/Lib/doctest.py", line 1094, in _get_test
    lineno = self._find_lineno(obj, source_lines)
             ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython2/Lib/doctest.py", line 1143, in _find_lineno
    obj = inspect.unwrap(obj).__code__
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'builtin_function_or_method' object has no attribute '__code__'. Did you mean: '__call__'?

Looks like I might have an idea on how to fix it! Thanks a lot for the report! 👍

pythongh-117692: Fix `AttributeError` in `DocTestFinder` on wrapped `…

Temporarily pin Python patch version for tests

gh-117692: Fix `AttributeError` in `DocTestFinder` on wrapped `builti…

pythongh-117692: Fix `AttributeError` in `DocTestFinder` on wrapped `…

[3.12] gh-117692: Fix `AttributeError` in `DocTestFinder` on wrapped …

@pablogsal

A fix has been merged, and backported to 3.12. Unfortunately, I think it probably can't be backported to 3.11 due to 3.11 now being in security-only mode; I'll let @pablogsal reopen if he thinks this warrants an exception

pythongh-117692: Fix `AttributeError` in `DocTestFinder` on wrapped `…

@pablogsal

@pablogsal The regression introduced in #115440 was released in the final bugfix of Python 3.11, meaning this bug will remain present indefinitely on that version of Python. I'm unsure if that's happened before, but it means that bugs like jaraco/path#231 have no proper fix until Python 3.11 is sunset. I can't even think of a good workaround, short of disabling doctests on all Python versions or finding some way to disable doctests in pytest for a specific Python version. Do you know if it's happened before that a regression has been introduced in the last bugfix release for a minor version?

I think this bug deserves consideration for an extra bugfix patch release (maybe a selective one against 3.11.9 with only this bugfix).

Disable doctests on Python 3.11.9+ but not 3.12.

I think this bug deserves consideration for an extra bugfix patch release (maybe a selective one against 3.11.9 with only this bugfix).

That would certainly be very helpful to me. I maintain several domain-specific scientific Python packages with doctests affected by this, and I have had to skip the doctests for Python 3.11, which makes me nervous given that I still claim that I support that version.

I think this bug deserves consideration for an extra bugfix patch release (maybe a selective one against 3.11.9 with only this bugfix).

I think it makes sense and I am happy to back port and batch this into the next security update of 3.11.

I think it makes sense and I am happy to back port and batch this into the next security update of 3.11.

Works for me.

The problem with only rolling the fix into a security-only release is that the installers (and others) don't get updated with security updates, so users of the installers will be perpetually stuck with the bug. I worry that things like GitHub CI on Windows may just use the latest installer and thus may not pick up the fix. I guess we can see how many environments that leaves without a solution. Hopefully the intersection of that with projects that rely on doctests is exceedingly small.

That would be fine with me. My affected packages don't even build on Windows!

Any update on backporting this?

@pablogsal

I think this bug deserves consideration for an extra bugfix patch release (maybe a selective one against 3.11.9 with only this bugfix).

I think it makes sense and I am happy to back port and batch this into the next security update of 3.11.

@pablogsal I was making the case for making another non-security-only release with this fix, so that the most recent packaged release will include the bugfix, even if it's 3.11.9.1 or 3.11.9+1 or similar). I realize this falls out of the standard operating procedure for Python releases, but without some special handling, this bug will exist essentially in perpetuity in the Python 3.9 line. Since the issue was introduced in the latest non-security release, I think that deserves another non-security release to address the regression.

Navigation Menu

Repository navigation

doctest fails to collect tests from a C function that has been wrapped #117692

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

doctest fails to collect tests from a C function that has been wrapped #117692

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

sobolevn commented on Apr 9, 2024

sobolevn commented on Apr 9, 2024

AlexWaygood commented on Apr 10, 2024

jaraco commented on Jul 27, 2024

lpsinger commented on Aug 12, 2024

pablogsal commented on Aug 13, 2024

jaraco commented on Aug 13, 2024

lpsinger commented on Aug 14, 2024

lpsinger commented on Dec 5, 2024

jaraco commented on Dec 28, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions