Python 3.13 compatibility

It seems that with new Python 3.13 alpha 3.13.0a5 (note with previous alpha this seemed not to happen as besides the small fix of #5035 it worked fine) tests fail very early:


+ cd robotframework-7.0
+ /usr/bin/python3 utest/run.py
Traceback (most recent call last):
  File "/builddir/build/BUILD/robotframework-7.0/utest/run.py", line 99, in <module>
    tests = get_tests(args.directory)
            ~~~~~~~~~^^^^^^^^^^^^^^^^
  File "/builddir/build/BUILD/robotframework-7.0/utest/run.py", line 56, in get_tests
    tests.extend(get_tests(fullname))
                 ~~~~~~~~~^^^^^^^^^^
  File "/builddir/build/BUILD/robotframework-7.0/utest/run.py", line 66, in get_tests
    module = __import__(modname)
             ~~~~~~~~~~^^^^^^^^^
  File "/builddir/build/BUILD/robotframework-7.0/utest/parsing/test_model.py", line 41, in <module>
    EXPECTED = File(sections=[
               ~~~~^^^^^^^^^^^
        ImplicitCommentSection(
        ^^^^^^^^^^^^^^^^^^^^^^^
    ...<86 lines>...
        )
        ^
    ])
    ^^
  File "/builddir/build/BUILD/robotframework-7.0/utest/../src/robot/parsing/model/blocks.py", line 72, in __init__
    super().__init__()
    ~~~~~~~~~~~~~~~~^^
SystemError: <method-wrapper '__init__' of File object at 0x7fc54cea0190> returned NULL without setting an exception

I tried to investigate a bit but sincerely got a bit lost in the flow of inheritance and what that super call should achieve. I know there is still some time for Python 3.13, but still reporting if it may give you an early warning.

If you have any proposal on a fix and need help to retest please just let me know!

This looks pretty strange and I tried reproducing this on Linux Mint, but deadsnakes only had alpha 4 and even that had issues with pip due to distutils being removed. Anyway, this looks pretty odd and could even be a bug in Python itself. That super().__init__() is there to call __init__ of ast.AST that's the root of the node hierarchy. I'm not entirely certain is calling it mandatory, at least simple tests seem to run fine also without it, but at least PyCharm complains if it's removed.

How urgent fixing this is for you? If you'd like this to be fixed in RF 7.0.1, the fix is needed pretty soon. If it's enough to get this fixed in RF 7.1 that's planned for June/July, then we have more time. One good thing with targeting RF 7.1 would be that Python 3.13 gets to beta phase in May and would then be a more stable target.

Thanks for checking this out! It was indeed very strange to me too being just the AST init after all.

As for urgency: please no worries! This would be needed for Fedora 41 which should be out somewhere around November, which also means by the time 3.13 should be final. Also in ~2 weeks a new 3.13 alpha should be released (and as you mention betas then in May), I can recheck there and report!

Sounds good to me. Let's concentrate on Python 3.13 support in RF 7.1. Delayed evaluation of annotations may also require us to make changes.

Indeed seems like a Python 3.13 bug. Minimal Python reproducer:

import ast


class File(ast.AST):
    _fields = ("xxx",)

    def __init__(self):
        super().__init__()


File()

$ python3.13 method_wrapper.py
Traceback (most recent call last):
  File ".../method_wrapper.py", line 11, in <module>
    File()
    ~~~~^^
  File ".../method_wrapper.py", line 8, in __init__
    super().__init__()
    ~~~~~~~~~~~~~~~~^^
SystemError: <method-wrapper '__init__' of File object at 0x7fadc9f71010> returned NULL without setting an exception

I've reported python/cpython#117266

@hroncok

Thanks for testing and reporting the issue @hroncok.

If the problem is in Python itself, I guess this issue can be closed. In that case a new generic issue about Python 3.13 compatibility should be submitted. We've had similar issues with Python 3.12 (#4771) and Python 3.11 (#4401).

Thanks for finding and reporting this bug in CPython this early in the release cycle!

It's possible that you will need to make changes in your code to accommodate the deprecations in https://docs.python.org/3.13/whatsnew/3.13.html#ast. I'm happy to help with that, but I looked at the file involved (https://github.com/robotframework/robotframework/blob/master/src/robot/parsing/model/statements.py) and it doesn't look like it has much to do with the Python AST. Perhaps it is enough if you add the _field_types field to your classes.

@JelleZijlstra

Thanks for the info @JelleZijlstra. Do I get it right that the reported issue will be fixed in beta 1, but there's a deprecation that we need to take into account? Is the _field_types field that you mention documented somewhere? ast module docs only mentions _fields.

That's right. I will have to change the docs to document _field_types.

It's true that our ast module usage doesn't have anything to do with Python AST. We simply used it as a base when we rewrote Robot Framework's parsing model few years ago. It got us started quickly and also these days has a benefit that we can use tools like ast.dump and the external astpretty module for debugging:

>>> import astpretty
>>> from robot.api.parsing import get_model
>>> m = get_model('''\
... *** Test Cases ***
... Example
...     Log    Hello, world!
... ''')
>>> astpretty.pprint(m)
File(
    source=None,
    languages=[],
    lineno=1,
    col_offset=0,
    end_lineno=3,
    end_col_offset=25,
    errors=(),
    sections=[
        TestCaseSection(
            lineno=1,
            col_offset=0,
            end_lineno=3,
            end_col_offset=25,
            errors=(),
            header=SectionHeader(type='TESTCASE HEADER', tokens=(Token(TESTCASE_HEADER, '*** Test Cases ***', 1, 0), Token(EOL, '\n', 1, 18)), lineno=1, col_offset=0, end_lineno=1, end_col_offset=19, errors=()),
            body=[
                TestCase(
                    lineno=2,
                    col_offset=0,
                    end_lineno=3,
                    end_col_offset=25,
                    errors=(),
                    header=TestCaseName(type='TESTCASE NAME', tokens=(Token(TESTCASE_NAME, 'Example', 2, 0), Token(EOL, '\n', 2, 7)), lineno=2, col_offset=0, end_lineno=2, end_col_offset=8, errors=()),
                    body=[KeywordCall(type='KEYWORD', tokens=(Token(SEPARATOR, '    ', 3, 0), Token(KEYWORD, 'Log', 3, 4), Token(SEPARATOR, '    ', 3, 7), Token(ARGUMENT, 'Hello, world!', 3, 11), Token(EOL, '\n', 3, 24)), lineno=3, col_offset=0, end_lineno=3, end_col_offset=25, errors=())],
                ),
            ],
        ),
    ],
)

I understand that although it has worked well, the ast module isn't designed for this kind of usage. If there are changes that case bigger problems for us in the future, we can rewrite the core of our AST so that it doesn't use ast anymore.

Thanks! Definitely an unusual use of AST, but since you're already doing it, from the CPython side we should preserve compatibility and continue to support it. I just opened an issue on CPython about this and I'll try to make it so that your use case continues to work well.

The original reported issue ought to be fixed in Python 3.13 itself, but we may still need to adjust our code to avoid deprecation warnings.

We obviously also want to support Python 3.13 otherwise and I changed the issue title to cover that. Supporting new Python versions hasn't typically required too much work from us, but in this case deferred evaluation of annotations (PEP 649) is likely to require at least some changed.

I'm about to open a PR updating my AST changes in CPython to relax the requirements on third-party classes that inherit from ast.AST. With that PR, the robotframework test suite passes for me locally on Python 3.13 with no deprecations.

Of course, there may be other changes that land in 3.13 yet, such as PEP 649. Ideally PEP 649 would not require changes for most user code, but we can't know for sure until we have the implementation.

Great to hear ast changes will land soon. We process annotations at run time so PEP 649 certainly can affect us.

PEP 649 didn't make it into Python 3.13 in the end, it was postponed to Python 3.14, but we still need to test against Python 3.13. One problem is that jsonschema that several of our tests use doesn't yet support Python 3.13. If that situation doesn't change before RF 7.1 is getting ready, we need to make tests using jsonschema optional so that they can be excluded similarly as, for example, tests requiring lxml.

jsonschema now supports Python 3.13 so I could run our tests without problems. Luckily there were only these issues that were easy to resolve:

telnetlib has been removed from the standard library. Easiest to fix by installing an external telnetlib module for tests.
There are subtle changes for how classmethod descriptors work. They aren't valid keyword in any version, but how that is detected is version dependent and changes reporting. Anyway, only test changes needed.
Traceback have ~~~^^^ markers in new places. Small change to tests needed.

Very importantly, no code changes were needed so also older Robot Framework versions are Python 3.13 compatible.

Now that we officially support Python 3.13, we can update classifiers in setup.py and version information in README and elsewhere. Setting up GitHub actions to run tests using Python 3.13 would be nice, but it requires some extra work until lxml supports Python 3.13. I believe that can wait because I'll run acceptance tests locally using Python 3.13 anyway.

Install external telnetlib w/ Python 3.13+ for tests

Python 3.13 compatibility in tests.

Declare Python 3.13 support.

Navigation Menu

Repository navigation

Python 3.13 compatibility #5091

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Python 3.13 compatibility #5091

Description

Activity

pekkaklarck commented on Mar 25, 2024

fedepell commented on Mar 26, 2024

pekkaklarck commented on Mar 26, 2024

hroncok commented on Mar 26, 2024

hroncok commented on Mar 26, 2024

pekkaklarck commented on Mar 26, 2024

pekkaklarck commented on Mar 26, 2024

JelleZijlstra commented on Mar 27, 2024

pekkaklarck commented on Apr 2, 2024

JelleZijlstra commented on Apr 2, 2024

pekkaklarck commented on Apr 2, 2024

JelleZijlstra commented on Apr 2, 2024

pekkaklarck commented on Apr 17, 2024

JelleZijlstra commented on Apr 24, 2024

pekkaklarck commented on Apr 26, 2024

pekkaklarck commented on Jun 4, 2024

pekkaklarck commented on Jul 12, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions