Repository navigation
3.11.0b1 regression: re.template removed without a deprecation period #92728
Description
Activity
- added a commit that references this issue
on May 12, 2022 I intend to send a PR later today.However, I am not sure if I can revert the removal + deprecate after the feature freeze, or if I must only revert the removal in 3.11 and deprecate it in 3.12+. Will start with a PR to main anyway.
EDIT: I won't send a PR before there is a consensus.
I don't understand the purpose of the
re.template()function and there.TEMPLATEflag. It doesn't seem to disable backtracking:>>> re.compile(r"((a)b\2)", re.IGNORECASE).match("aba").groups() ('aba', 'a') >>> re.template(r"((a)b\2)", re.IGNORECASE).match("aba").groups() ('aba', 'a')The only thing which seems to be changed is that repetition is not supported by TEMPLATE:
>>> re.template(r"a+") re.error: internal: unsupported template operator MAX_REPEAT >>> re.template(r"a*") re.error: internal: unsupported template operator MAX_REPEATCode search for "TEMPLATE" in re/sre source code files of Python 3.10:
Lib/re.py: TEMPLATE = T = sre_compile.SRE_FLAG_TEMPLATE # disable backtracking Lib/sre_compile.py: if flags & SRE_FLAG_TEMPLATE: Lib/sre_constants.py:SRE_FLAG_TEMPLATE = 1 # template mode (disable backtracking) Lib/sre_constants.py: f.write("#define SRE_FLAG_TEMPLATE %d\n" % SRE_FLAG_TEMPLATE) Lib/sre_parse.py: "t": SRE_FLAG_TEMPLATE, Lib/sre_parse.py:GLOBAL_FLAGS = SRE_FLAG_DEBUG | SRE_FLAG_TEMPLATE Modules/sre_constants.h:#define SRE_FLAG_TEMPLATE 1To be honest, I don't understand the purpose of the
re.template()function either and I think that deprecating and removing it is a good thing.If we decide to not revert the change, I suggest to document the removal in What's New in Python 3.11 and explain how to port impacted applications. For example, replace
re.template()withre.compile().If we decide to not revert the change, I suggest to document the removal in What's New in Python 3.11 and explain how to port impacted applications. For example, replace
re.template()withre.compile().I suppose we should do that even if we "just" deprecate it.
9 remaining items
I don't find any
"re\.template\(in the top PypI 5,000 packages.GitHub's new search tool finds only 19 Python files matching
/\bre\.template\(/, many of which are forks or duplicates.- https://github.com/elyscape/dnf/blob/5965627a81be59f6dbb9ff2362be1aba8f897237/dnf/cli/term.py#L284
- https://github.com/venaxyt/mysterium/blob/8cac9b1bbdb882873a0e260bfb8bfd273bcfcbc3/modules/re.py#L99
- https://github.com/containerbuildsystem/osbs-client/blob/f719759af18ef9f3bb45ee4411f80a9580723e31/make_mock_json.py#L142
- https://github.com/th3r00t/pyShelf/blob/aef30f3dcab4014d16a0f65388e92b6310f673a4/src/backend/lib/config.py#L61
- https://github.com/shoptime/trex/blob/75fea0e2f79a6be25917011456bbfef31aebdbaf/support/mongoengine.py#L20
- https://github.com/brinkqiang/dnf/blob/6517f47c94bdf2da2b1ce11605babee4d8cb8756/dnf/cli/term.py#L291
- https://github.com/tizenorg/platform.upstream.yum/blob/40014cdf8aae2f0a8c3eccd7f908e6ed1f5ddd65/output.py#L268
- https://github.com/lovaya/Ryven/blob/a11e361528d982a9dd3c489dd536f8b05ffd56e1/Ryven/packages/auto_generated/re/nodes.py#L355
- https://github.com/Distrotech/yum/blob/f4e54aeed297158c563828aa3ebb93d0c8ce7e38/output.py#L334
- https://github.com/rpm-software-management/yum/blob/4ed25525ee4781907bd204018c27f44948ed83fe/output.py#L334
- https://github.com/TheApacheCats/yum/blob/5c9c8db7b380457f71482a5ed28cd5fdb653a37b/output.py#L334
- https://github.com/timgates42/KomodoEdit/blob/980b5b00afda0144dd204c19de175b1a32ff12a8/src/codeintel/test2/test_python.py#L2220
- https://github.com/ryanuber/dnf/blob/fd8c108be591d52da6758214cb2b32a54d1afc31/dnf/cli/output.py#L334
- https://github.com/itmightbemo/KomodoEdit/blob/7f125d5140994e213ee892afac0210d7fa293699/src/codeintel/test2/test_python.py#L2182
- https://github.com/fatherlinux/containers-deep-dive/blob/76b3dbeb21c84d9536dc47660e958da520815b9e/containers201/rootfs/usr/share/yum-cli/output.py#L334
- https://github.com/cgwalters/dnf/blob/b81feda59ab79bc8a1ade4a54ee4b35789f2b75b/dnf/cli/term.py#L276
- https://github.com/SublimeCodeIntel/CodeIntel/blob/5a84f609c55972297119c2d7bc9d045222ee6d93/codeintel/test2/test_python.py#L2230
- https://github.com/alamatic/komodoedit/blob/19b97c35c59f1d232fa51bb0c96cb4cf5cc86248/src/codeintel/test2/test_python.py#L2082
- https://github.com/rmccue/codeintel/blob/5a6d47672c35d5fb1911312a8e3973284abd165d/test2/test_python.py#L2165
I have no opinion on this particular removal/deprecation, but just quickly want to point to PEP8 (https://peps.python.org/pep-0008/#public-and-internal-interfaces) and it's directive that:
To better support introspection, modules should explicitly declare the names in their public API using the __all__ attribute. Setting __all__ to an empty list indicates that the module has no public API.. Previously,templatewas in__all__which is also pre-faced with a# public symbolscomment.Perhaps there are reasons why this does not hold true for
CPython, but it would be good if that could be documented somewhere. As a package maintainer I have recently been struggling to correctly determine what is and isn't public in a codebase that does not currently document this. So guidelines such as the one in the PEP and general agreement on those guidelines would be helpful for me and the wider ecosystem I think.Reacted by Petr Viktorin and Dominic Davis-FosterBut if you think that it is worth, I am not opposing to reverting #32300 and adding a deprecation warning instead.
Yes, I believe that would be in the spirit of the policy. I will try to draft a PR shortly and then I will leave it for the others to decide if it's worth it.
How long the deprecation period should be? Would the only 3.11 be enough?
The policy speaks of 2 releases.
Reacted by Petr Viktorin- added a commit that references this issue
on May 24, 2022 Undocumented functions do not require a deprecation period.
I disagree.
Is this documented somewhere, or is it tribal knowledge (which should be documented, now that we have PEP-387 written)?Reacted by Dominic Davis-FosterThank you @hugovk for your investigation and PR.
Bug report
In Python 3.10, the
re.templatefunction exists and is not deprecated:In Python 3.11.0b1 this function is gone:
This was removed in b09184bf05 without a depreciation period.
Despite being undocumented, projects actually use this function, see e.g. rpm-software-management/dnf#1827
Please restore this function and deprecate it in 3.11 and 3.12 if you want to remove it in 3.13. (That is how I understand PEP 387, section Making Incompatible Changes.)
Your environment