re.sub confusion between count and flags args

@rhettinger

BPO	11957
Nosy	@rhettinger, @terryjreedy, @vstinner, @ericvsmith, @jwilk, @ezio-melotti, @merwok, @serhiy-storchaka
Dependencies	bpo-23591: enum: Add Flags and IntFlags
Files	patch_11957 re_keyword_only.patch re_check_flags_type.patch re_deprecate_positional_count.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ezio-melotti'
closed_at = None
created_at = <Date 2011-04-29.18:27:10.198>
labels = ['expert-regex', 'type-feature', '3.7']
title = 're.sub confusion between count and flags args'
updated_at = <Date 2017-04-14.18:55:29.189>
user = 'https://bugs.python.org/mindauga'

bugs.python.org fields:

activity = <Date 2017-04-14.18:55:29.189>
actor = 'jwilk'
assignee = 'ezio.melotti'
closed = False
closed_date = None
closer = None
components = ['Regular Expressions']
creation = <Date 2011-04-29.18:27:10.198>
creator = 'mindauga'
dependencies = ['23591']
files = ['30821', '37067', '44825', '44826']
hgrepos = []
issue_num = 11957
keywords = ['patch']
message_count = 27.0
messages = ['134806', '134820', '134830', '135371', '135386', '135391', '136657', '143520', '186784', '186825', '186832', '186844', '186856', '192416', '230223', '230224', '230226', '230236', '230237', '230238', '230358', '230375', '277386', '277398', '291655', '291659', '291677']
nosy_count = 13.0
nosy_names = ['rhettinger', 'terry.reedy', 'vstinner', 'eric.smith', 'jwilk', 'ezio.melotti', 'eric.araujo', 'mrabarnett', 'python-dev', 'mindauga', 'serhiy.storchaka', 'mmilkin', 'umi']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue11957'
versions = ['Python 3.6', 'Python 3.7']

Linked PRs

re.sub don't substitute not ASCII characters:

Python 2.7.1 (r271:86832, Apr 15 2011, 12:11:58) Arch Linux

>>import re

>>a=u'aaa'
>>print re.search('(\w+)',a,re.U).groups()
(u'aaa')
>>print re.sub('(\w+)','x',a,re.U)
x

  BUT:

>>a=u'ąąą'
>>print re.search('(\w+)',a,re.U).groups()
(u'\u0105\u0105\u0105')
>>print re.sub('(\w+)','x',a,re.U)
ąąą

The 4th parameter to re.sub() is a count, not flags.

Since this has been reported already several times (see e.g. bpo-11947), and it's a fairly common mistake, I think we should do something to avoid it.

A few possibilities are:

add a warning in the doc;
make count and flag keyword-only argument (raising a deprecation warning in 3.3 and actually change it later);
change the regex flags to some object that can be distinguished from ints and raise an error when a flag is passed to count;

I like the idea of an internal REflag class with __new__, __or__, and __repr__==str. Str(re.A|re.L) might print as
"REflag: re.ASCII | re.IGNORE"
If it is *not* an int subclass, any attempt to use or mix with an int would raise. I checked and the doc only promises that flags can be or'ed. An __and__ method might be added if it were thought that people currently use & to check for flags set, though that is not currently promised.

Something like "<re.Flag ASCII | IGNORE>" may be more Pythonic.

Agreed, if we go that route.

I’d favor 1) or 2) over 3). Ints are short and very commonly used for flags.

See also bpo-12888 for an error in the stdlib caused by this.

I like option #2, and I was thinking of working on it today, poke me if anyone has a problem with this.

There is no sane way to issue a warning without changing the signature and we don't want to change the signature without issuing a deprecation warning for the function, so sadly option 3 is the only way for this to work, (Im going to not touch this till ENUMS are merged in.)

test.py: pass "count" to re.sub() with kwarg

Remove deprecated use of count as positional arg

test.py: pass "count" to re.sub() with kwarg

crashpad: fix update.py DeprecationWarning under Python 3.13

Fix Python 3.13 re DeprecationWarning: pass count and flags by keyword

'maxsplit' should not be passed as positional argument

artiq_flash: Fix incorrect re.sub() call

Navigation Menu

Repository navigation

re.sub confusion between count and flags args #56166

Linked PRs

29 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

re.sub confusion between count and flags args #56166

Description

Linked PRs

Activity

mindauga commented on Apr 29, 2011

ericvsmith commented on Apr 29, 2011

ezio-melotti commented on Apr 30, 2011

terryjreedy commented on May 6, 2011

mrabarnett commented on May 6, 2011

terryjreedy commented on May 7, 2011

merwok commented on May 23, 2011

ezio-melotti commented on Sep 5, 2011

mmilkin commented on Apr 13, 2013

mmilkin commented on Apr 13, 2013

29 remaining items

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions