Regex 0.1.20101210 #54912

Activity

1. stiv commented on Dec 14, 2010

   

   stivmannequin

   on Dec 14, 2010

   MannequinAuthor

   More actions

   The regex package doesn't seem to correctly implement the single grapheme match "\X" (\P{M}\p{M}*) for pre-Python 3. I'm using the string "íi-te" (i, U+0301, i, -, t, e -- where U+0301 is Unicode COMBINING ACUTE ACCENT), reading it in from a file to bypass Unicode c&p issues in the older IDLEs).

   stiv@x$ python3.1
   Python 3.1.2 (r312:79147, May 19 2010, 11:50:28) 
   [GCC 4.1.2 20080704 (Red Hat 4.1.2-46)] on linux2
   Type "help", "copyright", "credits" or "license" for more information.
   >>> import regex
   >>> file = open("test_data", "rt", encoding="utf-8")
   >>> s = file.readline()
   >>> print (s)
   íi-te
   >>> print (g.findall(s))
   ['í', 'i', '-', 't', 'e']

   • Correct in 3.1 - i+U+0301 considered one grapheme.

   stiv@x$ python2.7
   Python 2.7 (r27:82500, Oct  4 2010, 14:49:53) 
   [GCC 4.1.2 20080704 (Red Hat 4.1.2-48)] on linux2
   Type "help", "copyright", "credits" or "license" for more information.
   >>> import codecs                                
   >>> import regex
   >>> file = codecs.open("test_data", "r", "utf-8")
   >>> g = regex.compile("\X")
   >>> s = file.readline()
   >>> s
   u'i\u0301i-te'
   >>> print s.encode("utf-8")
   íi-te
   >>> print g.findall(s)
   [u'i', u'\u0301', u'i', u'-', u't', u'e']

   *Not correct -- accent is treated as a separate character.

   Thanks.
2. stiv

   added

   topic-regex

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Dec 14, 2010
3. abalkin commented on Dec 14, 2010

   

   abalkin

   on Dec 14, 2010

   Member

   More actions

   Regex 0.1.20101210 is not part of the standard Python distribution, so this bug report is invalid.
4. abalkin

   closed this as completedon Dec 14, 2010
5. abalkin

   added

   invalid

   on Dec 14, 2010
6. bitdancer

   assigned

   mdickinson

   on Dec 14, 2010
7. mrabarnett commented on Dec 14, 2010

   

   mrabarnettmannequin

   on Dec 14, 2010

   Mannequin

   More actions

   The regex module is intended to replace the re module, so its default behaviour is the same: in Python 2, regexes default to matching ASCII, and in Python 3, they default to matching Unicode.

   If you want to use a regex on a Unicode string in Python 2 then you need to set the Unicode flag, either by providing the UNICODE flag or by putting "(?u)" in the regex itself.
8. bitdancer

   unassigned

   mdickinson

   on Dec 14, 2010
9. stiv commented on Dec 15, 2010

   

   stivmannequin

   on Dec 15, 2010

   MannequinAuthor

   More actions

   (Forehead slap.)

   On Tue, 14 Dec 2010, Matthew Barnett wrote:

   Matthew Barnett <python@mrabarnett.plus.com> added the comment:

   The regex module is intended to replace the re module, so its default behaviour is the same: in Python 2, regexes default to matching ASCII, and in Python 3, they default to matching Unicode.

   If you want to use a regex on a Unicode string in Python 2 then you need to set the Unicode flag, either by providing the UNICODE flag or by putting "(?u)" in the regex itself.

   ----------

   ---

   Python tracker <report@bugs.python.org>
   <http://bugs.python.org/issue10703\>

   ---
10. ezio-melotti

    transferred this issue fromon Apr 10, 2022