Skip to content

Refactor code using unicode_encode_call_errorhandler() in unicodeobject.c #57418

Closed

Description

@vstinner
BPO 13209
Nosy @vstinner, @ezio-melotti

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2013-01-03.02:55:57.671>
created_at = <Date 2011-10-18.11:50:10.558>
labels = ['expert-unicode']
title = 'Refactor code using unicode_encode_call_errorhandler() in unicodeobject.c'
updated_at = <Date 2013-01-03.02:55:57.670>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2013-01-03.02:55:57.670>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2013-01-03.02:55:57.671>
closer = 'vstinner'
components = ['Unicode']
creation = <Date 2011-10-18.11:50:10.558>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 13209
keywords = []
message_count = 3.0
messages = ['145801', '145802', '178909']
nosy_count = 2.0
nosy_names = ['vstinner', 'ezio.melotti']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = 'needs patch'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue13209'
versions = ['Python 3.3']

Activity

  1. vstinner commented on Oct 18, 2011

    @vstinner
    MemberAuthor

    It's difficult to use unicode_encode_call_errorhandler() because the caller has to:

    • resize the output buffer (and check for integer overflow on the new size)
    • handle bytes and str for the replacement string: PyUnicode_EncodeDecimal() doesn't support bytes for example
    • encode replacement str: some encoders uses ASCII, unicode_encode_ucs1() uses Latin1, PyUnicode_EncodeCharmap() uses a recursive call (without check for infinite loop!), ... ; and raise a UnicodeEncodeError if the encoding fails

    It would be nice to factorize this code.

    I plan this implement this refactoring, it's just a reminder for me :-)

  2. ezio-melotti commented on Oct 18, 2011

    @ezio-melotti
    Member

    +1

  3. vstinner commented on Jan 3, 2013

    @vstinner
    MemberAuthor

    I tried to factorize the code, but it is too complex. Each encoder handles errors differently. The most tricky is charmap: it reencodes the result of the error handler for non-ASCII characters. I'm not happy with the current situtation, but I don't see how to factorize easily the code, so I prefer to leave it unchanged. I close the issue.

  4. transferred this issue fromon Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions