Skip to content

Adopt binacii.a2b_base64's strict mode in base64.b64decode #88856

Description

@idan22moral
BPO 44690
Nosy @gpshead, @idan22moral
PRs
  • bpo-44690: Adopt binacii.a2b_base64's strict mode in base64.b64decode #27272
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-08-23.23:45:16.798>
    created_at = <Date 2021-07-20.21:57:36.437>
    labels = ['type-bug', 'library', '3.11']
    title = "Adopt binacii.a2b_base64's strict mode in base64.b64decode"
    updated_at = <Date 2021-08-23.23:45:16.798>
    user = 'https://github.com/idan22moral'

    bugs.python.org fields:

    activity = <Date 2021-08-23.23:45:16.798>
    actor = 'gregory.p.smith'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-08-23.23:45:16.798>
    closer = 'gregory.p.smith'
    components = ['Library (Lib)']
    creation = <Date 2021-07-20.21:57:36.437>
    creator = 'idan22moral'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 44690
    keywords = ['patch']
    message_count = 3.0
    messages = ['397917', '400184', '400185']
    nosy_count = 2.0
    nosy_names = ['gregory.p.smith', 'idan22moral']
    pr_nums = ['27272']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'commit review'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue44690'
    versions = ['Python 3.11']

    Activity

    1. idan22moral commented on Jul 20, 2021

      idan22moralmannequin
      MannequinAuthor

      This is a follow-up PR to #68590.

      Currently, *base64.b64decode* uses a generic regex to validate *s* (when *validate* is true),
      which sometimes results in unexpected behavior and exception messages.

      Example:

      (1) base64.b64decode('ab==', validate=True) # b'i'
      (2) base64.b64decode('ab3==', validate=True) # b'i\xbd'
      (3) base64.b64decode('ab=3=', validate=True) # raises binascii.Error: Non-base64 digit found
      (4) base64.b64decode('ab==3', validate=True) # raises binascii.Error: Non-base64 digit found
      (5) base64.b64decode('ab===', validate=True) # raises binascii.Error: Non-base64 digit found
      (6) base64.b64decode('=ab==', validate=True) # raises binascii.Error: Non-base64 digit found

      The only strict-base64 valid example here is (1).
      (2), (4) and (5) should raise 'Excess data after padding',
      (3) should raise 'Discontinuous padding not allowed',
      and (6) should raise 'Leading padding not allowed'.

      To get this behavior, we can use the new (at the time of creating this PR) *binascii.a2b_base64* functionality of strict mode.

      I have one (not so big) concern - efficiency.
      I'm not that experienced with how fast regex-es are (in Python or in general) compared to the implementation of *binascii.a2b_base64* in C.
      So, I've no idea what would be the impact of migrating from regex pre-validation to input parsing.
      Let me know if you find it inefficient.

      -----

      Referenced issue (GH-24402): https://bugs.python.org/issue43086

    2. added
      3.11only security fixes
      stdlibStandard Library Python modules in the Lib/ directory
      type-bugAn unexpected behavior, bug, or error
      on Jul 20, 2021
    3. gpshead commented on Aug 23, 2021

      @gpshead
      Member

      I'm not worried about the regex vs binascii C implementation performance at all.

    4. gpshead commented on Aug 23, 2021

      @gpshead
      Member

      New changeset fa6304a by Idan Moral in branch 'main':
      bpo-44690: Adopt binacii.a2b_base64's strict mode in base64.b64decode (GH-27272)
      fa6304a

    5. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.11only security fixesstdlibStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or error

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions