Skip to content

generated email message exceeds RFC-mandated limit of 998 characters #84777

Closed
Closed
generated email message exceeds RFC-mandated limit of 998 characters#84777
Labels
3.8 (EOL)end of life3.9 (EOL)end of lifetopic-emailtype-bugAn unexpected behavior, bug, or error

Description

@ivyl
ivyl
mannequin
opened on May 11, 2020
BPO 40597
Nosy @warsaw, @msapiro, @bitdancer, @maxking, @miss-islington, @ivyl
PRs
  • bpo-40597: email: Don't shy from using CTE for US-ASCII-only emails #20038
  • [3.8] bpo-40597: email: Use CTE if lines are longer than max_line_length consistently (gh-20038) #20084
  • bpo-40597: Allow email.contextmanager set_content() to set a null string. #20542
  • [3.9] bpo-40597: Allow email.contextmanager set_content() to set a null string. (GH-20542) #21404
  • [3.8] bpo-40597: Allow email.contextmanager set_content() to set a null string. (GH-20542) #21405
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2020-07-09.00:18:35.262>
created_at = <Date 2020-05-11.19:00:19.898>
labels = ['type-bug', '3.8', 'expert-email', '3.9']
title = 'generated email message exceeds RFC-mandated limit of 998 characters'
updated_at = <Date 2020-07-09.00:19:01.131>
user = 'https://github.com/ivyl'

    bugs.python.org fields:

    activity = <Date 2020-07-09.00:19:01.131>
actor = 'maxking'
assignee = 'none'
closed = True
closed_date = <Date 2020-07-09.00:18:35.262>
closer = 'maxking'
components = ['email']
creation = <Date 2020-05-11.19:00:19.898>
creator = 'ivyl'
dependencies = []
files = []
hgrepos = []
issue_num = 40597
keywords = ['patch']
message_count = 11.0
messages = ['368641', '368646', '368810', '368811', '369157', '370395', '373350', '373353', '373354', '373362', '373363']
nosy_count = 6.0
nosy_names = ['barry', 'msapiro', 'r.david.murray', 'maxking', 'miss-islington', 'ivyl']
pr_nums = ['20038', '20084', '20542', '21404', '21405']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue40597'
versions = ['Python 3.8', 'Python 3.9']

    Activity

    1. ivyl commented on May 11, 2020

      @ivyl
      ivylmannequin
      on May 11, 2020
      MannequinAuthor

      RFC53220 in section 2.1.1 mandates that the line cannot be longer than 998 characters and should not be longer than 78 characters (excluding CRLF). When we use raw_data_manager (default for EmailPolicy, EmailMessage) it does the correct thing as long as the message contains characters outside of 7bit US-ASCII set - base64 or qp Content-Transfer-Encoding which respects the line wrapping at 78 characters.

      However if our message is limited the characters from the 7bit US-ASCII set no transfer encoding is applied, and such messages can easily go beyond 78 or even 998 characters.

    2. ivyl
      added
      3.7 (EOL)end of life
      3.8 (EOL)end of life
      3.9 (EOL)end of life
      topic-email
      type-bugAn unexpected behavior, bug, or error
      on May 11, 2020

    3. bitdancer commented on May 11, 2020

      @bitdancer
      bitdancer
      on May 11, 2020
      Member

      The PR looks good to me, but I describe the change differently. I'm not sure how I missed this in the original implementation, since I obviously checked it for the 8bit case. Too long ago to remember :)

    4. bitdancer commented on May 14, 2020

      @bitdancer
      bitdancer
      on May 14, 2020
      Member

      New changeset 6f2f475 by Arkadiusz Hiler in branch 'master':
      bpo-40597: email: Use CTE if lines are longer than max_line_length consistently (gh-20038)
      6f2f475

    5. bitdancer commented on May 14, 2020

      @bitdancer
      bitdancer
      on May 14, 2020
      Member

      Thanks, Arkadiusz.

    6. bitdancer
      removed
      3.7 (EOL)end of life
      on May 14, 2020

    7. bitdancer commented on May 18, 2020

      @bitdancer
      bitdancer
      on May 18, 2020
      Member

      New changeset c1f1ddf by Miss Islington (bot) in branch '3.8':
      bpo-40597: email: Use CTE if lines are longer than max_line_length consistently (gh-20038) (gh-20084)
      c1f1ddf

    8. msapiro commented on May 30, 2020

      @msapiro
      msapiromannequin
      on May 30, 2020
      Mannequin

      With the fix in PR 20038, committed at 6f2f475 it is no longer possible to set_content(''). Attempts to do so produce the following

        File "/var/MM/3/hk_39/hyperkitty/.tox/py39-django30/lib/python3.9/site-packages/django_mailman3/lib/scrub.py", line 95, in _get_all_attachments
    part.set_content('')
  File "/usr/local/lib/python3.9/email/message.py", line 1171, in set_content
    super().set_content(*args, **kw)
  File "/usr/local/lib/python3.9/email/message.py", line 1101, in set_content
    content_manager.set_content(self, *args, **kw)
  File "/usr/local/lib/python3.9/email/contentmanager.py", line 37, in set_content
    handler(msg, obj, *args, **kw)
  File "/usr/local/lib/python3.9/email/contentmanager.py", line 185, in set_text_content
    cte, payload = _encode_text(string, charset, cte, msg.policy)
  File "/usr/local/lib/python3.9/email/contentmanager.py", line 149, in _encode_text
    if max(len(x) for x in lines) <= policy.max_line_length:
ValueError: max() arg is an empty sequence

    9. miss-islington commented on Jul 8, 2020

      @miss-islington
      miss-islington
      on Jul 8, 2020
      Contributor

      New changeset 4fa61a7 by Mark Sapiro in branch 'master':
      bpo-40597: Allow email.contextmanager set_content() to set a null string. (GH-20542)
      4fa61a7

    10. miss-islington commented on Jul 8, 2020

      @miss-islington
      miss-islington
      on Jul 8, 2020
      Contributor

      New changeset c1c5034 by Miss Islington (bot) in branch '3.8':
      bpo-40597: Allow email.contextmanager set_content() to set a null string. (GH-20542)
      c1c5034

    11. miss-islington commented on Jul 8, 2020

      @miss-islington
      miss-islington
      on Jul 8, 2020
      Contributor

      New changeset e689789 by Miss Islington (bot) in branch '3.9':
      bpo-40597: Allow email.contextmanager set_content() to set a null string. (GH-20542)
      e689789

    12. maxking commented on Jul 9, 2020

      @maxking
      maxking
      on Jul 9, 2020
      Contributor

      Closing this since the PRs for the behavior change reported by Mike and BPO-41206 is now merged too.

    13. maxking
      closed this as completedon Jul 9, 2020
    14. maxking
      closed this as completedon Jul 9, 2020

    15. maxking commented on Jul 9, 2020

      @maxking
      maxking
      on Jul 9, 2020
      Contributor

      Thanks Mark!

    16. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.8 (EOL)end of life3.9 (EOL)end of lifetopic-emailtype-bugAn unexpected behavior, bug, or error

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions