Repository navigation
smtplib: support for IDN (international domain names) #64282
Description
| BPO | 20083 |
|---|---|
| Nosy | @warsaw, @macfreek, @bitdancer, @zvyn |
| Dependencies |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2013-12-28.01:49:23.270>
labels = ['type-feature', 'library', 'expert-email']
title = 'smtplib: support for IDN (international domain names)'
updated_at = <Date 2014-06-12.20:23:23.410>
user = 'https://github.com/macfreek'bugs.python.org fields:
activity = <Date 2014-06-12.20:23:23.410>
actor = 'zvyn'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)', 'email']
creation = <Date 2013-12-28.01:49:23.270>
creator = 'macfreek'
dependencies = ['11783']
files = []
hgrepos = []
issue_num = 20083
keywords = []
message_count = 7.0
messages = ['207017', '207019', '207041', '207044', '207045', '207053', '207064']
nosy_count = 5.0
nosy_names = ['barry', 'macfreek', 'r.david.murray', 'jesstess', 'zvyn']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue20083'
versions = ['Python 3.5']
Activity
macfreek commentedon Dec 28, 2013
smtplib has limited support for non-ASCII domain names in the From to To mail address. It only works for punycode-encoded domain names, submitted as unicode string (e.g. server.rcpt(u"user@xn--e1afmkfd.ru").
The following two calls fail:
server.rcpt(u"user@пример.ru"):
File smtplib.py, line 332, in send
s = s.encode("ascii")
UnicodeEncodeError: 'ascii' codec can't encode character '\u03c0' in position 19: ordinal not in range(128)
http://hg.python.org/cpython/file/3.3/Lib/smtplib.py#l332
server.rcpt(b"user@xn--e1afmkfd.ru"):
File email/_parseaddr.py, line 236, in gotonext
if self.field[self.pos] in self.LWS + '\n\r':
TypeError: 'in <string>' requires string as left operand, not int
http://hg.python.org/cpython/file/3.3/Lib/email/_parseaddr.py#l236
There are three ways to solve this (from trivial to complex):
See also
References:
https://tools.ietf.org/html/rfc5891: Internationalized Domain Names in Applications (IDNA): Protocol
macfreek commentedon Dec 28, 2013
This issue deals with international domain names in email addresses (the part behind the "@"). See bpo-20084 for the issue that deals with the part before the "@".
bitdancer commentedon Dec 28, 2013
Thanks for the suggestion.
Once the bpo-11783 patch is committed, smtplib can be changed to use formataddr in quoteaddr, which will result in the domain being punycoded automatically. (It's too bad I forgot about that issue, since the 3.4 beta deadline has already passed :(
The input to the commands is string, not bytes, so you can already pre-encode yourself, as you noted. The commands don't accept bytes, and should not, since the data they cause to be sent on the wire may not contain non-ASCII characters; there is thus no need to generate binary. SMTPUTF8 will of course require generating binary data in these contexts, but in that case the correct way to generate the binary is by utf-8 encoding the unicode input, so there will again be no reason for the commands to accept binary input, and it will be better if they don't. (If you need to generate invalid data, say for a test scenario, you can drop down to executing 'send' calls manually.)
(Note: using the 'u' prefix in python3, while supported for backward compatibility, is only confusing when used outside of that context...I thought you were talking about 2.7 until I read carefully.)
macfreek commentedon Dec 28, 2013
Great to hear that a patch already exists (sorry I couldn't find in in the tracker).
Feel free to close this issue as duplicate of bpo-11783.
(As for the u"string", I wanted to distinguish it from b'string'. I don't use it in code (since the backward compatibility is only present in 3.3+, not in 3.2). Sorry for the confusion.)
bitdancer commentedon Dec 28, 2013
No, that issue is about the email library. So we need this one too for the equivalent enhancement to smtplib.
macfreek commentedon Dec 28, 2013
Since smtplib.quoteaddr() uses email.utils.parseaddr(), and the patch for bpo-11783 fixes email.utils.parseaddr(), that patch will hopefully solve this issue as well (though a test case wouldn't hurt for sure).
What I had not realised is that hostnames are also used elsewhere, in particular in the ehlo() and helo() but also in connect(). Do you consider that a separate issue or part of this issue?
Are there other places where you think a fix is needed?
I may be able to create a patch, though bear with me: I never checked out the source for Python or the standard library (other than installing point releases through my package manager).
bitdancer commentedon Dec 29, 2013
A call to formataddr will need to be added to quoteaddr. And yes, test cases are needed.
I don't believe that the format of the HELO/EHLO message is defined by the RFC, so I don't think we can automatically parse it. I think we just have to leave the domain name encoded as punycode there. Regardless, though, yes I would consider that a separate issue.
If you want to work on a patch, that would be great. For guidance on doing so, you can take a look at http://docs.python.org/devguide.
You can also help me to remember to commit 11783 after the final release of 3.4.0.