Repository navigation
Alternate RFC 3986 compliant URI parsing module #43453
Description
| BPO | 1500504 |
|---|---|
| Nosy | @ncoghlan, @orsenthil, @devdanzin, @merwok, @ambv, @berkerpeksag, @vadmium |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/orsenthil'
closed_at = None
created_at = <Date 2006-06-04.14:50:18.000>
labels = ['type-feature', 'library']
title = 'Alternate RFC 3986 compliant URI parsing module'
updated_at = <Date 2015-01-01.01:25:35.044>
user = 'https://github.com/ncoghlan'bugs.python.org fields:
activity = <Date 2015-01-01.01:25:35.044>
actor = 'berker.peksag'
assignee = 'orsenthil'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2006-06-04.14:50:18.000>
creator = 'ncoghlan'
dependencies = []
files = ['7307', '32591']
hgrepos = []
issue_num = 1500504
keywords = ['patch']
message_count = 16.0
messages = ['50411', '50412', '50413', '50414', '50415', '83920', '86301', '86308', '86348', '109963', '109966', '120245', '120264', '120344', '120349', '202728']
nosy_count = 7.0
nosy_names = ['ncoghlan', 'orsenthil', 'ajaksu2', 'eric.araujo', 'lukasz.langa', 'berker.peksag', 'martin.panter']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue1500504'
versions = ['Python 3.5']Metadata
Metadata
Assignees
Labels
Projects
- Status
Activity
orsenthil commentedon Jul 11, 2010
A new way for parsing URI. I have not reviewed it even after saying I would like to, but having the dependency issue resolved, I think it is good to look at it again, especially if it leads some helpful approaches to parsing IRI.
ncoghlan commentedon Jul 11, 2010
"accepted" is a little too strong for the current status of this :)
I've removed the easy tag as well (making the case for this or something like it in the standard library it going to involve a fair bit of effort - the coding was actually the comparatively easy part).
merwok commentedon Nov 2, 2010
Is this still relevant? Can’t the improvements make it into urllib.parse?
ncoghlan commentedon Nov 2, 2010
I still like the higher level API concept, although I might not do it exactly as presented here any more.
Independently of introducing a new parsing API, it would be worthwhile extracting the parsing tests from the attached module to make sure the *existing* parser can handle them all correctly.
merwok commentedon Nov 3, 2010
Sure, adding tests is a no-brainer.
Regarding the module, I’m a bit reluctant. I see value in providing low-level building blocks (think OS calls) and high-level utilities for regular use, but here it seems that urllib.parse and urischemes are at the same level. I’m not opposed to the functionality itself—I would like to use a class simply named “URI” (and generally get better names, that is RFC names instead of specific inventions), have components normalization and such goodies—but I think the existing module can get fixes and improvements. I fear the confusion that could be caused by having two modules for the same task, unless you want to propose that the new module deprecate urllib.parse.
Senthil, what is your opinion?
ncoghlan commentedon Nov 3, 2010
Just to be clear, even *I* don't think adding urischemes as it stands is a particularly great idea, and I wrote it. The only reason I haven't closed the issue is because I'd like to see it mined for additional tests in test_urlparse and perhaps even implementation or API enhancements in url.parse first.
(The latter becomes a lot more likely if the urischemes implementation passes tests that url.parse fails)
I also think, since I wrote this, the various urllib parsing methods were updated to return named tuple instances with properties, so a lot of the awkwardness of extracting partial values went away. (i.e. returning structured objects already raised the level of the urllib APIs from the "tuple-of-strings" level they used to be sitting at)
I do still assert that urischemes is slightly "higher level" than the current incarnation of similar functionality in urllib.parse. Universal Resource Identifiers are more encompassing than Universal Resource Locators and Universal Resource Names, and the new APIs explicitly deal with both kinds of URI. There are subtle differences in the assumptions you're allowed to make when you may have a URN rather than a URL, so I believe the current module sometimes does the wrong thing when given one of the former.
That said, it's been a long time since I've needed to remember the details, so I don't recall exactly where the current module gets URI handling wrong (or at least, did back in 2006). The intro to RFC 3986 is a good place to start in learning the differences though - Sir Tim writes good docs :)
akuchling commentedon Nov 13, 2013
Here's a slightly modified version of urischeme.py that can be run under Python 3 and compares its results with urllib.parse, printing out the mismatches.
The major differences seem to be 1) urischeme fills in the default port if it's not explicitly provided, e.g. http urls have the port set to 80, 2) the path is returned as '/', not the empty string, for the URL http://host, 3) urllib.parse.urljoin() doesn't get rid of ./ and ../ in URLs.
soreavis commentedon Jul 19, 2026
The test-mining this was kept open for looks done — I pulled the attachments (uriparse.py, testurlparse.py, akuchling's urischemes port) and ran their 129 urljoin cases against current main. Results:
http:gcase.uses_relativescheme gate (bpo-22118, 2014), and the empty-segment collapse (bpo-22278, 2014)..//../— since fixed by the RFC 3986 switch. The other two are deliberate:urllib.parsedoesn't fill scheme-default ports, and returns an empty path (not/) for an authority-only URL.With the module itself set aside in 2010, this looks closable as not planned.