Repository navigation
new os.path function to extract common prefix based on path components #54604
Description
| BPO | 10395 |
|---|---|
| Nosy | @loewis, @rhettinger, @pfmoore, @ronaldoussoren, @ericvsmith, @ezio-melotti, @merwok, @bitdancer, @ericsnowcurrently, @Fak3, @serhiy-storchaka |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2015-04-02.12:42:22.211>
created_at = <Date 2010-11-12.15:14:04.644>
labels = ['type-feature', 'library']
title = 'new os.path function to extract common prefix based on path components'
updated_at = <Date 2015-04-02.12:42:22.210>
user = 'https://github.com/ronaldoussoren'bugs.python.org fields:
activity = <Date 2015-04-02.12:42:22.210>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2015-04-02.12:42:22.211>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2010-11-12.15:14:04.644>
creator = 'ronaldoussoren'
dependencies = []
files = ['27883', '27899', '27974', '35942']
hgrepos = []
issue_num = 10395
keywords = ['patch']
message_count = 18.0
messages = ['121038', '121039', '121040', '121043', '121063', '121106', '141535', '174663', '174818', '174819', '174941', '175493', '222966', '222986', '238766', '239675', '239691', '239694']
nosy_count = 15.0
nosy_names = ['loewis', 'rhettinger', 'paul.moore', 'ronaldoussoren', 'eric.smith', 'ezio.melotti', 'eric.araujo', 'r.david.murray', 'santoso.wijaya', 'python-dev', 'eric.snow', 'Roman.Evstifeev', 'serhiy.storchaka', 'rafik', 'Paddy McCarthy']
pr_nums = []
priority = 'low'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue10395'
versions = ['Python 3.5']
Activity
ronaldoussoren commentedon Nov 12, 2010
The documentation for os.path.commonprefix notes:
And indeed:
This is IMHO useless behaviour for a function in the os.path namespace, I'd expect that os.path.commonprefix works with path elements (e.g. that the call above would have returned '/usr').
ericvsmith commentedon Nov 12, 2010
Indeed, that behavior seems completely useless.
I've verified that it works the same in 2.5.1.
ericvsmith commentedon Nov 12, 2010
Although there are test cases in test_genericpath that verify this behavior, so apparently it's intentional.
ronaldoussoren commentedon Nov 12, 2010
That's why I write 'broken by design' in the title.
A "fix" for this will have to a new function, if any get added (I've written a unix implementation that finds the longest shared path several times and can provide an implementation and tests when others agree that this would be useful)
loewis commentedon Nov 12, 2010
This goes back to bpo-400788 and
http://mail.python.org/pipermail/python-dev/2000-July/005897.html
http://mail.python.org/pipermail/python-dev/2000-August/008385.html
Skip changed it to do something meaningful (more than ten years ago), Mark Hammond complained that it was backwards incompatible, Tim Peters argued that you shouldn't change a function if the documented behavior matches the implementation, and Skip reverted the change and added more documentation to make the actual behavior more explicit.
It may be useless, but it's certainly not broken. In addition, it's very likely that applications of it rely on the very semantics that it has.
In any case, anybody proposing a change should go back and re-read the old threads.
bitdancer commentedon Nov 13, 2010
Indeed, as I remember it there are people using commonprefix as a string function in situations having nothing to do with os paths.
I'm changing the title to reflect the fact that this is really a feature request for a new function. IMO it is a reasonable feature request. Finding a name for it ought to be an interesting exercise.
I think that this should only be accepted if there is also a windows implementation.
[-]os.path.commonprefix broken by design[/-][+]new os.path function to extract common prefix based on path components[/+]ericsnowcurrently commentedon Aug 1, 2011
You can already get the better prefix using os.path, albeit less efficiently. Here's an example:
However, would it be better to implicitly normalize paths first rather than doing a character-by-character comparison? Here is an unoptimized demonstration of what I mean:
merwok commentedon Nov 3, 2012
Rafik is working on os.path.commonpath for the bug day.
rafik commentedon Nov 4, 2012
Here is a patch with an implementation of os.path.commonpath, along with tests and documentation. At the moment, this is only implemented for POSIX, as I don't feel like I know enough about Windows to tackle drive letters and UNC in paths without spending some more time on it.
This probably needs more tests for corner cases.
serhiy-storchaka commentedon Nov 4, 2012
Just use splitdrive() and first ensure that all drivespecs are same, then find common prefix for pathspecs.
rafik commentedon Nov 5, 2012
Here is a new patch addressing some of storchaka review comments, and implementing a version in ntpath.
For the Windows version, I did as proposed in msg174819, but as I am not familiar with the semantics and subtleties of paths in Windows maybe this version of ntpath.commonpath is too simplistic and would return wrong results in some cases. I would like someone more knowledgeable in Windows to take care of it, or maybe just provide a test suite with lots of different corner cases that I could use to provide a better implementation.
serhiy-storchaka commentedon Nov 13, 2012
Some conclusions of discussion at Python-ideas (http://comments.gmane.org/gmane.comp.python.ideas/17719):
commonpath() should eat double slashes in input (['/usr/bin', '/usr//bin'] -> '/usr/bin'). In any case the current implementation eats slashes on output (['/usr//bin', '/usr//bin'] -> '/usr/bin', not '/usr//bin').
commonpath() should raise an exception instead of returning None on incompatible input.
May be commonpath() should eat also '.' components and return '.' instead of '' when relative paths have no common prefix. I am not sure.
In general the current patch looks good enough.
serhiy-storchaka commentedon Jul 13, 2014
Here is revised patch. The behavior is changed in correspondence with results of Python-ideas discussion, extended tests, fixed several bugs.
rhettinger commentedon Jul 14, 2014
This patch looks reasonable except for the doc change to os.path.commonprefix(). Remember, that function IS working as documented and that our policy is to document in an affirmative manner (here is what the function does and how to use it versus being preachy about "broken-by-design" etc.)
PaddyMcCarthy commentedon Mar 21, 2015
Can we now:
This seems to have lingered for too long and yet people have been willing to do the work it seems (from 1999).
pfmoore commentedon Mar 31, 2015
The patch looks good to me.
rhettinger: I'm not sure I see a problem with the doc changes in the latest patch - noting that commonprefix may return an invalid path is fine, and what the current docs say. Directing people to commonpath if they don't want invalid paths also seems fine.
Paddy McCarthy: I don't think that the backward compatibility cost of moving os.path.commonprefix is worth it.
serhiy-storchaka commentedon Mar 31, 2015
The patch only adds a reference to commonpath() in commonprefix() documentation. The note about invalid paths already was here.
python-dev commentedon Mar 31, 2015
New changeset ec6c812fbc1f by Serhiy Storchaka in branch 'default':
Issue bpo-10395: Added os.path.commonpath(). Implemented in posixpath and ntpath.
https://hg.python.org/cpython/rev/ec6c812fbc1f