Skip to content

2.7: wrong user site directory on Linux #53005

Closed
Closed
2.7: wrong user site directory on Linux#53005
Assignees
tarekziade
Labels
stdlibStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or error

Description

@srid
srid
mannequin
opened on May 18, 2010
BPO 8759
Nosy @ronaldoussoren, @tarekziade, @merwok

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = 'https://github.com/tarekziade'
closed_at = <Date 2010-05-21.01:58:20.381>
created_at = <Date 2010-05-18.21:57:43.065>
labels = ['type-bug', 'library']
title = '2.7: wrong user site directory on Linux'
updated_at = <Date 2010-08-27.01:21:46.150>
user = 'https://bugs.python.org/srid'

bugs.python.org fields:

activity = <Date 2010-08-27.01:21:46.150>
actor = 'eric.araujo'
assignee = 'tarek'
closed = True
closed_date = <Date 2010-05-21.01:58:20.381>
closer = 'r.david.murray'
components = ['Library (Lib)']
creation = <Date 2010-05-18.21:57:43.065>
creator = 'srid'
dependencies = []
files = []
hgrepos = []
issue_num = 8759
keywords = []
message_count = 10.0
messages = ['106009', '106044', '106057', '106106', '106108', '106109', '106110', '106133', '106187', '115050']
nosy_count = 4.0
nosy_names = ['ronaldoussoren', 'tarek', 'eric.araujo', 'srid']
pr_nums = []
priority = 'high'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue8759'
versions = ['Python 2.7']

Activity

  1. srid commented on May 18, 2010

    @srid
    sridmannequin
    on May 18, 2010
    MannequinAuthor
    sridharr@triple:~$ /opt/ActivePython-2.7/bin/python -c "import sys; print sys.path; import cmdln"
['', '/opt/ActivePython-2.7/lib/python27.zip', '/opt/ActivePython-2.7/lib/python2.7', '/opt/ActivePython-2.7/lib/python2.7/plat-linux2', '/opt/ActivePython-2.7/lib/python2.7/lib-tk', '/opt/ActivePython-2.7/lib/python2.7/lib-old', '/opt/ActivePython-2.7/lib/python2.7/lib-dynload', '/home/sridharr/.local/lib/python/2.7/site-packages', '/opt/ActivePython-2.7/lib/python2.7/site-packages']
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named cmdln
sridharr@triple:~$ file /home/sridharr/.local/lib/python2.7/site-packages/cmdln.py 
/home/sridharr/.local/lib/python2.7/site-packages/cmdln.py: a python script text executable

    On OSX, I don't even see a ~/.local directory!

    python2.7 -c "import sys; print sys.path"
    ['', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages']

    Works as expected on Windows, though.

  2. srid
    added
    stdlibStandard Library Python modules in the Lib/ directory
    type-bugAn unexpected behavior, bug, or error
    on May 18, 2010
  3. pitrou
    assigned
    tarekziade
    on May 19, 2010

  4. tarekziade commented on May 19, 2010

    @tarekziade
    tarekziademannequin
    on May 19, 2010
    Mannequin

    Are you sure you didn't deactivate use site ? (by turning site.ENABLE_USER_SITE off)

    Please check its value:

      >>> import site
  >>> site.ENABLE_USER_SITE 
  True

    Pathes look fine on my Mac :

    Python 2.7b2+ (trunk:81228, May 16 2010, 14:42:34) 
[GCC 4.0.1 (Apple Inc. build 5490)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/local/lib/python2.7/site-packages/pip-0.6.1-py2.7.egg', '/usr/local/lib/python2.7/site-packages/distribute-0.6.11dev-py2.7.egg', '/usr/local/lib/python27.zip', '/MacDev/svn.python.org/python-trunk/Lib', '/MacDev/svn.python.org/python-trunk/Lib/plat-darwin', '/MacDev/svn.python.org/python-trunk/Lib/plat-mac', '/MacDev/svn.python.org/python-trunk/Lib/plat-mac/lib-scriptpackages', '/MacDev/svn.python.org/python-trunk/Lib/lib-tk', '/MacDev/svn.python.org/python-trunk/Lib/lib-old', '/MacDev/svn.python.org/python-trunk/build/lib.macosx-10.4-i386-2.7', '/Users/tarek/.local/lib/python/2.7/site-packages', '/usr/local/lib/python2.7/site-packages']

    If you still have the issue, you need to trace python load in site.py, where all the paths are added, by printing them out for example, and using the -v flag

  5. ronaldoussoren commented on May 19, 2010

    @ronaldoussoren
    ronaldoussoren
    on May 19, 2010
    Contributor

    The --user directory for framework installs of python on OSX has changed from a subdirectory of ~/.local to a subdirectory ~/Library. As described in the NEWS file:

    • Issue bpo-8084: PEP-370 now conforms to system conventions for framework
      builds on MacOS X. That is, "python setup.py install --user" will install
      into "~/Library/Python/2.7" instead of "~/.local".

    In python 2.6 both ~/.local and ~/Library/Python were added to sys path and that's confusing. As ~/.local does not conform to the filesystem layout conventions on OSX the --user directory now always refers to ~/Library/Python.

    For classic unix installs --user still uses the unix conventions. I'm not 100% sure that that is the right choice.

    In other words: works as designed.

  6. srid commented on May 19, 2010

    @srid
    sridmannequin
    on May 19, 2010
    MannequinAuthor

    @ronald: Ah, I see. So on Mac, Python 2.7 (with default site configuration) will not pickup packages installed in ~/.local, correct? It will, from now onwards, only consider ~/Library/Package as the user site directory? I ask because I just want to double check before modifying PyPM (which has ~/.local hardcoded in the client) to support this change.

    @tarek: I have user site enabled; didn't change anything .. just the default settings:
    $ /tmp/apy27/bin/python -m site
    sys.path = [
    '/home/sridharr/as/apy/trunk',
    '/tmp/apy27/lib/python27.zip',
    '/tmp/apy27/lib/python2.7',
    '/tmp/apy27/lib/python2.7/plat-linux2',
    '/tmp/apy27/lib/python2.7/lib-tk',
    '/tmp/apy27/lib/python2.7/lib-old',
    '/tmp/apy27/lib/python2.7/lib-dynload',
    '/home/sridharr/.local/lib/python/2.7/site-packages',
    '/tmp/apy27/lib/python2.7/site-packages',
    '/tmp/apy27/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg-info',
    ]
    USER_BASE: '/home/sridharr/.local' (exists)
    USER_SITE: '/home/sridharr/.local/lib/python/2.7/site-packages' (exists)
    ENABLE_USER_SITE: True

    ...

    That is strange user site location - "python/2.7/site-packages" really? Shouldn't it be "python2.7/site-packages"?

  7. srid
    changed the title [-]2.7: wrong user site directory on Linux; totally missing on OSX[/-] [+]2.7: wrong user site directory on Linux[/+] on May 19, 2010

  8. srid commented on May 19, 2010

    @srid
    sridmannequin
    on May 19, 2010
    MannequinAuthor

    On further analysis:

    $ /tmp/apy27/bin/python -c "from sysconfig import get_path; print get_path('purelib', 'posix_user')"
/home/sridharr/.local/lib/python/2.7/site-packages
$

    From sysconfig.py:

    'posix_user': {
    'stdlib': '{userbase}/lib/python/{py_version_short}',
    'platstdlib': '{userbase}/lib/python/{py_version_short}',
    'purelib': '{userbase}/lib/python/{py_version_short}/site-packages',
    'platlib': '{userbase}/lib/python/{py_version_short}/site-packages',

    Ah, there is the bug!

  9. tarekziade commented on May 19, 2010

    @tarekziade
    tarekziademannequin
    on May 19, 2010
    Mannequin

    "python/2.7/site-packages" is a typo in sysconfig, it should be "python2.7/site-packages" you are right.

    Fixing it..

  10. tarekziade commented on May 19, 2010

    @tarekziade
    tarekziademannequin
    on May 19, 2010
    Mannequin

    Fixed in r81371, r81372

    Thanks!

    I let you close the issue once you are through with the rest of the talk.

  11. ronaldoussoren commented on May 20, 2010

    @ronaldoussoren
    ronaldoussoren
    on May 20, 2010
    Contributor

    Shridar: that is correct a framework install will not look in ~/.local, but only in ~/Library/Python/2.7 (there is an install scheme in distutils that describes the exact layout).

    Technically the name "Python" subdirectory is sysconfig.get_config_var("PYTHONFRAMEWORK"), to ensure that frameworks with different values for --with-framework-name will use different subdirectories.

  12. srid commented on May 20, 2010

    @srid
    sridmannequin
    on May 20, 2010
    MannequinAuthor

    Thanks.

    there is an install scheme in distutils that describes the exact layout

    I figured that this is now exposed via the sysconfig module. There is also an old install scheme in distutils.command.install, but it seems to be outdated and is only applicable for 2.6.

    My understanding is that, installers such as PyPM should use the scheme in distutils.command.install on 2.6 and the new sysconfig scheme for 2.7+.

    Please feel free to close this issue (I can do it, but do not have the privileges to set the resolution field.)

  13. bitdancer
    added
    invalid
    on May 21, 2010
  14. bitdancer
    closed this as completedon May 21, 2010

  15. merwok commented on Aug 27, 2010

    @merwok
    merwok
    on Aug 27, 2010
    Member

    Fixed in the distutils2 backport in 7f0148944e28 (not pushed yet due to SSH problem with Bitbucket).

  16. merwok
    removed
    invalid
    on Aug 27, 2010
  17. ezio-melotti
    transferred this issue fromon Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

stdlibStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or error

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions