Repository navigation
Use directories from configure rather than hardcoded #39699
Description
| BPO | 858809 |
|---|---|
| Nosy | @loewis, @nijel, @vstinner, @matejcik, @tarekziade, @merwok, @agbuckley |
| Superseder | |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/tarekziade'
closed_at = <Date 2020-02-12.00:53:00.786>
created_at = <Date 2003-12-12.10:24:30.000>
labels = ['type-bug', 'library', 'build']
title = 'Use directories from configure rather than hardcoded'
updated_at = <Date 2020-02-12.00:53:00.783>
user = 'https://github.com/nijel'bugs.python.org fields:
activity = <Date 2020-02-12.00:53:00.783>
actor = 'vstinner'
assignee = 'tarek'
closed = True
closed_date = <Date 2020-02-12.00:53:00.786>
closer = 'vstinner'
components = ['Build', 'Distutils']
creation = <Date 2003-12-12.10:24:30.000>
creator = 'nijel'
dependencies = []
files = ['5739']
hgrepos = []
issue_num = 858809
keywords = ['patch']
message_count = 20.0
messages = ['45042', '45043', '45045', '45046', '65368', '73812', '76707', '83076', '83077', '87857', '87858', '88065', '88067', '112209', '114578', '114580', '114581', '114585', '114715', '361843']
nosy_count = 13.0
nosy_names = ['loewis', 'nijel', 'spektrum', 'vstinner', 'matejcik', 'tarek', 'sergiodj', 'eric.araujo', 'rpetrov', 'Arfrever', 'rothrock', 'andybuckley', 'soundmurderer']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '1294959'
type = 'behavior'
url = 'https://bugs.python.org/issue858809'
versions = ['Python 3.2']
Activity
nijel commentedon Dec 12, 2003
Current Makefile.pre.in contains paths like
$(exec_prefix)/lib, which bad especially for mixed
32-bit and 64-bit systems, where 64-bit things should
go to /lib64. Attached patch changes these paths to
@libdir@ (and does also simmilar changes for other
paths), which is usually outputed from configure.
spektrum commentedon Sep 11, 2006
Logged In: YES
user_id=631694
this patch confuses the python 2.5 build process in a very
mysterious way - setup.py ends up using
-L/usr/lib/python2.5/config instead of -L. for
libpython2.5.so, which is obviously wrong.
nijel commentedon Sep 11, 2006
Logged In: YES
user_id=192186
That doesn't surprise me, it was written in times of python
2.3...
spektrum commentedon Sep 11, 2006
Logged In: YES
user_id=631694
well obviously, but if you look at the patch, there is
exactly -nothing- wrong with it.
actually, i'm surprised that it didn't get into 2.4 release.
but what's more, it should have no effect on setup.py at all
sergiodj commentedon Apr 11, 2008
Hi,
Continuing with my effort to improve Python's build system, I'd really
like to know why this issue has not been solved yet. I mean, apparently
this problem is still present in Python 2.5, since I can't change the
library's path with --libdir configure's flag. Any news about it?
rothrock commentedon Sep 25, 2008
Hi,
This problem still exists in 2.5.2. Setting the libdir argument doesn't
correctly set LIBDIR in the Makefile.
[rothrock@su1 Python-2.5.2]$ ./configure --prefix=/home/y
--libdir=/home/y/lib64
...
[rothrock@su1 Python-2.5.2]$ cat Makefile | grep LIBDIR=
LIBDIR= $(exec_prefix)/lib
rpetrov commentedon Dec 1, 2008
The complete patch require redesign of python-distutils ;)
matejcik commentedon Mar 3, 2009
well, seeing as redesign of distutils probably won't happen anytime soon...
why don't we fix the obvious bug (broken configure) now and redesign
distutils later?
matejcik commentedon Mar 3, 2009
...the patch is absolutely trivial, doesn't have any deep consequences
and applies cleanly to head and all branches i know of.
...i wonder if the patch-fasttracking offer still applies...
soundmurderer commentedon May 16, 2009
I can confirm that --bindir and --libdir are broken in Python 2.6.2,
despite the fact that ./configure --help lists them as helpful options.
At the very least, ./configure should complain if you're using options
that don't do anything! That would be the easy fix.
Here is what I am doing:
./configure \
--prefix=/home/soundmurderer \
--bindir=/home/soundmurderer/bin/x86_64/Python-2.6.2 \
--libdir=/home/soundmurderer/lib/x86_64
OK, now let's see the relevant section of the generated Makefile:
# Install prefix for architecture-independent files
prefix= /home/soundmurderer/auzilov
# Install prefix for architecture-dependent files
exec_prefix= ${prefix}
# Install prefix for data files
datarootdir= ${prefix}/share
# Expanded directories
BINDIR= $(exec_prefix)/bin
LIBDIR= $(exec_prefix)/lib
MANDIR= ${datarootdir}/man
INCLUDEDIR= ${prefix}/include
CONFINCLUDEDIR= $(exec_prefix)/include
SCRIPTDIR= $(prefix)/lib
# Detailed destination directories$(LIBDIR)/python$ (VERSION)$(SCRIPTDIR)/python$ (VERSION)$(INCLUDEDIR)/python$ (VERSION)$(CONFINCLUDEDIR)/python$ (VERSION)$(LIBDIR)/python$ (VERSION)
BINLIBDEST=
LIBDEST=
INCLUDEPY=
CONFINCLUDEPY=
LIBP=
So, the --libdir and --bindir flags had no effect on LIBDIR and BINDIR,
and I wind up editing the Makefile by hand to get desired behavior.
soundmurderer commentedon May 16, 2009
I tried applying nijel's 2.3.2 patch to Python 2.6.2 Makefile.pre.in,
then configure/make/install with --bindir and --libdir flags to
./configure. It works in terms of producing the correct Makefile with
LIBDIR and BINDIR that I want, but I get problems after the "make
install" step when I launch the Python interpreter. This is what happens:
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
'import site' failed; use -v for traceback
Python 2.6.2 (r262:71600, May 15 2009, 22:46:19)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-44)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
I can work around bad imports and "could not find libraries" by adding
various subdirs of my desired LIBDIR to PYTHONHOME and PYTHONPATH. But
the point is -- I thought that a *properly* installed Python
distribution should know where to find its own standard libraries?? I
should not have to hack PYTHONHOME and PYTHONPATH to get my local
install to work... right?
Once upon a time in 2.5.x days, I was able to do a local install using
only ./configure --prefix, and that's it. No hacking PYTHONHOME or
PYTHONPATH that I recall.
How is the path to stuff like site.py getting encoded in the Python
installation? Because I suspect that although nijel's patch may work
for generating makefiles that put binaries/libs in the right place, that
encoding step is still getting botched, requiring to manually set
PYTHONHOME and PYTHONPATH as a workaround.
Or maybe I am just confused.
soundmurderer commentedon May 19, 2009
OK, I am now convinced that even with this patch, the Python build
process still won't work correctly because --libdir (post-patch) will
not correctly set the default PYTHONHOME. Moreover, the very way that
PYTHONHOME gets used needs to change. PYTHONHOME specifies "prefix"
and/or "exec_prefix", but the problem is that to find the lib dir,
Python tacks "lib/pythonX.X" onto the ends of these paths! More info here:
http://docs.python.org/using/unix.html#python-related-paths-and-files
http://docs.python.org/using/cmdline.html#envvar-PYTHONHOME
So, even with this patch, I still can't install, for example, to:
/home/soundmurderer/lib/x86_64/python2.6
I am forced to install to a location that ends with "lib/python2.6", e.g.:
/home/soundmurderer/x86_64/lib/python2.6
or else hack some stuff together with symlinks to make PYTHONHOME go to
the right place.
This seems to be a problem more fundamental than just incomplete
"./configure" in the build process and I think it warrants another
ticket, which I'm about to open.
Now, in regards to --libdir, even though the patch makes the
installation put the files in the expected location, Python will
probably not find it.
soundmurderer commentedon May 19, 2009
OK done, opened a feature request for this pet peeve of mine. ;-) Maybe
someday I'll complain about something substantial...
See: http://bugs.python.org/issue6060
9 remaining items
merwok commentedon Jul 31, 2010
FYI, disutils is being improved as a new package named distutils2. It is not a complete redesign, but we break backward compatibility when required, so there’s hope.
merwok commentedon Aug 21, 2010
About lib vs. lib64 vs. lib32, see also http://bugs.python.org/issue1294959#msg94941
merwok commentedon Aug 21, 2010
I don’t know much about Makefile.pre.in, so I can’t say anything about the patch. Martin, could you maybe comment?
merwok commentedon Aug 21, 2010
From http://bugs.python.org/issue1294959#msg94986
“The main problem is that Python's configuration system is not
geared up to having the lib directories for platform dependent
and platform independent parts use different names.
It currently only supports using different path *prefixes* for
such setups (--prefix and --exec-prefix), e.g. /usr and /usr64
would work just fine. It doesn't follow --libdir.”
So I don’t know whether there can be a simple fix for this or if we have to wait for the new sysconfig module (see http://bitbucket.org/tarek/distutils2/src/tip/docs/design/wiki.rst)
loewis commentedon Aug 21, 2010
I don't think anything in distutils can solve whatever the problem is that people perceive.
All I can say about the patch is that it apparently doesn't do any harm. I expect that, if it is accepted, people continue complain that "it doesn't work". Unfortunately, Michal Čihař never indicated what *specific* problem this patch is supposed to solve, so it is difficult to tell whether it actually achieves that objective (the literal description "64-bit things should go to /lib64" is not achieved by the patch - it never mentions "lib64".
matejcik commentedon Aug 22, 2010
the problem this patch was solving is "python libraries install into "prefix/lib" regardless of what is the correct local LIBDIR" - which is obviously broken on systems where LIBDIR is something other than "prefix/lib", most notable example being "prefix/lib64" on current 64bit linuxes.
but as far i can tell, this problem was fixed (to an extent) in 2.7 by partially applying this patch.
SCRIPTDIR is still prefix/lib, but the correctness of that is up for a long and winded debate.
vstinner commentedon Feb 12, 2020
Start of this issue:
"Current Makefile.pre.in contains paths like $(exec_prefix)/lib, which bad especially for mixed 32-bit and 64-bit systems, where 64-bit things should go to /lib64."
That's a duplicate of bpo-1294959. This issue discuss also modifying distutils but there is no clear plan on how it should behave or not. My PR 18381 of bpo-1294959 updates sysconfig and distutils to support $prefix/$python_libdir/ path.