Skip to content

during 'make install', pre-existing site-packages residents are recompiled. Twice. #72071

Open
Open
during 'make install', pre-existing site-packages residents are recompiled. Twice.#72071
Labels
buildThe build process and cross-buildextension-modulesC modules in the Modules dirtopic-installation

Description

@MattDMo
MattDMo
mannequin
opened on Aug 28, 2016
BPO 27884
Nosy @bitdancer, @MattDMo

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2016-08-28.14:53:14.003>
labels = ['extension-modules', 'build', 'expert-installation']
title = "during 'make install', pre-existing site-packages residents are recompiled. Twice."
updated_at = <Date 2016-08-28.15:09:22.717>
user = 'https://github.com/MattDMo'

bugs.python.org fields:

activity = <Date 2016-08-28.15:09:22.717>
actor = 'r.david.murray'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Build', 'Extension Modules', 'Installation']
creation = <Date 2016-08-28.14:53:14.003>
creator = 'MattDMo'
dependencies = []
files = []
hgrepos = []
issue_num = 27884
keywords = []
message_count = 2.0
messages = ['273824', '273825']
nosy_count = 2.0
nosy_names = ['r.david.murray', 'MattDMo']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue27884'
versions = ['Python 3.6']

Activity

  1. MattDMo commented on Aug 28, 2016

    @MattDMo
    MattDMomannequin
    on Aug 28, 2016
    MannequinAuthor

    I've been building Python 3 from source on Linux for a long time (at least since 3.2 days), and I honestly can't remember if this has happened before 3.6 or not, which is why I'm only tagging this 3.6.

    Basically, when running 'make install' (after building either static or shared, I haven't been messing around with any other options except install prefix - $HOME) you'll start seeing messages like:

    =====
    Listing '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression'...
    Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/init.py'...
    Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/anova_nistcertified.py'...
    Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/ar_panel.py'...
    Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/example_kernridge.py'...
    Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/gmm.py'...
    Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/kernridgeregress_class.py'...
    Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/ols_anova_original.py'...
    Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/onewaygls.py'...
    Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/penalized.py'...
    Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/predstd.py'...
    Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/runmnl.py'...
    Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/sympy_diff.py'...
    Listing '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/tests'...
    Compiling '/home/mattdmo/lib/python3.6/site-packages/statsmodels/sandbox/regression/tests/init.py'...
    =====

    and on and on and on. I usually don't pay much attention to them, but since I have nothing better to do I just stare idly at them and watch as all my third-party modules in 'site-packages' scroll by in alphabetical order very quickly. For the last few builds of 3.6, though, I've noticed that the modules get gone through twice. I have a pretty large number of modules (341, according to ls | wc -l), so it takes a while to go through them, and they're definitely being repeated.

    Unfortunately, I don't know a whole lot about Makefiles or the internals of the installation process, so I don't have the slightest idea of where to begin even looking for the bug. But, I bet you ladies and gentlemen do :)

    If it's necessary, I can rebuild and pipe all the output into a file for you to examine. Just let me know.

  2. MattDMo
    added
    buildThe build process and cross-build
    extension-modulesC modules in the Modules dir
    topic-installation
    on Aug 28, 2016

  3. bitdancer commented on Aug 28, 2016

    @bitdancer
    bitdancer
    on Aug 28, 2016
    Member

    As far as I can tell, this is working as designed, and in fact you ought to see site-packages getting compiled *three* times (normal, -O, and -OO).

    I wonder if this is a holdover from when we didn't have per-interpreter pyc files, and its just that no one noticed the need for recompile of site-packages didn't exist any more. On the other hand, I may be missing some piece of logic that means that it is still needed.

  4. ezio-melotti
    transferred this issue fromon Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    buildThe build process and cross-buildextension-modulesC modules in the Modules dirtopic-installation

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions