Regression in compileall ddir parameter when recursing #82896

Activity

1. hetman commented on Nov 6, 2019

   

   hetmanmannequin

   on Nov 6, 2019

   MannequinAuthor

   More actions

   There appears to have been a regression introduced in the compileall library back with Python 3.5 as a result of commit f1a8df0 .

   The issues appears when compiling a hierarchy of directories with the 'ddir' parameter specified (either in the compile_dir() function or via the '-d' command line parameter).

   Examining the code before the f1a8df0 commit suggests the original intent for the 'ddir' parameter, when recursing into subdirectories, was to alter the root of the path displayed in exception traces. That means that the hierarchical directory structure of the reported source path in each compiled file was maintained, and only the root directory being compiled was replaced with ddir.

   More recently two new parameters, 'stripdir' and 'prependdir', have been added with commit 8e7bb99 which seem to largely reimplement the missing behaviour in a different way.

   The current behaviour does not seem very useful or expected. I think it would be helpful to restore the original behaviour and would be happy to contribute a PR to do so. Additionally it may be worth consulting the submitter of bpo-38112 whether the alternative arguments are still necessary once the regression is fixed, though this is optional and not needed to fix the core issue.

   Here's a simple script to demonstrate the above:

   mkdir -p library/a/a

   cat <<'EOF' > library/init.py
   def go():
   raise AssertionError('dummy')
   EOF
   cp library/init.py library/a/init.py
   cp library/init.py library/a/a/init.py

   python3 -m compileall -b -d /usr/lib library
   rm library/**/*.py

   python3 -c 'from library import go; go()'
   python3 -c 'from library.a import go; go()'
   python3 -c 'from library.a.a import go; go()'

   And the resulting output which shows that each module is claiming to be raised from the same location:

   Listing 'library'...
   Compiling 'library/init.py'...
   Listing 'library/a'...
   Compiling 'library/a/init.py'...
   Listing 'library/a/a'...
   Compiling 'library/a/a/init.py'...

   Traceback (most recent call last):
     File "<string>", line 1, in <module>
     File "/usr/lib/__init__.py", line 2, in go
   AssertionError: dummy
   
   Traceback (most recent call last):
     File "<string>", line 1, in <module>
     File "/usr/lib/__init__.py", line 2, in go
   AssertionError: dummy
   
   Traceback (most recent call last):
     File "<string>", line 1, in <module>
     File "/usr/lib/__init__.py", line 2, in go
   AssertionError: dummy
2. hetman

   added

   3.7 (EOL)end of lifeend of life

   3.8 (EOL)end of lifeend of life

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Nov 6, 2019
3. FFY00 commented on May 3, 2020

   

   FFY00

   on May 3, 2020

   Member

   More actions

   Can this be closed? I can't reproduce.

   $ python -c 'from library.a.a import go; go()'
   Traceback (most recent call last):
     File "<string>", line 1, in <module>
     File "/usr/lib/a/a/__init__.py", line 2, in go
       raise AssertionError('dummy')
   AssertionError: dummy
   $ python -c 'from library.a import go; go()'
   Traceback (most recent call last):
     File "<string>", line 1, in <module>
     File "/usr/lib/a/__init__.py", line 2, in go
       raise AssertionError('dummy')
   AssertionError: dummy
   $ python -c 'from library import go; go()'
   Traceback (most recent call last):
     File "<string>", line 1, in <module>
     File "/usr/lib/__init__.py", line 2, in go
       raise AssertionError('dummy')
   AssertionError: dummy
4. FFY00 commented on May 3, 2020

   

   FFY00

   on May 3, 2020

   Member

   More actions

   Nevermind! I ran it with my system python binary, which is patched to against this.
5. ezio-melotti

   transferred this issue fromon Apr 10, 2022
6. hetmankp commented on Oct 4, 2022

   

   hetmankp

   on Oct 4, 2022

   Contributor

   More actions

   I believe this issue was identified again in #83950 and resolved with #18676, therefore this report can now be closed.