Skip to content

bpo-45020: Freeze the modules imported during startup. #28107

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ericsnowcurrently wants to merge 61 commits into from
Closed

bpo-45020: Freeze the modules imported during startup. #28107

Changes from 1 commit
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
a8bd3fc
Fix freeze_module() in freeze_modules.py.
ericsnowcurrently Sep 2, 2021
6555ae4
Add FrozenSource and FrozenModule to freeze_modules.py.
ericsnowcurrently Sep 1, 2021
3a63f92
Leave all non-required frozen modules uncommitted.
ericsnowcurrently Sep 2, 2021
0587328
Leave *all* frozen modules uncommitted.
ericsnowcurrently Sep 2, 2021
9881456
Generate the list of frozen modules used by test_ctypes.
ericsnowcurrently Sep 1, 2021
3df3e23
Freeze the stdlib modules loaded during startup.
ericsnowcurrently Aug 12, 2021
bfd733c
Add the -X frozen_modules CLI option and PyConfig.use_frozen_modules.
ericsnowcurrently Aug 31, 2021
ca952fc
Ignore frozen modules depending on "-X frozen_modules".
ericsnowcurrently Aug 31, 2021
b4ca34c
Explicitly control PyConfig.use_frozen_modules during tests.
ericsnowcurrently Sep 1, 2021
ef5f3f7
Remember the stdlib dir during startup (adding PyConfig.stdlib_dir).
ericsnowcurrently Jun 12, 2021
4b630b8
Add _Py_GetMainConfig().
ericsnowcurrently Aug 31, 2021
dd1a498
Let _Py_GetStdlibDir() fall back to a config.
ericsnowcurrently Aug 31, 2021
af07489
Add _PyConfig_InitImportConfig().
ericsnowcurrently Aug 31, 2021
4e4baeb
Identify whether or not the executable is running installed (adding _…
ericsnowcurrently Aug 31, 2021
344a5ec
Default to "off" if in development.
ericsnowcurrently Aug 31, 2021
8710d80
Remember if built with --enable-optimizations (adding the _Py_OPT mac…
ericsnowcurrently Aug 31, 2021
77cf801
Default to "on" for PGO builds.
ericsnowcurrently Aug 31, 2021
2702554
Add a NEWS entry.
ericsnowcurrently Aug 31, 2021
becae8f
Fix test_embed for PyConfig.stdlib_dir.
ericsnowcurrently Sep 7, 2021
7f0c8eb
In _Py_GetStdlibDir(), treat "" as though it were NULL.
ericsnowcurrently Sep 7, 2021
d07f63c
Fix test_embed for "Default to "off" if in development.".
ericsnowcurrently Sep 7, 2021
4787a70
Treat __main__ as an essential frozen module.
ericsnowcurrently Sep 7, 2021
a75b808
Fix test_cmd_line_script for "Default to "off" if in development.".
ericsnowcurrently Sep 7, 2021
2050793
Ignore frozen submodules in generate_stdlib_module_names.py.
ericsnowcurrently Sep 7, 2021
e16771d
Add a comment to the frozen modules manifest file.
ericsnowcurrently Sep 7, 2021
e5b4da7
_Py_IsInstalled -> _Py_IsDevelopmentEnv.
ericsnowcurrently Sep 7, 2021
8b54908
Look up $_PYTHONTESTFROZENMODULES using getenv() instead of in os.env…
ericsnowcurrently Sep 7, 2021
a82b06a
Remove zipimport.h from the repo.
ericsnowcurrently Sep 7, 2021
ead34a1
Add zipimport to the list of essential frozen modules.
ericsnowcurrently Sep 7, 2021
58ccaff
Show how the frozen manifest changed.
ericsnowcurrently Sep 7, 2021
6d068b1
Stop using the frozen helper in test_cmd_line_script.
ericsnowcurrently Sep 7, 2021
1021f53
Go back to keeping frozen modules in the repo.
ericsnowcurrently Sep 7, 2021
c01b532
Also stop tracking the frozen manifest.
ericsnowcurrently Sep 8, 2021
0c1061d
Mark the frozen manifest as a generated file.
ericsnowcurrently Sep 8, 2021
fa23009
Drop a superfluous prefix on makefile rule dependencies.
ericsnowcurrently Sep 8, 2021
03b9571
Do not generate test code.
ericsnowcurrently Sep 8, 2021
b3a7aaf
Drop unused code from freeze_modules.py.
ericsnowcurrently Sep 8, 2021
6a896f1
Flip around the arg to the "frozen_modules" test helper.
ericsnowcurrently Sep 8, 2021
211febe
Disable frozen modules if $_PYTHONTESTFROZENMODULES is 0.
ericsnowcurrently Sep 8, 2021
fbaf15c
Add the "usefrozen" arg to CleanImport.__init__().
ericsnowcurrently Sep 8, 2021
6243c54
Undo the changes to tests that were using the os module.
ericsnowcurrently Sep 8, 2021
da978a2
Undo the changes to tests that were using the os module.
ericsnowcurrently Sep 8, 2021
46d488f
Ignore decode errors in find_frozen().
ericsnowcurrently Sep 8, 2021
93fc5c0
Fix test_ctypes.
ericsnowcurrently Sep 8, 2021
dd36ba5
Allow for a .exe suffix on the executable.
ericsnowcurrently Sep 9, 2021
45e7509
Do not clear the frozen .h files with "make distclean", now that they…
ericsnowcurrently Sep 9, 2021
a7d607f
Update the frozen modules using a debug build.
ericsnowcurrently Sep 9, 2021
75556af
Always flush the printed "title" when freezing modules.
ericsnowcurrently Sep 10, 2021
37b79f1
Add the frozen manifest back into the repo.
ericsnowcurrently Sep 10, 2021
d93c1e7
Drop the PGO check (for now).
ericsnowcurrently Sep 10, 2021
25b60de
Drop _Py_IsDevelopmentEnv().
ericsnowcurrently Sep 10, 2021
358ab69
Drop -_Py_GetStdlibDir() and PyConfig.stdlib_dir.
ericsnowcurrently Sep 10, 2021
1fc1198
Default to "-X frozen_modules=off" if built with --with-debug.
ericsnowcurrently Sep 10, 2021
37eb1f8
Fix test_embed.
ericsnowcurrently Sep 10, 2021
2e05771
Clean up the marshal code.
ericsnowcurrently Sep 10, 2021
82b4f85
Add WFILE.refs.nonref.
ericsnowcurrently Sep 10, 2021
097fc40
Add _PyMarshal_WriteForFreezing().
ericsnowcurrently Sep 10, 2021
a333fa3
Fix the ASAN job.
ericsnowcurrently Sep 11, 2021
8abf4a0
Fix the Windows builds.
ericsnowcurrently Sep 11, 2021
1005321
Use a simpler change for test_idle.
ericsnowcurrently Sep 11, 2021
7d62747
Fix test_4_daemon_threads.
ericsnowcurrently Sep 13, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add the "usefrozen" arg to CleanImport.__init__().
  • Loading branch information
@ericsnowcurrently
ericsnowcurrently committed Sep 8, 2021
commit fbaf15c889dde8ecd202737e7b35ca77b00254dc
5 changes: 4 additions & 1 deletion Lib/test/support/import_helper.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ class CleanImport(object):
importlib.import_module("foo") # new reference
"""

def __init__(self, *module_names):
def __init__(self, *module_names, usefrozen=False):

@gvanrossum gvanrossum Sep 11, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you update the class docstring to mention that it also disabled frozen modules unless usefrozen=True is passed?

self.original_modules = sys.modules.copy()
for module_name in module_names:
if module_name in sys.modules:
Expand All @@ -197,12 +197,15 @@ def __init__(self, *module_names):
if module.__name__ != module_name:
del sys.modules[module.__name__]
del sys.modules[module_name]
self._frozen_modules = frozen_modules(usefrozen)

def __enter__(self):
self._frozen_modules.__enter__()
return self

def __exit__(self, *ignore_exc):
sys.modules.update(self.original_modules)
self._frozen_modules.__exit__(*ignore_exc)


class DirsOnSysPath(object):
Expand Down