Repository navigation
Fatal Python error: _PySys_BeginInit: can't initialize sys module #79894
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2019-01-18.17:52:46.947>
created_at = <Date 2019-01-10.22:35:31.620>
labels = ['build', '3.7']
title = "Fatal Python error: _PySys_BeginInit: can't initialize sys module"
updated_at = <Date 2019-01-23.14:06:29.967>
user = 'https://bugs.python.org/Tasy'bugs.python.org fields:
activity = <Date 2019-01-23.14:06:29.967>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2019-01-18.17:52:46.947>
closer = 'Tasy'
components = ['Build']
creation = <Date 2019-01-10.22:35:31.620>
creator = 'Tasy'
dependencies = []
files = []
hgrepos = []
issue_num = 35713
keywords = []
message_count = 14.0
messages = ['333423', '333425', '333427', '333428', '333429', '333733', '333853', '333984', '333992', '333999', '334219', '334233', '334258', '334259']
nosy_count = 3.0
nosy_names = ['vstinner', 'xtreak', 'Tasy']
pr_nums = ['11647', '11647', '11647', '11650', '11650', '11650', '11658', '11658', '11658']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue35713'
versions = ['Python 3.7']
Activity
Tasy commentedon Jan 10, 2019
.
.
.
./python -E -S -m sysconfig --generate-posix-vars ;\
if test $? -ne 0 ; then \
echo "generate-posix-vars failed" ; \
rm -f ./pybuilddir.txt ; \
exit 1 ; \
fi
Fatal Python error: _PySys_BeginInit: can't initialize sys module
Current thread 0x00002b4e5f9bf400 (most recent call first):
Aborted (core dumped)
generate-posix-vars failed
make[1]: *** [pybuilddir.txt] Error 1
make[1]: Leaving directory `/usr/local/mysoftware/Python-3.7.2/build'
make: *** [profile-opt] Error 2
vstinner commentedon Jan 10, 2019
Hello. Can you elaborate this part?
Tasy commentedon Jan 10, 2019
Sorry for the confusion. The three dots were for many lines of successful compilation output from make. The rest is the final few lines where the compilation fails.
vstinner commentedon Jan 10, 2019
Because it's late, I will say it shortly: if you don't elaborate how you get this error, I will simply close the issue. You have to describe what you are trying to do, your OS, etc.
(It works for me!)
tirkarthi commentedon Jan 10, 2019
Please attach the full build log too along with the command executed.
Tasy commentedon Jan 15, 2019
Configuration Options:
../configure --prefix=$HOME --enable-shared --enable-optimizations --with-system-expat --with-system-ffi --with-ensurepip=yes
Make throws the following warning:
*** WARNING: renaming "_curses_panel" since importing it failed: No module named '_curses'
Python build finished successfully!
The necessary bits to build these optional modules were not found:
_ssl _uuid
To find the necessary bits, look in setup.py in detect_modules() for the module's name.
The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc atexit pwd
time
Failed to build these modules:
_curses
Following modules built successfully but were removed because they could not be imported:
_curses_panel
Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, libressl/portable#381
There ther is a following error...
0:06:18 load avg: 0.55 [171/416] test_hashlib
*** Error in `./python': corrupted size vs. prev_size: 0x000000000276b7a0 ***
Fatal Python error: Aborted
Current thread 0x00002ba4468c7bc0 (most recent call first):
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/test_hashlib.py", line 904 in _test_pbkdf2_hmac
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/test_hashlib.py", line 935 in test_pbkdf2_hmac_c
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/unittest/case.py", line 615 in run
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/unittest/case.py", line 663 in call
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/unittest/suite.py", line 122 in run
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/unittest/suite.py", line 84 in call
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/unittest/suite.py", line 122 in run
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/unittest/suite.py", line 84 in call
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/unittest/suite.py", line 122 in run
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/unittest/suite.py", line 84 in call
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/support/testresult.py", line 162 in run
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/support/init.py", line 1895 in _run_suite
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/support/init.py", line 1991 in run_unittest
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/libregrtest/runtest.py", line 178 in test_runner
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/libregrtest/runtest.py", line 182 in runtest_inner
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/libregrtest/runtest.py", line 137 in runtest
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/libregrtest/main.py", line 407 in run_tests_sequential
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/libregrtest/main.py", line 514 in run_tests
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/libregrtest/main.py", line 615 in _main
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/libregrtest/main.py", line 582 in main
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/libregrtest/main.py", line 636 in main
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/regrtest.py", line 46 in _main
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/regrtest.py", line 50 in
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/runpy.py", line 85 in _run_code
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/runpy.py", line 193 in _run_module_as_main
Aborted (core dumped)
make[1]: Leaving directory
/usr/local/data/mySoftware/Python-3.7.2/build' make build_all_merge_profile make[1]: Entering directory/usr/local/data/mySoftware/Python-3.7.2/build'true
make[1]: Leaving directory `/usr/local/data/mySoftware/Python-3.7.2/build'
Remove profile generation binary since we are done with it.
make clean
make[1]: Entering directory
/usr/local/data/mySoftware/Python-3.7.2/build' find .. -depth -name '__pycache__' -exec rm -rf {} ';' find .. -name '*.py[co]' -exec rm -f {} ';' find . -name '*.[oa]' -exec rm -f {} ';' find . -name '*.s[ol]' -exec rm -f {} ';' find . -name '*.so.[0-9]*.[0-9]*' -exec rm -f {} ';' find build -name 'fficonfig.h' -exec rm -f {} ';' || true find build -name '*.py' -exec rm -f {} ';' || true find build -name '*.py[co]' -exec rm -f {} ';' || true rm -f pybuilddir.txt rm -f [Lib/lib2to3/](https://github.com/python/cpython/blob/main/Lib/lib2to3)*Grammar*.pickle rm -f [Programs/_testembed](https://github.com/python/cpython/blob/main/Programs/_testembed) [Programs/_freeze_importlib](https://github.com/python/cpython/blob/main/Programs/_freeze_importlib) find build -type f -a ! -name '*.gc??' -exec rm -f {} ';' rm -f [Include/pydtrace_probes.h](https://github.com/python/cpython/blob/main/Include/pydtrace_probes.h) rm -f profile-gen-stamp make[1]: Leaving directory/usr/local/data/mySoftware/Python-3.7.2/build'This is an expensive target to build and it does not have proper
makefile dependency information. So, we create a "stamp" file
to record its completion and avoid re-running it.
touch profile-run-stamp
Rebuilding with profile guided optimizations:
rm -f profile-clean-stamp
make build_all CFLAGS_NODIST=" -fprofile-use -fprofile-correction" LDFLAGS_NODIST=""
make[1]: Entering directory `/usr/local/data/mySoftware/Python-3.7.2/build'
And then finally :
Modules/_localemodule.o Modules/_iomodule.o Modules/iobase.o Modules/fileio.o Modules/bytesio.o Modules/bufferedio.o Modules/textio.o Modules/stringio.o Modules/zipimport.o Modules/faulthandler.o Modules/_tracemalloc.o Modules/hashtable.o Modules/symtablemodule.o Modules/xxsubtype.o Python/frozen.o
gcc -pthread -shared -Wl,--no-as-needed -o libpython3.so -Wl,-hlibpython3.so libpython3.7m.so
gcc -pthread -Xlinker -export-dynamic -o python Programs/python.o -L. -lpython3.7m -lpthread -ldl -lutil -lm
gcc -pthread -Xlinker -export-dynamic -o Programs/_testembed Programs/_testembed.o -L. -lpython3.7m -lpthread -ldl -lutil -lm
LD_LIBRARY_PATH=/usr/local/data/mySoftware/Python-3.7.2/build:$HOME/lib:/usr/local/cuda/lib64:$HOME/lib/cuda/lib64:$HOME/lib/cuda/extras/CUPTI/lib64:/usr/local/data/lib:/usr/local/lib/:/usr/lib/ ./python -E -S -m sysconfig --generate-posix-vars ;\
if test $? -ne 0 ; then \
echo "generate-posix-vars failed" ; \
rm -f ./pybuilddir.txt ; \
exit 1 ; \
fi
Fatal Python error: _PySys_BeginInit: can't initialize sys module
Current thread 0x00002adc936069c0 (most recent call first):
Aborted (core dumped)
generate-posix-vars failed
make[1]: *** [pybuilddir.txt] Error 1
make[1]: Leaving directory `/usr/local/data/mySoftware/Python-3.7.2/build'
make: *** [profile-opt] Error 2
vstinner commentedon Jan 17, 2019
What is your OS (name and version)?
What is your compiler (name and version)?
I have no idea why you get this error. You should try to run this function in a debugger like gdb and run the code step by step to see what happens.
"""
0:06:18 load avg: 0.55 [171/416] test_hashlib
*** Error in `./python': corrupted size vs. prev_size: 0x000000000276b7a0 ***
Fatal Python error: Aborted
Current thread 0x00002ba4468c7bc0 (most recent call first):
File "/usr/local/data/mySoftware/Python-3.7.2/Lib/test/test_hashlib.py", line 904 in _test_pbkdf2_hmac
"""
That's maybe unrelated, but _test_pbkdf2_hmac() is not supposed to crash.
Tasy commentedon Jan 18, 2019
Compiler:
$ gcc --version
gcc (Ubuntu 4.8.4-2ubuntu1~14.04.4) 4.8.4
OS:
$ uname -a
Linux machine 4.4.0-141-generic #167~14.04.1-Ubuntu SMP Mon Dec 10 13:20:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Home directory is of type nfs in case that is relevant.
I'll try to do the gdb thing over the weekend.
vstinner commentedon Jan 18, 2019
Hum. Are you aware that PGO with GCC is broken on such old and unsupported Ubuntu version?
My old note about that:
"PGO is broken on Ubuntu 14.04 LTS with GCC 4.8.4-2ubuntu1~14.04: Modules/socketmodule.c:7743:1: internal compiler error: in edge_badness, at ipa-inline.c:895"
https://pyperformance.readthedocs.io/usage.html#compile
See also https://bugs.python.org/issue31963
I suggest you to not use PGO compilation on old Ubuntu. Maybe upgrade to latest Ubuntu LTS?
Tasy commentedon Jan 18, 2019
compiling without optimizations worked.
vstinner commentedon Jan 22, 2019
New changeset bf4ac2d by Victor Stinner in branch 'master':
bpo-35713: Rework Python initialization (GH-11647)
bf4ac2d
vstinner commentedon Jan 22, 2019
New changeset 6d43f6f by Victor Stinner in branch 'master':
bpo-35713: Split _Py_InitializeCore into subfunctions (GH-11650)
6d43f6f
vstinner commentedon Jan 23, 2019
New changeset ab67281 by Victor Stinner in branch 'master':
bpo-35713: Reorganize sys module initialization (GH-11658)
ab67281
vstinner commentedon Jan 23, 2019
"""
I have no idea why you get this error. You should try to run this function in a debugger like gdb and run the code step by step to see what happens.
"""
I pushed 3 changes to get working exceptions and working sys.stderr earlier during Python initialization. It should help to display the current exceptions when Py_FatalError() is called during early stage of the Python initialization.
sergey-dryabzhinsky commentedon Jun 15, 2023
Same error while building package for Ubuntu 14.04.
--prefix=/usr --enable-ipv6 --enable-loadable-sqlite-extensions --enable-optimizations --with-dbmliborder=bdb:gdbm --with-computed-gotos --without-ensurepip --with-system-libmpdec --without-system-expat --with-openssl=/opt/openssl111r/usr --with-system-ffiPrevious python version compiled fine.
Wait. It's maybe cos PGO enabled somehow.
vstinner commentedon Jun 15, 2023
You're using old Python on old Ubuntu. Maybe use Ubuntu packages like deadsnakes instead of building Python yourself. Yes, enable-optimizations enables PGO.
This issue is closed and Python 3.8 is no longer supported. Give a try to Python 3.11.
sergey-dryabzhinsky commentedon Jun 15, 2023
Yeah, error gone if enable-optimizations removed from configure args.
yevgen-lasman commentedon Jan 22, 2024
The problem persists for me for either 3.11 or 3.12 Glad to provide more details about the build environment if it helps to solve this problem.
vstinner commentedon Jan 23, 2024
If you can reproduce the issue on a recent Python version, please open a new issue with complete instructions on how to reproduce it, the output and expected output. This issue is old and closed.