Update for latest CPython release

@mgorny

CPython did a security release
We should

Update libexpat to 2.6.0. This requires updating the build workers (linux Dockerfiles, windows dependency, macos via brew). Perhaps we should do like CPython and bundle libexpat into our sources for pypy/modules/pyexpat [3.9] Upgrade bundled libexpat to 2.6.0 (GH-115399) python/cpython#115474 which would statically link libexpat and reduce the number of bundled shared objects from 8 to 7
Update the stdlib to get the fixes for zipfile, tempfile, urllib, socket test, hidden pth, iso2022_jp_3 test, libexpat test, etc
Check that we properly handle the underlying socket function changed in [3.9] bpo-37013: Fix the error handling in socket.if_indextoname() (GH-13503) python/cpython#112600
Check that we pass the iso2022_jp_3 test [3.9] gh-101180: Fix a bug where iso2022_jp_3 and iso2022_jp_2004 codecs read out of bounds (gh-111695) python/cpython#111780
Use X509_STORE_get1_objects instead of X509_STORE_get0_objects in _ssl [3.10] gh-114572: Fix locking in cert_store_stats and get_ca_certs (GH-114573) python/cpython#115548, raising better error in _ssl test_asyncio: test_create_connection_ssl_failed_certificate() failed on ARM64 macOS 3.x buildbot python/cpython#107077
Update bundled OpenSSL to 3.0.11 (we already use 3.0.12)
Update zlib 1.3.1 (windows)
Use new libexpat API [3.9] gh-115398: Expose Expat >=2.6.0 reparse deferral API (CVE-2023-52425) (GH-115623) python/cpython#116272

cc @mgorny

Check that we pass the iso2022_jp_3 test python/cpython#111780

we have vendored a (modified?) copy of _codecs_iso2022.c, so I think we need to do some fixes here.

I updated py3.9 to 3.9.19

Newly failing tests, to be investigated:
On py3.9

lib-python.3.test.test_codecencodings_iso2022
lib-python.3.test.test_multiprocessing_spawn
lib-python.3.test.test_unicode

Additionally, on py3.10, maybe due to the way I updated the stdlib (only merged from py3.9, did not sync with cpython 3.10.14)

lib-python.3.test.test_ensurepip
lib-python.3.test.test_tarfile
lib-python.3.test.test_tempfile
lib-python.3.test.test_xml_etree
lib-python.3.test.test_xml_etree_c

Additionally, should try to figure out where this pytest collection failure is coming from, it appears as a line in the buildbot summary

E E E E E pypy-c-jit-linux-x86-64 pytest internal

Edit: fixed the pytest failure

The pytest error is coming from the -D tests

The tarfile failures come from stat.filemode(4095) which on CPython returns '?rwsrwsrwt' and on PyPy returns '-rwsrwsrwt'. The first letter is the file type.

We use the pure python implementation from lib-python/3/stat.py, CPython uses a c-implementation. The filemode implementation was fixed on CPython main in python/cpython#108638. I resynced in 27da920

Closing, the stdlib has been updated.

Thanks a lot!

Any chance for a new release, by the way? I think we've collected quite some fixes already.

Yes, a release is a good idea. I want to stabilize the churn a bit first, there are some weird buildbot failures on windows. There are also a few milestone issues to resolve

Navigation Menu

Repository navigation

Update for latest CPython release #4927

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Update for latest CPython release #4927

Description

Activity

cfbolz commented on Mar 20, 2024

mattip commented on Apr 6, 2024

mattip commented on Apr 11, 2024

mattip commented on Apr 12, 2024

mattip commented on Apr 15, 2024

mattip commented on Apr 15, 2024

mattip commented on Apr 16, 2024

mgorny commented on Apr 16, 2024

mgorny commented on Apr 16, 2024

mattip commented on Apr 16, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions