Skip to content

Commit 9c55c22

Browse files
dumoldumol
authored
[#5247] Stop using Homebrew's OpenSSL libs. (#130)
* Stop using Homebrew's OpenSSL libs. * Imported LibreSSL 2.2.7 sources * Use our own headers to link to LibreSSL libs. * Updated paver.conf to latest python-package rev. * Removed redundant OpenSSL prefix for cryptography/pyopenssl version strings. * We only build on 10.13, so don't build ZLIB on macOS. * Added spreadsheet with external deps in CSV and flat ODS formats. * Changes after own review. * Use 'macos' and 'osx' for Apple's OS'es. * Removed special treatment for SLES 11 with Security Module. * Updated external deps spreadsheet and csv files. * Changes after own review. * Minor improvements for external deps spreadsheet. * Documented the external deps files.
1 parent 690bed9 commit 9c55c22

1,311 files changed

Lines changed: 493950 additions & 53 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎README‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,3 +53,15 @@ src/python/Python-2.7.13/Lib/site.py::
5353
diff -ru Lib/site.py Lib/site-patched.py
5454
# Copy the diff result, including the command line into a patch file,
5555
# for example at at src/python/name-of-the.patch
56+
57+
58+
External dependencies and associated vulnerabilities
59+
----------------------------------------------------
60+
61+
This Python package builds on and links to third-party libraries.
62+
63+
To document them for all OS'es and to list their known vulnerabilities,
64+
consult the LibreOffice spreadsheet `external_deps.fods`.
65+
66+
For your convenience, this flat ODS file is also exported in CSV format in
67+
`external_deps.csv`.

‎chevah_build‎

Lines changed: 17 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -165,10 +165,6 @@ case "$OS" in
165165
sles10)
166166
SLES_PKGS="$COMMON_SLES_PKGS openssl-devel"
167167
;;
168-
sles11sm)
169-
# SLES 11 with Security Module detected, OpenSSL 1.0.x headers needed.
170-
SLES_PKGS="$COMMON_SLES_PKGS libopenssl1-devel"
171-
;;
172168
sles11|sles12)
173169
SLES_PKGS="$COMMON_SLES_PKGS libopenssl-devel"
174170
;;
@@ -397,16 +393,14 @@ case $OS in
397393
macos*)
398394
export CC="clang"
399395
export CXX="clang++"
400-
# No zlib headers on macOS 10.14.
401-
export BUILD_ZLIB="yes"
402-
# Up to macOS 10.12, Apple only provides OpenSSL version 0.9.8zh.
403-
# macOS 10.13 and newer come with LibreSSL, not clear if maintained.
404-
# Therefore, we require Homebrew's OpenSSL libs, located as follows.
405-
export LDFLAGS="-L/usr/local/opt/openssl/lib $LDFLAGS"
406-
export CPPFLAGS="$CPPFLAGS -I/usr/local/opt/openssl/include"
407396
# setup.py skips building readline by default, as it sets this to
408397
# "10.4", and then tries to avoid the broken readline in OS X 10.4.
409-
export MACOSX_DEPLOYMENT_TARGET=10.11
398+
export MACOSX_DEPLOYMENT_TARGET=10.13
399+
# 10.13 and newer come with LibreSSL instead of the old OpenSSL libs.
400+
# But the headers are missing, so we copy them ourselves when building.
401+
LIBRESSL_VERSION=2.2.7
402+
# Beware that zlib headers are not present either on macOS 10.14.
403+
# We don't care here because we build on 10.13.
410404
;;
411405
freebsd*|openbsd*)
412406
export CC="clang"
@@ -417,21 +411,6 @@ case $OS in
417411
# SLES's libffi headers from SDK do not match default-available package.
418412
export BUILD_LIBFFI="yes"
419413
;;
420-
sles11sm)
421-
# SLES's libffi headers from SDK do not match default-available package.
422-
export BUILD_LIBFFI="yes"
423-
# Use PyCryptodome's wheel, as there are issues not solved upstream yet.
424-
PIP_LIBRARIES="\
425-
setproctitle==${SETPROCTITLE_VERSION} \
426-
pycryptodomex==${PYCRYPTODOMEX_VERSION}upstream \
427-
cryptography==${CRYPTOGRAPHY_VERSION} \
428-
pyOpenSSL==${PYOPENSSL_VERSION} \
429-
scandir==${SCANDIR_VERSION} \
430-
psutil==${PSUTIL_VERSION} \
431-
Cython==${CYTHON_VERSION} \
432-
subprocess32==${SUBPROCESS32_VERSION} \
433-
"
434-
;;
435414
rhel5|sles11)
436415
# SLES's libffi headers from SDK do not match default-available package.
437416
export BUILD_LIBFFI="yes"
@@ -548,19 +527,11 @@ check_dependencies() {
548527
packages=$ARCH_PKGS
549528
check_command='pacman -Qi'
550529
;;
530+
# On remaining OS'es we just check for some of the needed commands.
551531
macos*)
552-
# On macOS we need OpenSSL installed via Homebrew, and here we
553-
# check that the expected version is installed, as hardcoded below.
554-
homebrew_openssl_version=`brew list --versions openssl`
555-
if [[ $homebrew_openssl_version != "openssl 1.0.2"* ]]; then
556-
(>&2 echo "Unsupported version: $homebrew_openssl_version")
557-
exit 102
558-
fi
559-
# But we also check for the required commands, as for below OS'es.
560532
packages="$CC make m4 libtool git patch"
561533
check_command="command -v"
562534
;;
563-
# On remaining OS'es we just check for some of the needed commands.
564535
windows)
565536
# The windows build is special.
566537
command -v choco
@@ -656,7 +627,16 @@ command_build() {
656627

657628
case $OS in
658629
windows*)
659-
build 'python' "Python-$PYTHON_BUILD_VERSION-windows" ${PYTHON_BUILD_FOLDER}
630+
build 'python' "Python-$PYTHON_BUILD_VERSION-windows" \
631+
${PYTHON_BUILD_FOLDER}
632+
;;
633+
macos*)
634+
# SSL headers are missing on macOS.
635+
echo "Copying LibreSSL $LIBRESSL_VERSION headers. Updating CFLAGS!"
636+
cp -r src/libressl/libressl-${LIBRESSL_VERSION}/include/openssl \
637+
$INSTALL_FOLDER/include/
638+
export CFLAGS="$CFLAGS -I${INSTALL_FOLDER}/include"
639+
build 'python' "Python-$PYTHON_BUILD_VERSION" ${PYTHON_BUILD_FOLDER}
660640
;;
661641
*)
662642
build 'python' "Python-$PYTHON_BUILD_VERSION" ${PYTHON_BUILD_FOLDER}

‎external_deps.csv‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
OS,AIX,,,Amazon,Alpine,Debian,,FreeBSD,,HP-UX,macOS,OS X,RHEL,,,,SLES,,,,Solaris,,,,,,,,Windows,
2+
OS Version,5.3²,6.1³,7.1²,2¹,3.7¹,7.11³,8.11¹,10.4³,11.2²,11.31²,10.13+¹,10.8+²,5.11³,6.10²,7.3²,7.4¹,10 SP3³,11SP4¹,11SP4SM³,12SP3¹,10u3-10u7³,10u8+²,11.0/11.1³,11.2²,11.4²,14.04²,16.04²,18.04¹,"XP, 2003, 2008²","2012, 2016, 2019¹"
3+
"OpenSSL /
4+
LibreSSL⁹",1.0.2k (from AIX Web Download Pack Programs),1.0.2k (from AIX Web Download Pack Programs),1.0.2k (from AIX Web Download Pack Programs),1.0.2k-fips,2.6.5 (LibreSSL),1.0.1t,1.0.1t,1.0.1u,1.0.2o,1.0.2h,2.2.7 (LibreSSL),"0.9.8r⁶ (only used for stdlib ssl)
5+
1.1.1b (bundled with upstream cryptography 2.6.1)","0.9.8e-fips
6+
1.1.1b (bundled with upstream cryptography 2.6.1)",1.0.1e-fips,1.0.1e-fips,1.0.2k-fips,0.9.8a,"0.9.8j-fips (only used for stdlib ssl)
7+
1.1.1b (bundled with upstream cryptography 2.6.1)",1.0.1g,1.0.2j-fips,0.9.7d,1.0.2n (from upstream Oracle patches),1.0.0x,1.0.1h,1.0.2o,1.0.1f,1.0.2g,1.1.0g,"1.0.2q (bundled with upstream Python 2.7.16)
8+
1.1.1b (bundled with upstream cryptography 2.6.1)","1.0.2q (bundled with upstream Python 2.7.16)
9+
1.1.1b (bundled with upstream cryptography 2.6.1)"
10+
Python,2.7.16,2.7.16,2.7.16,2.7.16,2.7.16,2.7.16,2.7.16,2.7.16,2.7.16,2.7.16,2.7.16,2.7.16,2.7.16,2.7.16,2.7.16,2.7.16,2.7.16,2.7.16,2.7.16,2.7.16,2.7.8⁴,2.7.8⁴,2.7.16,2.7.16,2.7.16,2.7.16,2.7.16,2.7.16,2.7.16,2.7.16
11+
SQLite,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1,3.27.1 (we overwrite version from upstream Python at build time),3.27.1 (we overwrite version from upstream Python at build time)
12+
Expat,2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.1.0⁵ (bundled with Python 2.7.8),2.1.0⁵ (bundled with Python 2.7.8),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python),2.2.6 (bundled with Python)
13+
zlib,p/o,p/o,p/o,p/o,p/o,p/o,p/o,p/o,p/o,1.2.11,p/o,p/o,p/o,p/o,p/o,p/o,p/o,p/o,p/o,p/o,1.2.11,p/o,p/o,p/o,p/o,p/o,p/o,p/o,1.2.11 (bundled with Python),1.2.11 (bundled with Python)
14+
libffi,3.2.1,3.2.1,3.2.1,p/o,p/o,3.2.1,3.2.1,3.2.1,3.2.1,3.2.1,p/o,p/o,3.2.1,p/o,p/o,p/o,3.2.1,3.2.1,3.2.1,3.2.1,n/a,n/a,n/a,n/a,p/o,p/o,p/o,p/o,n/a,n/a
15+
libedit,n/a,n/a,n/a,20170329-3.1,20170329-3.1,n/a,n/a,20170329-3.1,20170329-3.1,n/a,20170329-3.1,20170329-3.1,20170329-3.1,20170329-3.1,20170329-3.1,20170329-3.1,20170329-3.1,20170329-3.1,20170329-3.1,20170329-3.1,n/a,n/a,20170329-3.1,20170329-3.1,20170329-3.1,20170329-3.1,20170329-3.1,20170329-3.1,n/a,n/a
16+
GMP,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,6.1.2,n/a,n/a
17+
pysqlite,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,2.8.3,n/a,n/a
18+
pip,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3,9.0.3
19+
setuptools,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1,39.0.1
20+
wheel,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0,0.26.0
21+
pycparser,"2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14","2,14"
22+
setproctitle,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10,1.1.10
23+
cryptography,2.6.1,2.6.1,2.6.1,2.6.1,2.6.1,2.6.1,2.6.1,2.6.1,2.6.1,n/a,2.6.1,2.6.1 (wheel includes OpenSSL),2.6.1 (wheel includes OpenSSL),2.6.1,2.6.1,2.6.1,2.6.1,2.6.1 (wheel includes OpenSSL),2.6.1,2.6.1,n/a,n/a,n/a,n/a,2.6.1,2.6.1,2.6.1,2.6.1,2.6.1 (wheel includes OpenSSL),2.6.1 (wheel includes OpenSSL)
24+
six,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0,1.11.0
25+
ipaddress,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18,1.0.18
26+
cffi,1.12.2,1.12.2,1.12.2,1.12.2,1.12.2,1.12.2,1.12.2,1.12.2,1.12.2,n/a,1.12.2,1.12.2,1.12.2,1.12.2,1.12.2,1.12.2,1.12.2,1.12.2,1.12.2,1.12.2,n/a,n/a,1.12.2,1.12.2,1.12.2,1.12.2,1.12.2,1.12.2,1.12.2,1.12.2
27+
asn1crypto,"0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23","0,23"
28+
enum34,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6,1.1.6
29+
idna,"2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6","2,6"
30+
pyOpenSSL,19.0.0,19.0.0,19.0.0,19.0.0,19.0.0,19.0.0,19.0.0,19.0.0,19.0.0,0.13.1⁷,19.0.0,19.0.0,19.0.0,19.0.0,19.0.0,19.0.0,0.13.1⁷,19.0.0,19.0.0,19.0.0,0.13.1⁷,0.13.1⁷,0.13.1⁷,0.13.1⁷,19.0.0,19.0.0,19.0.0,19.0.0,19.0.0,19.0.0
31+
pycrypto,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸,2.6.1⁸
32+
scandir,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0,1.10.0
33+
psutil,n/a,5.6.1,5.6.1,5.6.1,5.6.1,5.6.1,5.6.1,5.6.1,5.6.1,n/a,5.6.1,5.6.1,5.6.1,5.6.1,5.6.1,5.6.1,5.6.1,5.6.1,5.6.1,5.6.1,n/a,n/a,n/a,n/a,5.6.1,5.6.1,5.6.1,5.6.1,n/a,5.6.1
34+
subprocess32,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3,3.5.3
35+
pywin32,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,n/a,223,223
36+
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
37+
Abbreviations:,n/a: not applicable,,,,,,,,,,,Notes:,"1. Tier 1 platforms, fully supported and tested",,,,,,,,,,,,,,,,,
38+
,p/o: provided with OS,,,,,,,,,,,,"2. Tier 2 platforms, partially supported, still built",,,,,,,,,,,,,,,,,
39+
Colour codes:,DARKGREY: Tier 2 platforms and their problematic dependencies,,,,,,,,,,,,"3. Tier 3 platforms, supported at some point, not built any more",,,,,,,,,,,,,,,,,
40+
,LIGHT GREY: Tier 3 platforms and their problematic dependencies,,,,,,,,,,,,4. https://www.cvedetails.com/vulnerability-list/vendor_id-10210/product_id-18230/version_id-92056/Python-Python-2.7.html,,,,,,,,,,,,,,,,,
41+
,GREEN: no known vulnerabilities for Tier 1 platforms,,,,,,,,,,,,5. https://github.com/libexpat/libexpat/blob/master/expat/Changes,,,,,,,,,,,,,,,,,
42+
,"BLUE: possible vulnerabilities found upstream, but no released version has them yet",,,,,,,,,,,,6. As Apple stopped backporting security patches for OpenSSL 0..9.8: https://www.cvedetails.com/vulnerability-list/vendor_id-217/product_id-383/version_id-26306/Openssl-Openssl-0.9.8.html,,,,,,,,,,,,,,,,,
43+
,ORANGE: minor vulnerabilities found,,,,,,,,,,,,"7. pyOpenSSL 0.14 and newer is a major rewrite, so it's not clear to what extent their vulnerabilities do apply",,,,,,,,,,,,,,,,,
44+
,RED: major vulnerabilities found,,,,,,,,,,,,"8. PyCrypto 2.6.1 is vulnerable upstream, but we use the patches from Ubuntu LTS 16.04 to avoid CVE-2013-7459 and CVE-2018-6594.",,,,,,,,,,,,,,,,,
45+
,MAGENTA: vulnerability status could not be established,,,,,,,,,,,,"9. Unless specified otherwise, OpenSSL libs are linked against dinamically",,,,,,,,,,,,,,,,,
46+
,DEFAULT COLOUR: maintained upstream or not applicable,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

0 commit comments

Comments
 (0)