Skip to content

Commit b4b218d

Browse files
linesightclaude
andcommitted
Modernize build toolchain for CEF 123+ and Python 3.10+
- Add tools/download_cef.py to auto-download CEF from Spotify CDN with SHA1 verification and progress output - Detect vcvarsall.bat dynamically via vswhere.exe instead of hardcoding VS2022 Community path; fall back through known editions - Remove dead code for VS2008/VS2010/VS2013 and simplify prepare_build_command() - Make VERSION argument optional in build.py, defaulting to {CHROME_VERSION_MAJOR}.0 from the version header - Add venv support notes and document download_cef.py workflow in Build-instructions.md Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 12ef37c commit b4b218d

6 files changed

Lines changed: 405 additions & 233 deletions

File tree

‎docs/Build-instructions.md‎

Lines changed: 30 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -94,9 +94,9 @@ pip install --upgrade -r ../tools/requirements.txt
9494

9595
8) Extract the archive in the "build/" directory.
9696

97-
9) Build cefpython and run examples (xx.x is version number):
97+
9) Build cefpython and run examples:
9898
```
99-
python ../tools/build.py xx.x
99+
python ../tools/build.py
100100
```
101101

102102

@@ -143,9 +143,9 @@ sudo pip install --upgrade -r ../tools/requirements.txt
143143

144144
7) Extract the archive in the "build/" directory.
145145

146-
8) Build cefpython and run examples (xx.x is version number):
146+
8) Build cefpython and run examples:
147147
```
148-
python ../tools/build.py xx.x
148+
python ../tools/build.py
149149
```
150150

151151

@@ -260,9 +260,9 @@ cd build/
260260
3) Extract the downloaded archive eg. "cef55_3.2883.1553.g80bd606_win32.zip"
261261
in the "build/" directory (using "extract here" option)
262262

263-
4) Run the build.py tool (xx.x is version number):
263+
4) Run the build.py tool:
264264
```
265-
python ../tools/build.py xx.x
265+
python ../tools/build.py
266266
```
267267

268268

@@ -281,22 +281,36 @@ mkdir build/
281281
cd build/
282282
```
283283

284-
2) Download CEF binaries from [Spotify Automated Builds](https://cef-builds.spotifycdn.com/index.html).
285-
The version of the binaries must match exactly the CEF version
286-
from the "cefpython/src/version/" directory (look for CEF_VERSION
287-
constant in .h file).
284+
2) Download and extract CEF binaries automatically using the
285+
download_cef.py tool. It reads the required CEF version from
286+
"cefpython/src/version/", queries the Spotify CDN index, downloads
287+
the standard distribution, verifies the SHA1 checksum, and extracts
288+
it to the build/ directory:
289+
```
290+
python ../tools/download_cef.py
291+
```
288292

289-
3) Extract the downloaded archive eg.
290-
"cef_binary_3.2883.1553.g80bd606_windows32.tar.bz2"
291-
in the build/ directory (using "extract here" option)
293+
Alternatively, download manually from
294+
[Spotify Automated Builds](https://cef-builds.spotifycdn.com/index.html).
295+
The version must match exactly the CEF version from
296+
"cefpython/src/version/" (look for CEF_VERSION constant in .h file).
297+
Extract the archive eg. "cef_binary_3.2883.1553.g80bd606_windows32.tar.bz2"
298+
in the build/ directory (using "extract here" option).
292299

293-
4) Run the automate.py tool. After it completes you should see a new
294-
directory eg. "cef55_3.2883.1553.g80bd606_win32/".
300+
3) Build libcef_dll_wrapper and prepare the CEF binaries directory using
301+
the automate.py tool. This processes the downloaded "cef_binary_*"
302+
directory and creates a new directory eg. "cef55_3.2883.1553.g80bd606_win32/"
303+
that build.py requires. Note: this step requires cmake and ninja on PATH.
295304
```
296305
python ../tools/automate.py --prebuilt-cef
297306
```
298307

299-
5) Run the build.py tool (xx.x is version number):
308+
4) Run the build.py tool. The version number is optional and defaults to
309+
{CHROME_VERSION_MAJOR}.0 read from "cefpython/src/version/":
310+
```
311+
python ../tools/build.py
312+
```
313+
To override the patch version (eg. for a second release off the same CEF):
300314
```
301315
python ../tools/build.py xx.x
302316
```

‎tools/automate.py‎

Lines changed: 13 additions & 78 deletions
Original file line numberDiff line numberDiff line change
@@ -368,8 +368,6 @@ def build_cef_projects():
368368
if WINDOWS:
369369
fix_cmake_variables_permanently_windows()
370370

371-
fix_cef_include_files()
372-
373371
# Find cef_binary directory.
374372
# Might already be set if --prebuilt-cef flag was passed.
375373
if not Options.cef_binary:
@@ -476,7 +474,7 @@ def build_wrapper_library_windows(runtime_library, msvs, vcvars):
476474
fix_cmake_variables_for_MD_library(try_undo=True)
477475

478476
# Command to build libcef_dll_wrapper
479-
cmake_wrapper = prepare_build_command(build_lib=True, vcvars=vcvars)
477+
cmake_wrapper = prepare_build_command()
480478
cmake_wrapper.extend(["cmake", "-G", "Ninja",
481479
"-DCMAKE_BUILD_TYPE="+Options.build_type, ".."])
482480

@@ -515,27 +513,14 @@ def build_wrapper_library_windows(runtime_library, msvs, vcvars):
515513
Options.gyp_msvs_version = msvs
516514
if runtime_library == RUNTIME_MD:
517515
fix_cmake_variables_for_MD_library()
518-
env = getenv()
519-
if msvs == "2010":
520-
# When Using WinSDK 7.1 vcvarsall.bat doesn't work. Use
521-
# setuptools.msvc.msvc9_query_vcvarsall to query env vars.
522-
from setuptools.msvc import msvc9_query_vcvarsall
523-
env.update(msvc9_query_vcvarsall(10.0, arch=VS_PLATFORM_ARG))
524-
# On Python 2.7 env values returned by both distutils
525-
# and setuptools are unicode, but Python expects env
526-
# dict values as strings.
527-
for env_key in env:
528-
env_value = env[env_key]
529-
if type(env_value) != str:
530-
env[env_key] = env_value.encode("utf-8")
531-
run_command(cmake_wrapper, working_dir=build_wrapper_dir, env=env)
516+
run_command(cmake_wrapper, working_dir=build_wrapper_dir)
532517
Options.gyp_msvs_version = old_gyp_msvs_version
533518
if runtime_library == RUNTIME_MD:
534519
fix_cmake_variables_for_MD_library(undo=True)
535520
print("[automate.py] cmake OK")
536521

537522
# Run ninja
538-
ninja_wrapper = prepare_build_command(build_lib=True, vcvars=vcvars)
523+
ninja_wrapper = prepare_build_command()
539524
ninja_wrapper.extend(["ninja", "-j", Options.ninja_jobs,
540525
"libcef_dll_wrapper"])
541526
run_command(ninja_wrapper, working_dir=build_wrapper_dir)
@@ -581,7 +566,6 @@ def fix_cmake_variables_for_MD_library(undo=False, try_undo=False):
581566
# Warnings are treated as errors so this needs to be ignored:
582567
# >> warning C4275: non dll-interface class 'stdext::exception'
583568
# >> used as base for dll-interface class 'std::bad_cast'
584-
# This warning occurs only in VS2008, in VS2013 not.
585569
# This replacements must be unique for the undo operation
586570
# to be reliable.
587571

@@ -632,7 +616,7 @@ def build_wrapper_library_mac():
632616
# On Mac it is required to link libcef_dll_wrapper against
633617
# libc++ library, so must build this library separately
634618
# from cefclient.
635-
cmake_wrapper = prepare_build_command(build_lib=True)
619+
cmake_wrapper = prepare_build_command()
636620
cmake_wrapper.extend(["cmake", "-G", "Ninja",
637621
"-DPROJECT_ARCH=x86_64",
638622
"-DCMAKE_CXX_FLAGS=-stdlib=libc++",
@@ -664,56 +648,22 @@ def build_wrapper_library_mac():
664648
run_command(cmake_wrapper, build_wrapper_dir)
665649
print("[automate.py] cmake OK")
666650
# Ninja
667-
ninja_wrapper = prepare_build_command(build_lib=True)
651+
ninja_wrapper = prepare_build_command()
668652
ninja_wrapper.extend(["ninja", "-j", Options.ninja_jobs,
669653
"libcef_dll_wrapper"])
670654
run_command(ninja_wrapper, build_wrapper_dir)
671655
print("[automate.py] ninja OK")
672656
assert os.path.exists(wrapper_lib)
673657

674658

675-
def prepare_build_command(build_lib=False, vcvars=None):
676-
"""On Windows VS env variables must be set up by calling vcvarsall.bat"""
677-
command = list()
659+
def prepare_build_command():
660+
"""On Windows VS env variables must be set up by calling vcvarsall.bat."""
661+
command = []
678662
if platform.system() == "Windows":
679-
if build_lib:
680-
if vcvars == VS2010_VCVARS:
681-
# When using WinSDK 7.1 vcvarsall.bat is broken. Instead
682-
# env variables are queried using setuptools.msvc.
683-
return command
684-
if vcvars:
685-
command.append(vcvars)
686-
else:
687-
command.append(get_vcvars_for_python())
688-
command.append(VS_PLATFORM_ARG)
689-
else:
690-
if int(Options.cef_branch) >= 2704:
691-
command.append(VS2015_VCVARS)
692-
else:
693-
command.append(VS2013_VCVARS)
694-
command.append(VS_PLATFORM_ARG)
695-
command.append("&&")
663+
command.extend([VCVARS, VS_PLATFORM_ARG, "&&"])
696664
return command
697665

698666

699-
def fix_cef_include_files():
700-
"""Fixes to CEF include header files for eg. VS2008 on Windows."""
701-
# TODO: This was fixed in upstream CEF, remove this code during
702-
# next CEF update on Windows.
703-
if platform.system() == "Windows" and get_msvs_for_python() == "2008":
704-
print("[automate.py] Fixing CEF include/ files")
705-
# cef_types_wrappers.h
706-
cef_types_wrappers = os.path.join(Options.cef_binary, "include",
707-
"internal", "cef_types_wrappers.h")
708-
with open(cef_types_wrappers, "rb") as fp:
709-
contents = fp.read().decode("utf-8")
710-
# error C2059: syntax error : '{'
711-
contents = contents.replace("s->range = {0, 0};",
712-
"s->range.from = 0; s->range.to = 0;")
713-
with open(cef_types_wrappers, "wb") as fp:
714-
fp.write(contents.encode("utf-8"))
715-
716-
717667
def create_prebuilt_binaries():
718668
"""After building copy binaries/libs to build/cef_xxxx/.
719669
Not all projects may have been built on all platforms."""
@@ -866,25 +816,10 @@ def copy_app(app):
866816

867817

868818
def get_available_python_compilers():
869-
all_python_compilers = OrderedDict([
870-
("2008", VS2008_VCVARS),
871-
("2010", VS2010_VCVARS),
872-
("2015", VS2015_VCVARS),
873-
])
874-
ret_compilers = OrderedDict()
875-
for msvs in all_python_compilers:
876-
vcvars = all_python_compilers[msvs]
877-
if os.path.exists(vcvars):
878-
ret_compilers[msvs] = vcvars
879-
else:
880-
print("[automate.py] INFO: Visual Studio compiler not found:"
881-
" {vcvars}".format(vcvars=vcvars))
882-
return ret_compilers
883-
884-
885-
def get_vcvars_for_python():
886-
msvs = get_msvs_for_python()
887-
return globals()["VS"+msvs+"_VCVARS"]
819+
if not os.path.exists(VCVARS):
820+
print("[automate.py] ERROR: vcvarsall.bat not found: {}".format(VCVARS))
821+
return OrderedDict()
822+
return OrderedDict([("2015", VCVARS)])
888823

889824

890825
def getenv():

‎tools/build.py‎

Lines changed: 12 additions & 100 deletions
Original file line numberDiff line numberDiff line change
@@ -18,12 +18,13 @@
1818
from sources or use ready binaries from Spotify Automated Builds.
1919
2020
Usage:
21-
build.py VERSION [--rebuild-cpp] [--unittests] [--fast] [--clean] [--kivy]
22-
[--hello-world] [--enable-profiling]
23-
[--enable-line-tracing]
21+
build.py [VERSION] [--rebuild-cpp] [--unittests] [--fast] [--clean] [--kivy]
22+
[--hello-world] [--enable-profiling]
23+
[--enable-line-tracing]
2424
2525
Options:
26-
VERSION Version number eg. 50.0
26+
VERSION Version number eg. 50.0. Optional: defaults to
27+
{CHROME_VERSION_MAJOR}.0 read from src/version/.
2728
--unittests Run only unit tests. Do not run examples while
2829
building cefpython modules. Examples require
2930
interaction such as closing window before proceeding.
@@ -131,12 +132,14 @@ def command_line_args():
131132
REBUILD_CPP, VERSION, UNITTESTS
132133

133134
VERSION = get_version_from_command_line_args(__file__)
134-
# Other scripts called by this script expect that version number
135-
# is available in sys.argv, so don't remove it like it's done
136-
# for all other args starting with "--".
137135
if not VERSION:
138-
print(__doc__)
139-
sys.exit(1)
136+
cef_ver = get_cefpython_version()
137+
VERSION = "{major}.0".format(major=cef_ver["CHROME_VERSION_MAJOR"])
138+
# Inject into sys.argv so child scripts that call
139+
# get_version_from_command_line_args() can find it.
140+
sys.argv.insert(1, VERSION)
141+
print("[build.py] No version specified, defaulting to {v}"
142+
.format(v=VERSION))
140143

141144
print("[build.py] Parse command line arguments")
142145

@@ -415,97 +418,6 @@ def compile_cpp_projects_with_setuptools():
415418
shutil.copy(SUBPROCESS_EXE, CEFPYTHON_BINARY)
416419

417420

418-
def compile_cpp_projects_windows_DEPRECATED():
419-
"""DEPRECATED. Not used currently.
420-
Build C++ projects using .vcproj files."""
421-
422-
# TODO: Remove code after setuptools compilation was tested for some time
423-
424-
print("[build.py] Compile C++ projects")
425-
426-
print("[build.py] ~~ Build CLIENT_HANDLER vcproj")
427-
vcproj = ("client_handler_py{pyver}_{os}.vcproj"
428-
.format(pyver=PYVERSION, os=OS_POSTFIX2))
429-
vcproj = os.path.join(SRC_DIR, "client_handler", vcproj)
430-
build_vcproj_DEPRECATED(vcproj)
431-
432-
print("[build.py] ~~ Build LIBCEFPYTHONAPP vcproj")
433-
vcproj = ("libcefpythonapp_py{pyver}_{os}.vcproj"
434-
.format(pyver=PYVERSION, os=OS_POSTFIX2))
435-
vcproj = os.path.join(SRC_DIR, "subprocess", vcproj)
436-
build_vcproj_DEPRECATED(vcproj)
437-
438-
print("[build.py] ~~ Build SUBPROCESS vcproj")
439-
vcproj = ("subprocess_{os}.vcproj"
440-
.format(os=OS_POSTFIX2))
441-
vcproj = os.path.join(SRC_DIR, "subprocess", vcproj)
442-
ret = build_vcproj_DEPRECATED(vcproj)
443-
444-
# Copy subprocess executable
445-
subprocess_from = os.path.join(
446-
SUBPROCESS_DIR,
447-
"Release_{os}".format(os=OS_POSTFIX2),
448-
"subprocess_{os}.exe".format(os=OS_POSTFIX2))
449-
subprocess_to = os.path.join(CEFPYTHON_BINARY, "subprocess.exe")
450-
if os.path.exists(subprocess_to):
451-
os.remove(subprocess_to)
452-
if ret == 0:
453-
print("[build.py] Copy subprocess executable")
454-
# shutil.copy() will also copy Permission bits
455-
shutil.copy(subprocess_from, subprocess_to)
456-
457-
print("[build.py] ~~ Build CPP_UTILS vcproj")
458-
vcproj = ("cpp_utils_{os}.vcproj"
459-
.format(os=OS_POSTFIX2))
460-
vcproj = os.path.join(SRC_DIR, "cpp_utils", vcproj)
461-
build_vcproj_DEPRECATED(vcproj)
462-
463-
464-
def build_vcproj_DEPRECATED(vcproj):
465-
"""DEPRECATED. Not used currently."""
466-
467-
# TODO: Remove code after setuptools compilation was tested for some time
468-
469-
# In VS2010 vcbuild.exe was replaced by msbuild.exe.
470-
# Ufortunately WinSDK 7.1 does not come with msbuild.exe,
471-
# so it would be required to install Visual Studio 2010,
472-
# and to support both 32-bit ad 64-bit compilations it
473-
# a non-express version would have to be installed, which
474-
# is not free. So to make it free open-source it was
475-
# required migrate to a new compilation system that uses
476-
# distutils/setuptools packages.
477-
478-
# msbuild.exe flags:
479-
# /clp:disableconsolecolor
480-
# msbuild /p:BuildProjectReferences=false project.proj
481-
# MSBuild.exe MyProject.proj /t:build
482-
483-
VS2008_BUILD = ("%LocalAppData%\\Programs\\Common\\"
484-
"Microsoft\\Visual C++ for Python\\9.0\\"
485-
"VC\\bin\\amd64\\vcbuild.exe")
486-
VS2008_BUILD = VS2008_BUILD.replace("%LocalAppData%",
487-
os.environ["LOCALAPPDATA"])
488-
489-
if PYVERSION == "27":
490-
args = list()
491-
args.append(VS2008_VCVARS)
492-
args.append(VS_PLATFORM_ARG)
493-
args.append("&&")
494-
args.append(VS2008_BUILD)
495-
args.append("/nocolor")
496-
args.append("/nologo")
497-
args.append("/nohtmllog")
498-
if REBUILD_CPP:
499-
args.append("/rebuild")
500-
args.append(vcproj)
501-
ret = subprocess.call(args, shell=True)
502-
if ret != 0:
503-
compile_ask_to_continue()
504-
return ret
505-
else:
506-
raise Exception("Only Python 2.7 32-bit is currently supported")
507-
508-
509421
def compile_ask_to_continue():
510422
# noinspection PyUnboundLocalVariable
511423
what = input("[build.py] make failed, 'y' to continue, Enter to stop: ")

‎tools/build_cpp_projects.py‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -110,8 +110,6 @@ def print_compiler_options():
110110

111111

112112
def get_compiler(static=False):
113-
# NOTES:
114-
# - VS2008 and VS2010 are both using distutils/msvc9compiler.py
115113
compiler = new_compiler()
116114
# Must initialize so that "compile_options" and others are available
117115
compiler.initialize()

0 commit comments

Comments
 (0)