Repository navigation
Improvements to Emscripten build script #145219
Description
Activity
@freakboy3742 I can make these changes but I want to wait until #145177 is merged to avoid conflicts.
Reacted by Russell Keith-MageeI pushed fixes for 1 and 2 on https://github.com/hoodmane/cpython/tree/configurable-cross-build-dir on top of #145177.
@freakboy3742 for caching libraries, I'm thinking maybe we do the following:
- Move
emsdkfrom<emsdk-cache>/<version>/emsdk_env.shto<emsdk-cache>/<version>/emsdk/emsdk_env.sh - If
--emsdk-cacheis provided, put the prefix at<emsdk-cache>/<version>/prefix.
Sound reasonable to you?
- Move
Yes, I think that makes sense.
An additional suggestion: For consistency with other builds, we should simplify the
<cross-build>/wasm32-emscripten/pythondirectory to just be the Python build (i.e., build Python in<cross-build>/wasm32-emscripten), use<emsdk-cache>/<version>/mpdecimal-x.y.zas the location for a build for mpdecimal if a cache is provided, or<cross-build>/dependencies/wasm32-emscripten/mpdecimal-x.y.zif it isn't, and shortcut dependency builds if that directory already exists.That would allow build artefacts for mpdecimal and libffi to persist between builds, increases consistency between Android, iOS and Emscripten build structures, and in particular makes the cleaning strategy consistent.
(I guess strictly we should be looking for one of the the installed products in
<prefix>rather than the build directory as the shortcut trigger; I'm not sure if it will make that much difference in practice)- added a commit that references this issue
on Mar 9, 2026 For consistency with other builds, we should simplify the
<cross-build>/wasm32-emscripten/pythondirectory to just be the Python build
<cross-build>/dependencies/wasm32-emscripten/mpdecimal-x.y.zI'm confused about this: it looks to me like both android and iOS have their prefix under
<cross-build>/<host>/prefixand don't make a dependencies directory at all? I do see that they build Python directly at<cross-build>/<host>but that seems weird given thatprefixis underneath it and has nothing directly to do with the Python build.in particular makes the cleaning strategy consistent.
Seems to me that cleaning is simplest if everything about
wasm32-emscriptenstays undercross-build/wasm32-emscripten?For consistency with other builds, we should simplify the
<cross-build>/wasm32-emscripten/pythondirectory to just be the Python build
<cross-build>/dependencies/wasm32-emscripten/mpdecimal-x.y.zI'm confused about this: it looks to me like both android and iOS have their prefix under
<cross-build>/<host>/prefixand don't make a dependencies directory at all? I do see that they build Python directly at<cross-build>/<host>but that seems weird given thatprefixis underneath it and has nothing directly to do with the Python build.The difference is that Android and iOS don't have to build their dependencies as part of the CPython build. The build of mpdecimal-1.0.0 can be shared across all builds of all versions of CPython; there's no analog of the EMSDK version dependency. As a result, they both use downloads; the
prefixis just the unpacked version of the downloads; and the downloads can be cached outside the build tree.What I'm trying to optimize here is needing to do a full rebuilt of mpdecimal, libffi et al on every rebuild of wasm32-emscripten.
in particular makes the cleaning strategy consistent.
Seems to me that cleaning is simplest if everything about
wasm32-emscriptenstays undercross-build/wasm32-emscripten?As long as what we clean up doesn't mean re-building the same version of mpdecimal with the same verision of EMSDK.
I guess we could do the build for mpdecimal somewhere into the emsdk directory, install it into a standalone prefix, then zip up that artefact and cache it in the downloads folder; then, for the actual build, you look for the download, unpack it into the "live" prefix, and proceed as normal.
Just reviewed the PR - that approach looks good to me. I've flagged one more cleanup that I'd like to make, but it requires a parallel build in the buildbot, so I suspect that will be easier to make when we get to the "move to Platforms" PR.
- added a commit that references this issue
on Mar 11, 2026 - added a commit that references this issue
on Mar 11, 2026
@freakboy3742's desired improvements to the Emscripten build script:
cleanshould be a little more selective. It currently destroys all ofcross-build; it should only be purging the parts of cross-build that are emscripten specific (so that Emscripten, iOS and Android builds can co-exist). This is admittedly a "me" problem... but it's a problem I have :-)cross-builddirectory name should be configurable. Again, this is somewhat a "me" problem... but I currently need to have cross builds for 3.10-3.15, and they can't currently co-exist.Linked PRs