Conversation
jrandolf
changed the title
Describe executables and ZIPs with one private application specification, and share environment preparation, storage ownership and packaging across ordinary binaries, legacy executable ZIPs and py_zipapp entry points. Use native POSIX exec to preserve the application PID, signals and terminal behavior, with a detached watcher for temporary-file cleanup. On Windows, wait for the application child and its console cleanup before removal. Publish complete persistent images without replacing concurrent entries. Preserve public providers, custom templates and stage-two behavior. Keep historical raw-template adapters available and support Python 3.9 ZIP loaders. Document cleanup lifetime and legacy compatibility limitations. Validation: 727 root targets pass; the launcher matrix passes 313 cases with two skips. Native checks pass on macOS, Windows and Linux arm64/x64, including final loader/ZIP checks on all four pools. Changed-file hooks, documentation, distribution and independent implementation audit pass.
70c2c88 to
7fac02f
Compare
|
This PR, both description and diff, are quite large. It's going to take a long while to digest this. Just skimming it, I'm inclined to think all the additional plumbing and complexity is necessary. What problem is trying to be solved, exactly? The issue where zipapps, because extraction is handled outside of the main binary, don't use exec() and can't properly forward on signals? |
|
Forgot to say: if the goal is to fix the signal handing of zipapps, I think an easier way is to have the stage1 bootstrap perform unzip, but push cleanup into stage2. This allows using exec() in stage1. The zip directory to cleanup can be passed by using e.g. the |
|
Yes, that’s the original failure: when the launcher keeps a parent process around to remove temporary files, a signal sent to the launcher PID can leave the Python application running. It also affects ordinary binaries that create a temporary venv at startup. Your stage1-extract → exec → stage2-cleanup approach could fix the signal-delivery problem with a much smaller change, but the extra requirement I was trying to cover was removing temporary files even when Python’s cleanup doesn’t run: for example, an unhandled SIGTERM, os._exit(), or the application exec’ing another program. That motivated the external watcher. Stage-two cleanup would also need to preserve files through application exit handlers and thread shutdown. I combined that lifecycle work with a broader packaging/cache refactor, which made the PR much harder to review. I can split those changes out. For the focused fix, would cleanup on normal interpreter shutdown be sufficient, or should cleanup also cover exits that bypass Python’s shutdown handlers? That would help settle whether a watcher belongs in this change. P.S. We created something called bound. This could actually resolve all the problems here and streamline the binary significantly. We use this internally for all our rules (we have a custom JS ruleset as well as some custom py rulesets using bound.) We're open to extending it to more OSes/arches if needed. |
This branch has not been deployed
2 participants
Temporary virtual environments and executable ZIPs currently leave a launcher
process in front of Python so it can remove files afterward. A termination signal
sent directly to that PID can leave the application running and skip its cleanup.
The different executable and ZIP startup paths also disagree about interpreter
arguments, cache identity, and ownership during failed or concurrent preparation.
Describe the application once, then launch the same prepared image through an
ordinary executable, a legacy executable ZIP, or either
py_zipapp_binaryentrypoint. On POSIX the application replaces the launcher in its original PID, keeping
native signal delivery, process groups, terminal input, and exit status. A detached
watcher removes only temporary files after that PID exits. Windows starts and
reaps the application child before removal, allowing the application's own
Ctrl-C cleanup to finish without forwarding another interrupt.
The implementation separates the private application description, environment
preparation, storage ownership, process launch, and thin entry adapters:
virtual environment, entry point, and explicit links. Both ZIP rule families
use the same packager. Ordinary builds keep their runfiles without copying the
dependency tree.
honoring cancellation. The bundled ZIP shell reaps its preparation children
and validates a fixed six-field result before native execution.
relative directory symlink to an owned backing directory. Publication cannot
replace an existing entry, including one created concurrently. Content, modes,
layout, bootstrap code, and resolved runtime facts contribute to identity.
Completed bundled ZIP cache hits can execute directly from the shell.
precompilation, and coverage behavior. Custom templates preserve their bytes
and historical substitutions; the original raw templates remain usable by
older expanders. Public-only executable providers retain a compatibility path.
Validation uses Bazel 9.1.1 and
--config=fast-testsfor test targets:matrix has 315 cases (313 passed, two skipped), passing under macOS Bash 3.2
and Bash 5.3.20. The final 315-case matrix also passes on Bazel 7.7.1 in
WORKSPACE mode and Bazel 9.1.1, each with the same two expected skips.
targets, Linux arm64 13/13, Linux x86_64 13/13, and Windows x86_64 7/7
passed. Final affected custom/public-provider checks also pass on all four
platform pools.
Execution logs distinguish actual remote test execution from cache hits.
regression covers ZIP loaders that predate
exec_module; all ten affectedlocal ZIP/template targets pass. The final loader and packaged ZIP follow-up
also passes three targets on each of the four native Buildbarn pools.
finally,inherited ignored signals and masks, terminal job control, nesting, arguments,
wrapper runtimes, setup/exec failures, cache publication races, Windows console
handling and exit status, custom templates, and watcher readiness/descriptor
lifetime. Native macOS kqueue and Linux subreaper checks pass. Linux pidfd
and procfs checks pass in a separate Debian guest where those APIs are available;
these backend proofs reject fallback and verify owned-child/descriptor lifetime.
module: 11/11. The Sphinx check privately registers its required Python 3.11;
no dependency configuration changes are included.
documentation and distribution builds pass. Original ordinary and ZIP launchers
reproduce the direct-PID termination failure; the new launchers preserve the
PID, execute application cleanup, and remove their temporary files.
Temporary execution costs one resident watcher, measured at roughly 4.7–6.8 MiB
RSS locally. Runtime-created virtual environments also enter Python for
preparation. In the latest six-sample, 500-file comparison on a loaded shared
host, baseline/current startup medians were 75/73 ms for prepared executables,
144/289 ms for runtime-created venvs, 1724/1559 ms for cold ZIPs, and 77/74 ms for
warm ZIPs. Earlier quieter runs varied substantially; these are measurements,
not performance guarantees. Both builds spawn three build actions; the new
description adds three internal metadata writes.
Removal is asynchronous and follows the original interpreter, including exec;
it does not extend to descendants that outlive it. Namespace PID 1 and subreapers
can adopt the watcher, and namespace/cgroup shutdown or SIGKILL before setup
registration can prevent cleanup. A persistent extract root avoids the temporary
watcher where supported. Its published backing directories belong to the caller;
removing only a cache symlink does not reclaim the backing image. These lifetime
limits and cache cleanup semantics are documented.
Older custom rules exposing only
PyExecutableInfouse the unchanged raw-templateadapter. Its Windows persistent extraction can fail when a repeated launch meets
an existing directory symlink. This also reproduces with the original ZIP rule,
templates, and packaging tools; normal native launch and archive structure pass
in both versions. Temporary extraction is the documented workaround for that
compatibility path. The new default image path passes repeated persistent-launch
checks on Windows.