Repository navigation
Speeding up the Docs / Docs CI job #459
Description
Activity
cc @ned-deily re: #435
I would suggest moving to
actions/setup-pythonfor the Docs job, but I don't know if there's been prior discussion3 on this that I haven't been able to find, or if there are other reasons it makes sense to trade off speed for building Python from HEAD.actions/setup-pythonalso suggested at #429 (comment), with three 👍.
The big question is: do we need to build docs against Python from HEAD?
- One reason to build from source (or with a recent build): to see deprecations.
If not, I think using
actions/setup-pythonis a great idea.And we have options of what Python to test with [with approx setup timings]:
- [~2s] Default, a recent Python 3 release (as in the patch, currently 3.10.4)
- [~2s] Explicitly setting a stable version e.g.
3.10 - [~14s] Explicitly setting a dev version e.g.
3.11-devis the latest alpha/beta/RC release - [~32s] Alternatively, use https://github.com/deadsnakes/action to test a nightly build, e.g.
3.11-dev,3.12-dev(for example)
Building Python from source takes about 141s, so the above options are quite a bit quicker.
But the patch saves much more time than ~141s in setup though.
- One reason is it splits the two build into two parallel jobs, docs build on
actions/setup-python, and doctest on HEAD (as new syntax doesn't exist in the latest stable release).
The real big saving:
- The docs build ("Build HTML documentation" step) with
actions/setup-pythontakes only ~2m compared to 6-15m with Python on HEAD!
Reacted by Adam TurnerThanks for finding the comment from #429!
One reason to build from source (or with a recent build): to see deprecations
I think this is a less compelling reason -- the Sphinx project tests with the latest development builds (currently deadsnakes' 3.11-dev nightly builds), so will catch deprecations there. Were there non-Sphinx deprecations you were referring to? (The link was to the collapsed summary page, so I'm not sure which deprecation warnings you're talking about).
A
doctest in CI job should be run with HEAD Python.
Reacted by Zachary WareI think this is a less compelling reason -- the Sphinx project tests with the latest development builds (currently deadsnakes' 3.11-dev nightly builds), so will catch deprecations there. Were there non-Sphinx deprecations you were referring to? (The link was to the collapsed summary page, so I'm not sure which deprecation warnings you're talking about).
Agreed, if they're just from third-party libraries it's not important. Much better to speed up the builds.
Reacted by Adam Turnerpython/cpython#93736 has been merged, thanks all.
A
Reacted by Alex Waygood, Hugo van Kemenade and Ezio Melotti
The short story
I think the Docs CI job can be faster, which would improve the contribution and review experience.
Long version
In the documentation team meeting we've been talking about a few workflow improvements. The Docs / Docs CI job is quite slow, taking around 15-20 minutes. From some quick testing, it seems this is largely due to that we use Python from HEAD and build Python in the job1, rather than a build with optimisation etc. Tests of my speed-up patch reduce the core docs job to ~3m15s2.
As a reviewer, a green tick is a quick signal that the syntax of the documentation is probably correct, and I don't need to be as thorough, whereas if the build fails I can look in the logs and point out the error and a potential fix to the proponenet of the PR.
I would suggest moving to
actions/setup-pythonfor the Docs job, but I don't know if there's been prior discussion3 on this that I haven't been able to find, or if there are other reasons it makes sense to trade off speed for building Python from HEAD.A
Footnotes
https://github.com/python/cpython/actions/workflows/doc.yml ↩
Patch, Run 1, Run 2, Run 3. ↩
Find a way to avoid redundant CI workflow builds of cpython just to test Docs #435 discusses not building Python, but in the context of reusing another build from HEAD rather than an optimised version. ↩