Repository navigation
Reduce CI duration #167
Description
Activity
Now even a single CI workflow takes around 27 minutes.
In my opinion, we can consider checking
.rstfilesusingwhen running CI. On the other hand, CD should be kept as unchanged as it requires full html build.sphinx-build -b gettextAfter investigating the
Makefile, I found it is more practical to change the build target to "text". It is not elegant because it occupies extra space for text files. However, it effectively reduces the validation time. In the future we can further optimizations.Is this issue specific to the zh-tw translations? Have you checked if other Python translation projects have the same issue, and if/how they fixed it?
CPython repo has a Makefile for doc build, and all translation repos that run doc build with GitHub Action depend on it. It provides different build modes and we currently setup our CI action to run a full doc build with the
autobuild-dev-htmlmode, which is the sphinx-nativehtmlmode with-E(always rebuild completely) and-a(write all output files) options. We've tried outtext&htmlmode locally and they do much faster thanautobuild-dev-html, and that's why we believe the-Eaoption increase the duration and changing the build mode would probably be a decent solution for us.Here are some metrics from those translation repos that have adopted GitHub Action. French repo also builds with
autobuild-dev-htmlbut I have no idea why their actions are still blazing fast like the regularhtmlmode...language build mode in GitHub Action translation progress (calculated with potodo) GitHub Action appr. duration (min) Spanish html 95.73% @3.11 4 - 7 Simplified Chinese html 81.16% @3.10 7 - 9 Turkish html 1.27% @3.11 4 - 5 Brazilian Portuguese html 43.46% @3.12 5 - 6 French autobuild-dev-html 48.14% @3.11 2 - 4 Japanese html 77.09% @3.11 7 - 9 Polish html & latex 4.2% @3.12 2 - 3 for html part Ukrainian html & latex 84.48% @3.12 3 - 5 for html part Traditional Chinese autobuild-dev-html 28.34% @3.11 ~25 Reacted by cschan and Ezio MelottiI think we can have a simple but effective solution first, then keep working on optimizations.
Reacted by Ezio Melotti@ken71301
As we mentioned at the sprint, please refer to the discussion above.My untested thoughts:
- New workflow (e.g. daily) to do the full rebuild (
make all), and utilize the cache action to cache the build artifacts. - PR CI workflow restores the cached artifacts, and try building for each updated .po files (e.g.
for filename in <find-updated-files>; do make build $$filename; done) -> expected to be much faster than full build
- New workflow (e.g. daily) to do the full rebuild (
@mattwang44
It seems like FR also use html mode now.
python/python-docs-fr#426In latest sprint Josix and me just tried load of setting and it all seems not effective at all.
So I start believing that cache might be our next option, though I still need to check other's test flow to make sure how we can do it.Reacted by W. H. Wang and Ezio MelottiWith this patch (issue) released along with Sphinx 8.1, the build duration in CI has significantly dropped from ~25 mins to ~4 mins 🚀. I believe doc build time would then not be an issue for us. I'll close this issue if no objections in the following days.
Since we currently run a complete HTML build process in CI, the CI has an unacceptable duration (~16minutes).
Possible solutions: