Faster website build - #3660
Faster website build#3660Jake Bailey (jakebailey) wants to merge 14 commits into
Conversation
Bootstrap already builds the workspace packages. Repeating the build in CI and deployment adds about 45 seconds on the Linux runner without producing anything new.
Theme explanations dominate highlighting time but are only consumed by TSConfig's JSON property hovers. Keep those scopes without computing unused explanations for every other language, and reuse tokenization for repeated examples without sharing mutable tokens. All 920 localized page-data files remain identical. Rendered HTML is unchanged apart from existing random links and build identifiers.
Several package bootstrap scripts are aliases for their build, which runs again at the end of workspace bootstrapping. TSConfig generation also repeats work already done by its build. Do the metadata setup and one topologically ordered build instead, preserving the package-level bootstrap commands for direct use. Compiled package outputs are unchanged.
Reusing compiled examples across builds must not confuse identical code in different languages or with different compiler settings. Include those inputs and the VFS version before enabling persistent CI reuse.
Production Gatsby builds spend most of their time compiling examples and recreating Webpack bundles. Reuse that work across compatible builds without restoring Gatsby's page-query cache or generated pages. Keep translations, metadata, and playground cache busting fresh. A fully localized local site build took 253 seconds without caches and 31 seconds restoring only Twoslash and Webpack work, including the production cache-busting step. All 920 page queries still run. Separate build steps also make future Actions bottlenecks visible.
Restored snapshots carry obsolete examples forward indefinitely unless entries are explicitly evicted. Record usage on successful cache reads and prune the least-recently-used entries to a 64 MiB budget before saving, preserving usage timestamps across archive roundtrips.
GitHub cache archives are immutable, so repeated builds of the same commit cannot otherwise save newly compiled examples or refreshed usage order. Restore the latest compatible snapshot and save a pruned replacement under a unique run-and-attempt key as soon as the site build succeeds, even if subsequent tests or deployment fail.
Playground changes should not force unchanged documentation examples to compile again. Hashing workspace sources, including generated inputs, makes those iterations cold. Broader snapshot reuse requires example-level checks of compiler implementations and filesystem dependencies. Package versions alone cannot detect rebuilt workspace compilers or changed declarations, and missed module lookups must be reconsidered when new files appear.
Regenerating the same reference after the CLI documentation adds eight seconds to package preparation on CI without changing its output.
CI already builds every package before building the site. Repeating the generators during tests adds 42-54 seconds without providing another check. Keep standalone tests unchanged. Validate generated documentation before running independent package tests with bounded concurrency, avoiding the unrelated dependency barriers in the test phase.
Documentation linting compiles examples with different settings from the site build. Saving only before tests discards that work, costing 65-76 seconds on each CI run. Save a second bounded snapshot after successful tests with a distinct immutable key. Keep the earlier checkpoint so failed tests still preserve the completed site build.
The website and package tests do not need to share a runner or wait for each other. Run both operating-system matrices independently so package validation no longer extends the site build critical path. Keep Gatsby-dependent checks and documentation validation with the site, retain every required check, and preserve the CI workflow and site artifact consumed by preview deployment. No additional interbuild caches are needed.
|
Azure Static Web Apps: Your stage site is ready! Visit it here: https://victorious-plant-05c166c10-3660.centralus.5.azurestaticapps.net |
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
It changes dependency internals and CI, publishing, and production build behavior across multiple platforms.
Review effort: Balanced
Findings: None
What changed in this PR
Improves website build performance through reusable Twoslash/Webpack caches and streamlined build and CI workflows.
Changes:
- Adds validated Twoslash and syntax-highlighting caches with pruning and tests.
- Splits CI validation and integrates website cache restoration/saving.
- Removes redundant package build and documentation-generation work.
| File | Description |
|---|---|
pnpm-lock.yaml |
Registers updated dependency patches. |
patches/remark-shiki-twoslash@3.1.3.patch |
Adds persistent Twoslash caching. |
patches/gatsby-remark-shiki-twoslash@3.0.38.patch |
Caches highlighting tokens. |
packages/typescriptlang-org/lib/bootup/shiki.test.js |
Tests highlighting, caching, and pruning. |
packages/tsconfig-reference/package.json |
Removes duplicate Markdown generation. |
package.json |
Streamlines bootstrap and splits CI tests. |
.github/workflows/publish-packages.yml |
Removes a redundant build. |
.github/workflows/deploy-prod-static.yml |
Adds build caches and simplifies artifact upload. |
.github/workflows/CI.yml |
Separates site and package validation. |
.github/actions/website-cache/action.yml |
Restores website build caches. |
.github/actions/website-cache/save/action.yml |
Prunes and saves caches. |
.github/actions/website-cache/prune-twoslash-cache.js |
Implements bounded LRU pruning. |
Files not reviewed (1)
- pnpm-lock.yaml: Generated file
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
This is all kind of misc crap, but, saves a significant amount of time in builds over time. If we're redoing the site none of this matters, but, the extra caching does really help before then. |
| "generate:md:cli": "tsc --build && node scripts/cli/generateMarkdown", | ||
| "generate:md:msbuild": "tsc --build && node scripts/msbuild/generateMarkdown", | ||
| "generate:md": "pnpm generate:md:tsconfig && pnpm generate:md:cli && pnpm generate:md:tsconfig", | ||
| "generate:md": "pnpm generate:md:tsconfig && pnpm generate:md:cli", |
There was a problem hiding this comment.
Wow, what?
Junky attempt to speed up the build without changing the site