Skip to content

perf: avoid copying complete asset buffers - #71

Merged
chenjiahan merged 3 commits into
mainfrom
chenjiahan/perf-memfs-buffer-copy
Sep 29, 2026
Merged

chenjiahan merged 3 commits into
mainfrom
chenjiahan/perf-memfs-buffer-copy

Conversation

@chenjiahan

@chenjiahan chenjiahan commented Sep 28, 2026 •

Copy link
Copy Markdown
Member

Reuse complete Rspack asset buffers in the default memfs output filesystem, avoiding a second copy and unused growth capacity. Pin memfs/fs-core to 4.79.0 because this uses the private _setBuf API; callers must not mutate buffers after complete writes.

In a real Rspack + middleware benchmark (6 processes per variant, 100 rebuilds each), median results were:

Workload Rebuild time CPU time Peak RSS File buffer capacity
32 × 4 MiB 9.600 → 6.870 ms (−28.4%) 40.379 → 31.088 ms (−23.0%) 921.4 → 844.5 MiB (−8.3%) 128 → 128 MiB
32 × 5 MiB 11.177 → 7.725 ms (−30.9%) 44.899 → 34.279 ms (−23.7%) 1118.0 → 996.4 MiB (−10.9%) 256 → 160 MiB (−37.5%)

Measured on Apple M5 Max, Node 24.19.0, Rspack 2.2.2, and memfs/fs-core 4.68.2, before the dependency upgrade to 4.79.0. These synthetic asset workloads measure output-heavy rebuilds, not application HMR.

Related: web-infra-dev/rsbuild#8595

@chenjiahan
chenjiahan marked this pull request as ready for review September 29, 2026 02:57
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-29T03:01:12.721859Z 9cdffc2 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chenjiahan
chenjiahan merged commit e0e4433 into main Sep 29, 2026
5 checks passed
@chenjiahan
chenjiahan deleted the chenjiahan/perf-memfs-buffer-copy branch September 29, 2026 02:58

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9cdffc250a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +25 to +27
/** @type {{ _setBuf(buffer: Buffer): void }} */ (
/** @type {unknown} */ (this)
)._setBuf(buffer);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve write-buffer snapshot semantics

When a plugin or other caller reuses or mutates a Buffer after a synchronous write returns (or after an asynchronous write's callback), _setBuf(buffer) leaves the file backed by that same mutable storage. A later buffer.fill() therefore silently changes the bytes served by the middleware without another filesystem write, metadata update, or watcher event; the previous Node.write path copied the bytes. Because this hook applies to every complete write rather than only buffers whose ownership Rspack can guarantee, the filesystem must preserve the normal post-write snapshot semantics.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant