Skip to content

Commit e78a670

Browse files
authored
transpile: report a file: URL as a stripped script's sourceURL (#800)
* transpile: report a file: URL as a stripped script's sourceURL Node's type stripping emits pathToFileURL(filename).href as the `//# sourceURL` of a stripped script, and the inspector surfaces that value as Debugger.scriptParsed's `url` — the key editors and DevTools match breakpoints against. nub emitted the bare filesystem path, so every transpiled .ts/.cts/.mts reached them under a different script identity than the same file under node. The percent-encoding matches pathToFileURL byte for byte. Its set is wider than the WHATWG path percent-encode set — `[`, `]`, `^`, `|` and `~` are escaped too — so it is derived from a sweep against v26.7.0 rather than from the URL spec. CACHE_SCHEMA goes to "7": the sourceURL is baked into the cached body, so a "6"-era entry would keep serving the old identity. Upstream's test/parallel/test-inspector-strip-types.js now passes under nub on 4 of its 5 scenarios, matching plain node on the same checkout (the fifth fails identically under both). test-debugger-probe-typescript reports the right `url` and still differs on line/column, which is transpiled-position fidelity rather than script identity. * transpile: spell the sourceURL for the host Node's escape band The sourceURL is percent-encoded to match `pathToFileURL`, but that escape set is not fixed across nub's supported range. nodejs/node#54545 widened it to cover `[`, `]`, `^`, `|` and `~`, and it reached the release lines out of order, so it takes bands rather than one floor — measured on real binaries at each boundary tag: 18.x (incl. 18.20.8) narrow 20.18.3+ wide 20.0.0 - 20.18.2 narrow 22.12.0+ wide 21.x (incl. 21.7.3) narrow 23.1.0+ wide 22.0.0 - 22.11.x narrow 24.0.0+ wide 23.0.x narrow Emitting one fixed set gave one file two identities inside a single process on the narrow band: Node computes a module's own URL with its own `pathToFileURL`, so on v22.11 the loader carried `w[x]y^z~q/probe.ts` while `Debugger.scriptParsed` reported `w%5Bx%5Dy%5Ez%7Eq/probe.ts`, and a breakpoint set against the module URL matched no script. `transformCached` now takes `process.version` and derives the band from it. The URL FORM stays unconditional. Node's own type stripping only began reporting a `file:` URL in v26.4.0, backported to v24.19.0, and v25.x never got it; following the host there would flip nub's script identity on a patch bump. `url_band` joins the cache-key preimage, because the band changes the bytes that get stored: without it an `nvm use` across a boundary serves a body carrying the other band's spelling. The schema stays at "7" — both halves land together, and neither shipped. The URL helpers move to `nub-cache-key`, which exists so the cache's pure logic can be unit-tested off the `test = false` cdylib. Both `windows` branches are now covered against captured `pathToFileURL(p, { windows: true })` output. The integration test asserts the cold pass actually populated the cache directory, so a disabled cache fails instead of quietly running cold twice. * transpile: cover the host-band plumbing end-to-end, record Cause 7 fixed The integration fixture directories held only characters escaped the same way in both bands, so nothing exercised `process.version` → `transformCached`'s 9th argument → `host_widens_url_path` → the emitted bytes. A wrong value there fell through to widened and still passed on any modern host. The all-platform fixture directory is now `w[x]y^z~q`. Those four characters are band-sensitive and legal in a Windows filename, so the Windows leg exercises the same path; `|` is band-sensitive but illegal there, so it joins the unix-only fixture as `a b%c#d?e|f`. The fixture already derives its expectation from the host's own `pathToFileURL`, so this is a differential: forcing the 9th argument to a narrow-band version fails both tests on a v26.7.0 host. wiki/research/nub-v0.5-augmentation-regressions.md records Cause 7 as fixed, drops the stale emitter line reference for the current symbols, and keeps the original analysis. Two additions: the inspector script-identity symptom that shares the same string, and the two facts the original recommendation did not anticipate — the host-dependent escape set, and why the URL form is deliberately not host-gated. Both symptoms re-verified against the built binary. Coverage: with the bare path `nub --test --experimental-test-coverage` drops every TypeScript row and reports a fake 100%; with the fix `lib.ts | 100.00 | 100.00 | 50.00` matches plain node. * wiki: state which Cause 7 symptom carries a committed test
1 parent 874f842 commit e78a670

5 files changed

Lines changed: 550 additions & 28 deletions

File tree

‎crates/nub-cache-key/src/lib.rs‎

Lines changed: 51 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,39 @@
1-
//! Transpile-cache key derivation, factored out of `nub-native` so the
2-
//! invalidation contract is unit-testable (see this crate's Cargo.toml for why a
3-
//! test can't live in the cdylib `nub-native`).
1+
//! The transpile cache's pure core, factored out of `nub-native` so it is
2+
//! unit-testable (see this crate's Cargo.toml for why a test can't live in the
3+
//! cdylib `nub-native`): the key derivation here, and in [`source_url`] the
4+
//! `//# sourceURL` spelling the key has to cover.
45
//!
56
//! The key preimage is NUL-separated and order-fixed (no trailing separator):
6-
//! `nub_version \0 schema \0 build_id \0 source \0 ext \0 tsconfig_hash \0 pkg_type \0 filename`
7+
//! `nub_version \0 schema \0 build_id \0 source \0 ext \0 tsconfig_hash \0 pkg_type \0 filename \0 url_band`
78
//! → blake3 → 64-hex lowercase → the cache FILENAME. Every component is hashed
89
//! IN, so a change to ANY of them (a nub release, a CACHE_SCHEMA bump, a rebuild
910
//! at a new build-id) yields a disjoint filename — old on-disk entries are
1011
//! silently ignored (a miss), never mis-read. `nub-native` calls `cache_key` with
1112
//! its `NUB_VERSION` / `CACHE_SCHEMA` / `BUILD_ID` consts.
1213
//!
1314
//! `filename` is in the preimage because the cached body carries a per-file
14-
//! `//# sourceURL=<absolute path>` magic comment: two byte-identical sources in
15+
//! `//# sourceURL=<file: URL>` magic comment: two byte-identical sources in
1516
//! the same directory would otherwise collide on one entry and the second file
1617
//! would be served the first's `sourceURL`, misattributing V8 stack frames and
1718
//! debugger source mapping (issue #171).
19+
//!
20+
//! `url_band` is in the preimage for the same reason one step further out: that
21+
//! sourceURL is spelled to match the HOST Node's `pathToFileURL`, whose escape set
22+
//! changed mid-release-line ([`source_url::host_widens_url_path`]). Without it, a
23+
//! project built under one Node and then run under another — an `nvm use`, or
24+
//! `nub node pin` — would be served a body carrying the other band's spelling.
25+
26+
pub mod source_url;
1827

1928
/// blake3 of the NUL-separated key preimage → 64-hex lowercase.
2029
///
2130
/// `nub_version` / `schema` / `build_id` are the cross-build invalidation
2231
/// components (compile-time consts on the production path); `source` / `ext` /
2332
/// `tsconfig_hash` / `pkg_type` / `filename` are the per-file inputs. `filename`
2433
/// is keyed in because it is baked into the cached body as the `//# sourceURL`
25-
/// comment, so distinct files must not share an entry (issue #171).
34+
/// comment, so distinct files must not share an entry (issue #171); `url_band`
35+
/// ([`source_url::band_tag`]) is keyed in because that same comment is spelled to
36+
/// match the host Node, so one file has two valid bodies across hosts.
2637
#[allow(clippy::too_many_arguments)]
2738
pub fn cache_key(
2839
nub_version: &str,
@@ -33,6 +44,7 @@ pub fn cache_key(
3344
tsconfig_hash: &str,
3445
pkg_type: &str,
3546
filename: &str,
47+
url_band: &str,
3648
) -> String {
3749
let mut h = blake3::Hasher::new();
3850
h.update(nub_version.as_bytes());
@@ -50,6 +62,8 @@ pub fn cache_key(
5062
h.update(pkg_type.as_bytes());
5163
h.update(b"\0");
5264
h.update(filename.as_bytes());
65+
h.update(b"\0");
66+
h.update(url_band.as_bytes());
5367
h.finalize().to_hex().to_string()
5468
}
5569

@@ -63,9 +77,12 @@ mod tests {
6377
const TSCONFIG: &str = "tsconfig-hash";
6478
const PKG: &str = "module";
6579
const FILENAME: &str = "/proj/src/x.ts";
80+
const BAND: &str = "url-wide";
6681

6782
fn key(version: &str, schema: &str, build_id: &str) -> String {
68-
cache_key(version, schema, build_id, SRC, EXT, TSCONFIG, PKG, FILENAME)
83+
cache_key(
84+
version, schema, build_id, SRC, EXT, TSCONFIG, PKG, FILENAME, BAND,
85+
)
6986
}
7087

7188
/// A rebuilt binary (new build-id) must not serve a prior build's entries:
@@ -115,6 +132,7 @@ mod tests {
115132
TSCONFIG,
116133
PKG,
117134
"/proj/alpha.ts",
135+
BAND,
118136
);
119137
let beta = cache_key(
120138
"0.0.1",
@@ -125,10 +143,36 @@ mod tests {
125143
TSCONFIG,
126144
PKG,
127145
"/proj/beta.ts",
146+
BAND,
128147
);
129148
assert_ne!(
130149
alpha, beta,
131150
"identical content at different paths must yield distinct cache keys"
132151
);
133152
}
153+
154+
/// The cached body carries a sourceURL spelled for the host Node's escape
155+
/// band, so the same file under two hosts must not share one entry — this is
156+
/// what stops `nvm use` from serving the other band's spelling.
157+
#[test]
158+
fn cache_key_distinguishes_the_url_bands() {
159+
let wide = cache_key(
160+
"0.0.1", "6", "abc1234", SRC, EXT, TSCONFIG, PKG, FILENAME, "url-wide",
161+
);
162+
let narrow = cache_key(
163+
"0.0.1",
164+
"6",
165+
"abc1234",
166+
SRC,
167+
EXT,
168+
TSCONFIG,
169+
PKG,
170+
FILENAME,
171+
"url-narrow",
172+
);
173+
assert_ne!(
174+
wide, narrow,
175+
"the two sourceURL escape bands must yield distinct cache keys"
176+
);
177+
}
134178
}

0 commit comments

Comments
 (0)