Self-contained handoff doc. Generated 2026-07-17 from the latest committed run.
This repo (elide-dev conformance testsuite) runs upstream conformance suites
against pinned Elide builds and publishes versioned
reports. The wpt-wintertc suite is a sparse Web Platform Tests slice covering
WinterTC / ECMA-429 JavaScript-facing APIs: encoding (40 files), url
(28 files), and fetch (245 files). The manifest is
manifests/wintertc-wpt-2025.toml; the WPT checkout is a sparse submodule
under suites/wpt.
This suite has the single largest and most concentrated compat wins across all five suites in the repo.
| pass | fail | error | skip | total | pass rate (excl. skips) |
|---|---|---|---|---|---|
| 10,480 | 3,422 | 12 | 661 | 14,575 | 75.3% |
Counts are WPT subtests (file :: subtest name). Latest reports:
reports/1.4.1+20260716.b8b6531/3d3ea83ed640/wpt-wintertc/impact.md— failures clustered by root-cause signature, largest first.../wpt-wintertc/impact.json— same data, machine-readable (bySignature).../wpt-wintertc/wpt-wintertc.md— full run reportexpectations/wpt-wintertc.ratchet.toml— the known-failure baseline (3,438 lines)
bun run setup # once: deps + submodules + sparse checkouts
# smoke (seconds):
bun run testsuite --elide nightly --suite wpt-wintertc --include 'url/urlsearchparams-constructor.any.js' --log
# a target area:
bun run testsuite --elide nightly --suite wpt-wintertc --include 'encoding/textdecoder-fatal-single-byte.any.js' --log
# by name, any file whose path contains 'encodeInto' (case-insensitive glob, see docs/running.md 'Targeted runs'):
bun run testsuite --elide nightly --log --filter 'wpt-wintertc:*encodeinto*'
# full suite (~12s of harness runtime inside Docker):
bun run testsuite --elide nightly --suite wpt-wintertc --threads 8
# after fixes land in an Elide nightly, re-baseline:
bun run testsuite --elide nightly --suite wpt-wintertc --ratchetGreen = zero regressions vs the expectations baseline; new passes show up in
changes.md and are locked in by --ratchet.
Failures are extremely concentrated. Directory rollup:
| failing | where | root cause |
|---|---|---|
| 1,158 | fetch/api/** |
no WPT server: relative URLs fail (Failed to parse URL: ../resources/...), promise_test unhandled rejections |
| 1,092 | encoding/textdecoder-fatal-single-byte.any.js |
missing WHATWG single-byte encodings + wrong fatal-mode behavior |
| 342 | encoding/legacy-mb-schinese/** |
gbk/gb18030 decoder tables wrong/missing |
| 136 | fetch/fetch-later/** |
fetchLater not defined (browser-only API; also needs window/iframe) |
| 136 | fetch/http-cache/** |
needs WPT server |
| 105 | url/url-setters-stripping.any.js |
URL setters don't strip U+0000/tab/newline per spec |
| 66 | encoding/encodeInto.any.js |
SharedArrayBuffer ctor missing; detached-buffer handling |
| 51 | encoding/textdecoder-mistakes.any.js |
decoder error-recovery divergences |
| ~350 | rest of encoding/*, url/*, fetch/* |
mixed, mostly encoding-related |
Signature-level highlights (see impact.md for the full list):
TextDecoder: encoding ISO-8859-10 / -14 not available on this JVM— 524TextDecoder: encoding macintosh not available on this JVM— 262TextDecoder: encoding ISO-8859-8-I not available on this JVM— 226decoding error (fatal mode): UNMAPPABLE[n](should NOT throw) — 68- throws
RangeErrorwhere spec wantsTypeError(fatal-flag tests) — 47 sabConstructor is not a constructor(no SharedArrayBuffer) — 71- URL:
assert_equals: property expected "https:" but got "http:"etc. — 80;Invalid URL port— 24
Root cause for the encoding mass: Elide's TextDecoder delegates to JVM
Charset. The JVM lacks several WHATWG-required encodings (ISO-8859-10,
ISO-8859-14, ISO-8859-8-I, macintosh, x-mac-cyrillic, x-user-defined; label
aliases like unicode11utf8) and diverges from the Encoding Standard on
error handling (fatal mode, replacement-character emission, ISO-2022-JP state
machine).
- WHATWG single-byte encodings (Elide runtime) — unlocks ~1,400
subtests, complexity low-medium. Implement the Encoding Standard's
single-byte decoders directly from its index tables (128 entries each;
https://encoding.spec.whatwg.org/indexes.json) instead of JVM
Charset. Include the full label→encoding table and correct fatal/replacement behavior. Verify withencoding/textdecoder-fatal-single-byte.any.js(1,092 failing),textdecoder-labels.any.js,single-byte-decoder.window.js. - URL setter stripping (Elide runtime) — unlocks ~130 subtests,
complexity low-medium. URL component setters must strip tab/newline and
handle U+0000 per the URL Standard; port setter must ignore invalid input
rather than throw. Verify with
url/url-setters-stripping.any.js. - Legacy multi-byte decoders: gbk/gb18030, ISO-2022-JP (Elide runtime) — unlocks ~450 subtests, complexity medium (stateful decoders, larger index tables). Natural follow-on to item 1 once the index-table infrastructure exists.
- WPT server support in the harness (THIS repo, not Elide) — un-gates up
to ~1,300 subtests (
fetch/api,fetch/http-cache, …), complexity medium-high. The runner executes.any.jsfiles without a wptserve instance or base URL, so anything fetching a relative resource dies in setup. Standing up wptserve (or a minimal substitute honoring WPT's{{host}}/{{ports}}substitution) inside the Docker run converts these from permanently-failing into a real fetch-conformance signal. Expect real fetch bugs to surface after; that's the point. - Park / skip candidates:
fetch/fetch-later(136 — browser-only, needs window/iframe/BFCache),.window.jstests requiringdocument/XMLHttpRequest(~90). Consider adding these to[skip]inexpectations/wpt-wintertc.tomlwith a comment. Note this only cleans up the "vs expectations" rate; the overall pass rate keeps skipped tests in its denominator on purpose.
Items 1–3 alone should move the suite from 75.3% to roughly 88–90%.
- Elide fixes happen in the Elide runtime repo; this repo only pins nightlies
and measures. Use
--elide nightlyto pick up a fresh build. - Failure text like
not available on this JVMcomes from Elide's own error messages — grep the Elide codebase for it to find the TextDecoder impl fast. - The suite runs in ~12s, so iterate with
--include 'encoding/**' --log. - After a fix lands: run full suite → check
changes.mdfor new passes and no regressions →--ratchet→ commit reports + expectations together.