url: accept ASCII hosts whose xn-- labels fail ToASCII (whatwg/url#914) by robobun · Pull Request #43872 · oven-sh/bun

robobun · 2026-09-24T02:39:09Z

Problem

new URL("https://xn--8i7caa.famitei.net/") throws ERR_INVALID_URL. The site is live. The URL Standard (Add Unicode ToASCII fallback for ASCII domains whatwg/url#914), WPT and Node v26.7.0+ parse it.
URL.canParse, the href/host/hostname setters, url.parse, url.domainToASCII and url.domainToUnicode reject the same hosts.
Cause: Bun::hasValidPunycodeHost, a UTS Bun v0.0.41 #46 check that bun runs after WebKit's host parser. WebKit's parser already passes an ASCII host through.

Fix

Delete the check, its decoder (ASCIIHostPunycodeCheck.h) and its callers in DOMURL.cpp and URLDecomposition.cpp.
url.parse returns an ASCII hostname lowercased, as ada::idna::to_ascii does.
Still rejected: a non-ASCII domain with a bad label, forbidden host code points, a host that ends in a number.
Verified: WPT IdnaTestV2.json in test/js/web/url/url-wpt-constructor.test.ts (main fails 760 of 2670 rows). Self-reviewed: 6 concerns raised, 6 addressed.

Background

An xn-- label is the Punycode form of a Unicode label. UTS Bun v0.0.41 #46 ToASCII decodes and validates it.
Bun 1.3 and 1.4 reject these hosts, and List of breaking changes for 1.4 #28792 lists that as a 1.4 breaking change. This PR needs a maintainer decision.
Last of a stack: merge test: re-sync urltestdata.json from WPT and pin the 8 rows that whatwg/url#914 changed #43851 and url: domainToUnicode converts per label like ada (bad label unchanged, linear time) #41464 first.
Considered: keep the check for node:url only. Current Node accepts them there too.

Downsides

Behavior change: code that uses new URL or URL.canParse as a strict host check now accepts hosts such as xn--a.com (replacement in Notes).
One case gets worse until WebKit's setters are fixed: u.hostname = "é.%78n--a" sets xn--9ca.xn--a. Node refuses it.
No cost added: URL.canParse("https://xn--ls8h.com/") goes from 2469 to 529 instructions. Release .text shrinks by 5,632 B.

Notes

Evidence for the direction.

Add Unicode ToASCII fallback for ASCII domains whatwg/url#914, "Add Unicode ToASCII fallback for ASCII domains", merged 2026-06-25. It names live sites that fail ToASCII: xn--8i7caa.famitei.net, xn--board-ngr.palungjit.org. Implementer bugs: Gecko 2049177, WebKit 316554, Deno 35411, Node 64045.
Node landed ada 4.0.0 on 2026-07-29 (deps: update ada to 4.0.0 nodejs/node#64790). deps/ada/ada.h says 4.0.0 at v26.7.0 and v24.20.0, and 3.4.4 at v26.6.0 and v24.19.0.
WPT c48d58747e: toascii.json has 7 rows that changed from null to the input, urltestdata.json has 8 rows that changed from failure to parse, IdnaTestV2.json expects failure for 2 of its 994 ASCII inputs (both contain %).
.claude/docs/landing-prs.md: for URL "the WHATWG/W3C spec and WPT are the bar", for node:* modules "real Node's observed behavior is the spec".
WebKit: URLParser::domainToASCII lowercases a containsOnlyASCII() domain and calls ICU only for a non-ASCII one.

History. new URL, URL.canParse, the hostname setter, url.parse and new Request reject xn--8i7caa.famitei.net on bun 1.3.7, 1.3.13 and 1.3.14 (through WebKit's parser of that time). url.domainToASCII passed it through in those releases. On 1.4.x the rejection comes from the layer this PR deletes, url.domainToASCII returns "", and new Request accepts the host. Node v26.3.0 rejects at every door, Node v26.10.0 accepts at every door.

Merge order and open PRs that conflict.

Merge test: re-sync urltestdata.json from WPT and pin the 8 rows that whatwg/url#914 changed #43851, then url: domainToUnicode converts per label like ada (bad label unchanged, linear time) #41464, then this PR.
fetch, Request, WebSocket: reject hosts with an invalid punycode label, like new URL #41772 extends the same rejection to fetch, Request and WebSocket, and url: give blob: URLs a null origin when the inner host has invalid punycode #39404 extends it to blob: origins. Both were written against Node before ada 4.0.0. They must close when this PR merges. url: give blob: URLs a null origin when the inner host has invalid punycode #39404 adds a hasAcceptableHost call in URLDecomposition::origin(), a hunk this PR does not touch, so both would merge cleanly and break the build.
url: follow the spec for the host/port setters, origin, and URLPattern groups #32826, url: strip ASCII tab and newline in the host, hostname and pathname setters before inspecting the value #39458 and url: do not abort when a host is too long for the IDNA conversion buffers #42856 touch lines that this PR deletes. Each drops the check when it rebases.
The WebKit change for the setter case below is in progress in a separate PR.

Self-review. Six concerns, all addressed: the stack is rebased on main after #43040, which already made tls.checkServerIdentity match an ASCII host as typed, so the TLS test and claim of an earlier version of this PR are gone. The history above is corrected (an earlier version said the rejection was new in 1.4). new Request, Response.redirect, blob: origin, fetch and WebSocket rows pin the entry points that #41772 and #39404 name. WPT IdnaTestV2.json, IdnaTestV2-removed.json and setters_tests.json are vendored with runners. The ASCII pass-through of url.parse lives in one helper.

Door comparison. Inputs: every input and output of WPT IdnaTestV2.json and toascii.json plus 27 hand-written hosts, 2794 in total. Doors: new URL, URL.canParse, hostname=, host=, href=, url.parse, url.domainToASCII, url.domainToUnicode. Against Node v26.10.0, a release build of main 6d504dd differs on 785 inputs (777 at each WHATWG door, 784 at url.parse, 781 at domainToUnicode). This PR differs on 1 input, é.%78n--a, at hostname= and host=.

The setter case. WTF::URL::setHost runs ICU on a non-ASCII value before it splices the value into the URL and parses again (appendEncodedHostname). Percent-decoding happens in the parser, after that pass, so ICU never sees xn--a next to é. The planned change in oven-sh/WebKit hands the raw value to the parser, which runs IDNA once, after percent-decoding.

A quadratic path goes away. With 32768 bad labels behind one right-to-left label ("xn--mgbh0fb." + "xn--xn--zca-hia.".repeat(32768) + "com"), URL.canParse, url.domainToASCII and url.domainToUnicode each take 555 ms on main (10, 37, 141, 555 ms for 4096 to 32768 labels). The time is in hasValidPunycodeHost: ICU's whole-host ToASCII inserts U+FFFD after each bad label and moves the rest of the host. With this PR the same calls take 0 to 12 ms.

Possible follow-up. After this PR the internalBinding("icu") shim (jsIcuToASCII, jsIcuToUnicode, icuToUnicode) has one consumer, the vendored test-icu-punycode.js, which Node deleted together with its ICU-based punycode (nodejs/node#55156). Removing the shim and that test is a separate change.

Strict check in user code. url.domainToUnicode(host).split(".").some(label => label.startsWith("xn--")) is true when a label did not decode. It needs #41464.

TLS. TLS matching does not change. tls.checkServerIdentity and the native matcher call idnaToASCII only for a name with a non-ASCII character, and that path is the same UTS #46 conversion as on main. An ASCII name is matched as typed (#43040).

Instructions per call. Release builds of main 6d504dd and of this branch at 40fb040 on that base, inclusive user instructions of one call, single-stepped under gdb with BUN_JSC_useJIT=0 after a warm-up pass. Later commits moved the ASCII check of url.parse into the idnaToASCII helper and rebased the stack.

door	input	main	this PR
`URL.canParse`	`https://example.com/`	674	523
	`https://my-site.example.com/a--b`	789	583
	`https://xn--ls8h.com/`	2469	529
	`https://xn--mgbh0fb.xn--kgbechtv/`	11560	497
	`https://münchen.de/`	5933	5117
`hostname=`	`other.example`	2814	2445
	`xn--ls8h.com`	5412	2308
	`xn--mgbh0fb.xn--kgbechtv`	23341	2510
	`münchen.de`	9191	5545
`url.domainToASCII`	`example.com`	1414	1204
	`xn--ls8h.com`	2419	1244
	`xn--mgbh0fb.xn--kgbechtv`	11553	1457
	`münchen.de`	7161	5778
`url.parse` IDNA step	`example.com`	765	504
	`xn--ls8h.com`	4094	517
	`xn--mgbh0fb.xn--kgbechtv`	6962	584
	`münchen.de`	3776	3775

Release .text on that base: main 80,682,903 B, #41464 80,683,559 B, this PR 80,677,927 B. That is 5,632 B less than #41464 and 4,976 B less than main.

Tests.

test/js/web/url/url.test.ts: the three blocks that pinned the rejection now pin the values of Node v26.10.0, with new blocks for the hosts that must still fail, for the three setters, and for new Request, Response.redirect, blob: origins, fetch and WebSocket (through a local proxy, so no DNS).
test/js/web/url/url-wpt-constructor.test.ts: the 8 pinned rows of test: re-sync urltestdata.json from WPT and pin the 8 rows that whatwg/url#914 changed #43851 go away, and a new block runs WPT IdnaTestV2.json (2670 rows) and IdnaTestV2-removed.json (20 rows). The 993 ASCII rows run everywhere. The non-ASCII rows need the Unicode data of ICU 78, so they skip on an older system ICU (macOS).
test/js/web/url/url-wpt-setters.test.ts (new, one runner per WPT data file, beside url-wpt-constructor.test.ts): WPT setters_tests.json, 278 rows. Two rows that have nothing to do with IDNA are pinned as known deviations (host = "example.com:80" on a URL with port 8080, and a whitespace-only port). Main fails 4 rows.
test/js/node/url/url-domain-ascii-unicode.test.js: new domainToASCII and domainToUnicode rows, and the knownRejectDeviations list of url: domainToUnicode converts per label like ada (bad label unchanged, linear time) #41464 goes away: all 32 rows of its table now give the value of Node.
test/js/node/url/url.test.ts: url.parse rows.
toascii.json moves to WPT c48d58747e (7 rows changed from null to the input).
test-whatwg-url-custom-domainto.js is a vendored Node test that Node deleted in 7443a28c28 (2024). Its round trip domainToASCII(domainToUnicode(input)) fails on Node v26.10.0 too for xn--a.xn--zca, which becomes xn--a.ß. The file now expects "" for such a row, with a comment.
One url.test.ts row needs ICU 76 (the Unicode 15.1 rule for a decoded xn-- prefix inside a non-ASCII host) and is guarded.
Debug build: test/js/web/url/, test/js/node/url/, test/js/web/fetch/fetch.tls.wildcard.test.ts, and the vendored test-whatwg-url-*, test-url-*, test-icu-punycode.js and test-tls-check-server-identity.js. url-canParse-whatwg.test.js times out on main too with a debug build.

no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/node/url/url.test.ts

robobun · 2026-09-24T02:39:29Z

^{Updated 12:49 PM PT - Sep 24th, 2026}

✅ @robobun, your commit af2c7a4bb40116447f21769c2327141354e120ab passed in Build #120378! 🎉

🧪 To try this PR locally:

bunx bun-pr 43872

That installs a local version of the PR into your bun-43872 executable, so you can run:

bun-43872 --bun

robobun · 2026-09-24T02:39:59Z

Status: draft. The code is ready. It waits for one decision from a maintainer (@Jarred-Sumner): does bun follow whatwg/url#914 and stop rejecting all-ASCII hosts whose xn-- labels fail ToASCII?

Reproduction: bun -e 'new URL("https://xn--8i7caa.famitei.net/")' throws ERR_INVALID_URL on every release (1.3.7, 1.3.14 and 1.4.2 checked) and on main. Node v26.10.0 and this branch parse it. The rejection is listed as a 1.4 breaking change in List of breaking changes for 1.4 #28792, so this PR reverses a deliberate choice, for the reason that its references (Node, WPT, the URL Standard) changed.
Yes: merge test: re-sync urltestdata.json from WPT and pin the 8 rows that whatwg/url#914 changed #43851, then url: domainToUnicode converts per label like ada (bad label unchanged, linear time) #41464, then this PR, and close fetch, Request, WebSocket: reject hosts with an invalid punycode label, like new URL #41772 and url: give blob: URLs a null origin when the inner host has invalid punycode #39404, which extend the rejection to more entry points.
No: close this PR. test: re-sync urltestdata.json from WPT and pin the 8 rows that whatwg/url#914 changed #43851 and url: domainToUnicode converts per label like ada (bad label unchanged, linear time) #41464 do not depend on the answer. fetch, Request, WebSocket: reject hosts with an invalid punycode label, like new URL #41772 and url: give blob: URLs a null origin when the inner host has invalid punycode #39404 then complete the rejection at the entry points that still accept these hosts (fetch, Request, WebSocket, blob: origins).
Verified with the debug build: test/js/web/url/, test/js/node/url/ and the vendored Node URL tests. The door comparison with Node v26.10.0 over 2794 WPT inputs leaves 1 difference, listed under Downsides.

robobun · 2026-09-24T10:03:03Z

Related: oven-sh/WebKit#724 (bun side: #43883, draft) deletes the IDNA pass that WTF::URL::setHost ran on the raw setter value. With it the parser percent-decodes first, so u.hostname = "é.%78n--a" is refused by the host parser itself, with or without hasAcceptableHost. Checked on the preview build of that PR: the setter, URLPattern and constructor results for that value match Node.js v26.10.0.

The vendored copy was WPT 9504a83e01. The new copy is byte-identical to WPT c48d58747e and to Node main's test/fixtures/wpt. It adds about 450 lines of rows. Bun passes all of them except the 8 rows that whatwg/url#914 changed from failure to parse (an all-ASCII host whose xn-- label fails Unicode ToASCII). url-wpt-constructor.test.ts lists those 8 inputs and asserts that bun still rejects them, so the fix shows up there. toascii.json stays at the old snapshot because its 7 changed rows need the behavior change.

An all-ASCII host now passes through every door, lowercased, even when an xn-- label fails Unicode ToASCII: new URL, URL.canParse, URL.parse, the href, host and hostname setters, url.parse, url.domainToASCII and url.domainToUnicode. WebKit's host parser already does this. The rejection came from a layer that bun added on top of it to match Node before ada 4.0.0, and that layer is deleted: ASCIIHostPunycodeCheck.h, NodeURLHelpers.h, Bun::hasValidPunycodeHost, hasValidParsedHost and containsXNDashDash in DOMURL.cpp, hasAcceptableHost in URLDecomposition.cpp, and the ASCII fast path of icuToASCII. One helper, idnaToASCII, mirrors ada::idna::to_ascii for url.parse and the TLS matchers. A non-ASCII domain with a bad label, a forbidden domain code point and a host that ends in a number still fail. Tests: toascii.json moves to WPT c48d58747e, IdnaTestV2.json, IdnaTestV2-removed.json and setters_tests.json are vendored from the same commit with grouped runners, and the 8 rows that #43851 pinned in url-wpt-constructor.test.ts are unpinned.

robobun assigned Jarred-Sumner Sep 24, 2026

github-actions Bot added the claude label Sep 24, 2026

robobun force-pushed the robobun/a19eb3ca/ascii-host-passthrough branch from 40fb040 to 4ccacde Compare September 24, 2026 07:23

This was referenced Sep 24, 2026

Bump WebKit (oven-sh/WebKit#724 preview): URL host setters and URLPattern no longer turn IDNA-mapped delimiters into URL structure #43883

Draft

[WTF] URL::setHost parses the raw host: IDNA-mapped delimiters no longer become URL structure oven-sh/WebKit#724

Open

robobun force-pushed the robobun/a19eb3ca/reject-nested-xn-labels branch from e4325c2 to 3db6208 Compare September 24, 2026 18:43

robobun force-pushed the robobun/a19eb3ca/ascii-host-passthrough branch from 4ccacde to de4acb4 Compare September 24, 2026 19:03

robobun added 2 commits September 24, 2026 19:20

robobun force-pushed the robobun/a19eb3ca/ascii-host-passthrough branch from de4acb4 to af2c7a4 Compare September 24, 2026 19:22

robobun mentioned this pull request Sep 24, 2026

test: re-sync urltestdata.json from WPT and pin the 8 rows that whatwg/url#914 changed #43851

Open

Navigation Menu

Repository navigation

url: accept ASCII hosts whose xn-- labels fail ToASCII (whatwg/url#914) - #43872

url: accept ASCII hosts whose xn-- labels fail ToASCII (whatwg/url#914)#43872
robobun wants to merge 2 commits into
robobun/a19eb3ca/reject-nested-xn-labelsfrom
robobun/a19eb3ca/ascii-host-passthrough

robobun commented Sep 24, 2026 •

edited

Loading

Uh oh!

robobun commented Sep 24, 2026 •

edited

Loading

Uh oh!

robobun commented Sep 24, 2026 •

edited

Loading

Uh oh!

robobun commented Sep 24, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

robobun commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Fix

Background

Downsides

Uh oh!

robobun commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

robobun commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

robobun commented Sep 24, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

robobun commented Sep 24, 2026 •

edited

Loading

robobun commented Sep 24, 2026 •

edited

Loading

robobun commented Sep 24, 2026 •

edited

Loading