Skip to content

fix: support SJIS-win exports with iconv - #20487

Open
mvanhorn wants to merge 3 commits into
phpmyadmin:masterfrom
mvanhorn:fix/20128-sjis-win-iconv-conversion
Open

mvanhorn wants to merge 3 commits into
phpmyadmin:masterfrom
mvanhorn:fix/20128-sjis-win-iconv-conversion

Conversation

@mvanhorn

Copy link
Copy Markdown
Contributor

Description

First confirm the alias diagnosis on an affected iconv backend by comparing conversion to SJIS-win, SJIS-win//TRANSLIT, CP932, and CP932//TRANSLIT, recording PHP and iconv implementation versions; the expected distinction is support for the charset name, not a blanket incompatibility with transliteration. In src/Encoding.php, normalize the exact SJIS-win name case-insensitively to CP932 for both source and destination only when the selected engine is iconv, immediately before the existing conversion dispatch. Preserve the equality fast path, configured iconv suffix, mbstring behavior, and engine-none behavior, and do not substitute plain SJIS or suppress conversion errors.

Custom SQL export with SJIS-win selected fails with an iconv wrong-encoding error on the reporter's PHP 8.5.3 installation, affecting both 5.2.4-dev and 6.0.0-dev. The issue supplies a MyISAM table and a Japanese テスト row as a concrete reproduction. On the current master checkout, Encoding::convertString() passes the configured charset straight to iconv and appends the default //TRANSLIT suffix; SJIS-win is offered by the default charset configuration. The thread suggests removing the suffix, but does not establish that transliteration is the cause; acceptance of charset aliases varies between iconv implementations.

Fixes #20128

@codecov

codecov Bot commented Sep 16, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.47368% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.20%. Comparing base (1859a20) to head (9ffc95a).
⚠️ Report is 781 commits behind head on master.

Files with missing lines Patch % Lines
src/Encoding.php 89.47% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #20487      +/-   ##
============================================
+ Coverage     63.49%   64.20%   +0.71%     
- Complexity    16060    16099      +39     
============================================
  Files           676      678       +2     
  Lines         59891    57852    -2039     
============================================
- Hits          38028    37145     -883     
+ Misses        21863    20707    -1156     
Flag Coverage Δ
dbase-extension 64.15% <89.47%> (+0.73%) ⬆️
unit-8.2-ubuntu-latest 64.16% <89.47%> (+0.71%) ⬆️
unit-8.3-ubuntu-latest 64.16% <89.47%> (+0.71%) ⬆️
unit-8.4-ubuntu-latest 64.12% <89.47%> (+0.72%) ⬆️
unit-8.5-ubuntu-latest 64.17% <89.47%> (+0.78%) ⬆️
unit-8.6-ubuntu-latest 64.12% <89.47%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Fixes phpmyadmin#20128

Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
@mvanhorn
mvanhorn force-pushed the fix/20128-sjis-win-iconv-conversion branch from 8ab2e30 to e5e883a Compare September 16, 2026 15:11
@mvanhorn

Copy link
Copy Markdown
Contributor Author

The PHP 8.5 job wasn't failing an assertion — it was hitting max_execution_time. EncodingTest::testIconvSjisWin hangs: the SJIS-win path normalises to CP932 and hands it to iconv() with the caller's //TRANSLIT or //IGNORE suffix, and on glibc that combination can fail to consume any input for an unconvertible sequence, so the conversion loop spins until PHP kills it at 300s.

Fixed in 1e73b67:

  • prefer mbstring when the build lists SJIS-win natively, since mbstring implements Windows CP932 directly rather than via an iconv alias
  • when falling back to iconv for CP932, drop the //TRANSLIT / //IGNORE suffixes, so an unconvertible sequence returns false instead of spinning
  • a false result from either backend now yields the original string rather than propagating false
  • the mbstring capability lookup is cached, since convertString() runs once per exported line

Two things worth a reviewer's eye. Dropping //TRANSLIT on the iconv CP932 fallback is a real behaviour change — unconvertible characters are no longer transliterated on that path. I think that's the right trade against a hang, but say the word if you'd rather keep transliteration and bound the loop a different way. And I could not reproduce the hang on my machine (macOS, libiconv 1.11) — it's glibc-specific, so the fix is reasoned from the mechanism and confirmed by the suite now finishing in ~35s rather than by watching the original spin stop.

The SJIS-win export path normalised to CP932 and passed it to iconv with
the caller's //TRANSLIT or //IGNORE suffix. On glibc that combination can
fail to consume any input for an unconvertible sequence, so the export
loop span until PHP hit max_execution_time -- the CI job died at the 300s
limit rather than on an assertion.

Prefer mbstring when the build lists SJIS-win natively, since mbstring
implements Windows CP932 directly. When falling back to iconv for CP932,
drop the //TRANSLIT and //IGNORE suffixes so a failed conversion returns
false instead of spinning. Either way a false result now yields the
original string rather than propagating false. The mbstring capability
lookup is cached because convertString runs per exported line.

Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
@mvanhorn
mvanhorn force-pushed the fix/20128-sjis-win-iconv-conversion branch from 1e73b67 to 86ff447 Compare September 19, 2026 02:03
PHPStan reported the ignore pattern for Encoding::convertString() returning
string|false as unmatched, since this branch no longer produces that error.
Verified locally: phpstan analyse reports no errors.

Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
@mvanhorn
mvanhorn force-pushed the fix/20128-sjis-win-iconv-conversion branch from 0c8eb6e to 9ffc95a Compare September 30, 2026 21:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: ErrorException: iconv(): Wrong encoding, conversion from "utf-8" to "SJIS-win//TRANSLIT" is not allowed

1 participant