Conversation
* src/unexpand.c (unexpand): Clamp a negative c32width() to 1 when accumulating a blank, matching the sibling non-blank path and expand and fold. A separator whose display width is negative (e.g. U+0085 where c32issep() accepts it) left 'column' un-advanced, so the pending blank buffer, sized for one byte of advance per blank, grew without bound. * NEWS: Mention the bug fix.
|
I'll squash in a test... diff --git a/tests/unexpand/mb.sh b/tests/unexpand/mb.sh
index 84ba0354e..dba7e00c9 100755
--- a/tests/unexpand/mb.sh
+++ b/tests/unexpand/mb.sh
@@ -178,4 +178,12 @@ ideo_space=$(env printf '\u3000')
unexpand -t1 >out 2>err; ret=$?
test "$ret" = 0 || { cat err; fail=1; }
+# On some platforms (musl) U+0085 is a blank with a negative display width.
+# Such blanks must advance the column, otherwise the pending-blank
+# buffer grows without bound. Elsewhere they pass through unchanged.
+next_line=$(env printf '\u0085')
+{ yes "$next_line" | head -n 40000 | tr -d '\n'; echo; } |
+ unexpand -t1 >out 2>err; ret=$?
+test "$ret" = 0 || { cat err; fail=1; }
+
Exit $fail |
|
Thanks for the report and patch @isl-Ramzi! @pixelb I'm curious, were you able to reproduce it on glibc? I haven't been able to test it yet. I guess we should probably take another look at the other c32width calls. It's always possible for places to have strange locale definitions. |
|
No it's only with the combo of c32issep() allowing 0x0085 through (on non glibc). |
I did a quick audit there, and they seem OK. |
Nice, thanks for looking into that and the Perhaps it is worth mentioning that it can't happen on glibc in the NEWS entry? If it can be done without making it too wordy, that is. Maybe |
3 participants
In unexpand(), the blank-accumulation path runs
column += c32width (g.ch)without clamping, unlike the non-blank path just below it and the matching code in expand and fold, which treat a negative width as 1. A separator whose display width is negative never advancescolumn, so thecolumn >= next_tab_columnflush never fires and the fixedpending_blankbuffer (sizedmax_column_width * MB_CUR_MAX, on the assumption that every pending blank advancescolumnby at least one) grows past its end. It is reachable from input in a multi-byte locale on platforms whosec32issep()accepts such a character as a blank, for example U+0085 (NEL), whichiswspace()reports as a space on musl whilewcwidth()returns -1; on glibcc32issep()isiswblank(), so only TAB and SPACE reach this path and the bug stays latent there. The 9.12 fix for this same buffer covered oversized tab values and over-long multi-byte blanks, but not a negative-width blank. Clamping the negative width to 1 matches the sibling sites and restores the one-column-per-blank invariant the buffer size relies on.Verified under ASAN: feeding leading U+0085 characters (with the musl classification) writes two bytes past the 32-byte pending_blank region at src/unexpand.c:210 before this change, and runs clean afterwards.