Repository navigation
_winapi.LCMapStringEx fails when encountering a string containing null characters #106844
Description
Activity
There is a problem not only with the null character handling.
I tested on 3.12rc1. Note that -1 passed as the input size, so
LCMapStringEx()determines the size of the string by the terminating null character, so it is not a limitation of the API.>>> x = _winapi.LCMapStringEx(_winapi.LOCALE_NAME_INVARIANT, _winapi.LCMAP_LOWERCASE, 'x'*(2**32+5)) >>> len(x) 5 >>> del x >>> x = _winapi.LCMapStringEx(_winapi.LOCALE_NAME_INVARIANT, _winapi.LCMAP_LOWERCASE, 'x'*(2**31+5)) Traceback (most recent call last): File "<stdin>", line 1, in <module> MemoryError >>> x = _winapi.LCMapStringEx(_winapi.LOCALE_NAME_INVARIANT, _winapi.LCMAP_LOWERCASE, 'x'*(2**30+5)) >>> len(x) 1073741829 >>> len(x) - 2**30 5 >>> del x >>> x = _winapi.LCMapStringEx(_winapi.LOCALE_NAME_INVARIANT, _winapi.LCMAP_LOWERCASE, 'x'*(2**31-1)) Traceback (most recent call last): File "<stdin>", line 1, in <module> MemoryError >>> x = _winapi.LCMapStringEx(_winapi.LOCALE_NAME_INVARIANT, _winapi.LCMAP_LOWERCASE, 'x'*(2**31-2)) >>> len(x) - 2**31 -2 >>> del x >>> x = _winapi.LCMapStringEx(_winapi.LOCALE_NAME_INVARIANT, _winapi.LCMAP_LOWERCASE, 'x'*(2**32-2)) Traceback (most recent call last): File "<stdin>", line 1, in <module> MemoryError >>> x = _winapi.LCMapStringEx(_winapi.LOCALE_NAME_INVARIANT, _winapi.LCMAP_LOWERCASE, 'x'*(2**32-1)) Traceback (most recent call last): File "<stdin>", line 1, in <module> OSError: [WinError 6] The handle is invalid
- Size up to
2**31-2including works correctly. - Size between
2**31-1and2**32-2raises MemoryError. I think thatLCMapStringEx()return a correct size, but converted tointit becomes negative, than it is promoted toPy_ssize_t, and interpreted as a very largesize_tvalue, which causes a MemoryError. - Size
2**32-1raises OSError.LCMapStringEx()returns the output size including the terminating null character.(int)(2**32-1 + 1) == 0, and it is interpreted as a signal of error. - Size larger than
2**32-1loses highest bits.
Rejecting size larger than INT_MAX is safe, in any case
LCMapStringEx()does not support it now (and in worst case it silently returns a truncated result). Perhaps size exactly equal to INT_MAX produces an OSError on 32-bit Windows. And with different content we can get an OSError for strings smaller than INT_MAX characters.Perhaps we can make it supporting strings between
2**31-1and2**32-2characters long if cast sizes tounsigned int. But I am not sure that it is guaranteed and that we need this fornormcase().- Size up to
Perhaps size exactly equal to INT_MAX produces an OSError on 32-bit Windows.
No, it always less than INT_MAX on 32-bit Windows.
- added 4 commits that reference this issue
on Aug 11, 2023
Bug report
This bug was encountered when working through gh-106816, specifically this comment chain.
_winapi.LCMapStringEx()raises the errorValueError: embedded null characterwhen passed a string containing a null character.This can be reproduced by running the following:
From the discussion in the linked PR it looks like the
LCMapStringEximplementaion in 3.11 has already been improved to handle null characters and so maybe the same change can be brought forward into the 3.12 and 3.13 implementations?Your environment
Linked PRs
LCMapStringExin_winapi#106857