subprocess makes environment blocks with duplicate keys on Windows #91018

Activity

1. benrg commented on Feb 26, 2022

   

   benrgmannequin

   on Feb 26, 2022

   MannequinAuthor

   More actions

   On Windows, if one writes

       env = os.environ.copy()
       env['http_proxy'] = 'whatever'

   or either of the documented equivalents ({**os.environ, ...} or (os.environ | {...})), and passes the resulting environment to subprocess.run or subprocess.Popen, the spawned process may get an environment containing both HTTP_PROXY and http_proxy. Most Win32 software will see only the first one, which contains the unmodified value from os.environ.

   Because os.environ forces all keys to upper case, it's possible to work around this by using only upper case keys in the update, but that behavior of os.environ is nonstandard (bpo-46861), and subprocess shouldn't depend on it always being true, nor should end users have to.

   Since dicts preserve order, the user's (presumable) intent is preserved in the env argument. I think subprocess should do something like

       env = {k.upper(): (k, v) for k, v in env.items()}
       env = dict(env.values())

   to discard duplicate keys, keeping only the rightmost one.
2. benrg

   added

   3.10 (EOL)end of lifeend of life

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   OS-windows

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Feb 26, 2022
3. eryksun commented on Feb 26, 2022

   

   eryksun

   on Feb 26, 2022

   Contributor

   More actions

   This should be handled in _winapi.CreateProcess(). An environment block is technically required to be sorted. (Ages ago this was a MUST requirement for getting and setting variables to work correctly, since the implementation depended on the sort order, but I think nowadays it's a SHOULD requirement.) For example, see the documentation of CreateProcessW() [1]:

   If an application provides an environment block, ... explicitly create
   these environment variable strings, sort them alphabetically (because
   the system uses a sorted environment)
   

   "Changing Environment Variables" is more specific [2]:

   All strings in the environment block must be sorted alphabetically by name.
   The sort is case-insensitive, Unicode order, without regard to locale.
   

   CompareStringOrdinal() [3] implements a case-insensitive ordinal comparison. When a key compares as equal, either keep the current one in the sorted list, or replace it with the new key.

   ---

   [1] https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createprocessw
   [2] https://docs.microsoft.com/en-us/windows/win32/procthread/changing-environment-variables
   [3] https://docs.microsoft.com/en-us/windows/win32/api/stringapiset/nf-stringapiset-comparestringordinal
4. eryksun

   added

   3.9 (EOL)end of lifeend of life

   3.11only security fixesonly security fixes

   on Feb 26, 2022
5. eryksun commented on Feb 26, 2022

   

   eryksun

   on Feb 26, 2022

   Contributor

   More actions

   I suggest closing this issue as a duplicate of bpo-43702.

   👍React with 👍1Reacted by Steve Dower
6. ezio-melotti

   transferred this issue fromon Apr 10, 2022
7. zooba commented on Aug 25, 2022

   

   zooba

   on Aug 25, 2022

   Member

   More actions

   Duplicate of #87868
8. zooba

   marked this as a duplicate of [_winapi] correctly sort and remove duplicates in getenvironment() #87868 on Aug 25, 2022
9. zooba

   closed this as not plannedon Aug 25, 2022