Py.exe not working after 3.11rc1 install

Bug report

I have a Windows 10 VM that I use to build extensions for python.

Previously I had 3.11a3 installed.

After installing both python 3.11rc1 32 and 64 bit versions the py command is not working. Here is what I see in cmd:

: 10:12:15.43 C:\Users\barry
: \\VM-WIN10\barry> py
No suitable Python runtime found
Pass --list (-0) to see all detected environments on your machine

: 10:12:18.43 C:\Users\barry
: \\VM-WIN10\barry> py -0
 -V:3.11          Python 3.11 (64-bit)
 -V:3.11-32       Python 3.11 (32-bit)
 -V:3.10          Python 3.10 (64-bit)
 -V:3.10-32       Python 3.10 (32-bit)
 -V:3.9           Python 3.9 (64-bit)
 -V:3.9-32        Python 3.9 (32-bit)
 -V:3.8           Python 3.8 (64-bit)
 -V:3.8-32        Python 3.8 (32-bit)
 -V:3.7           Python 3.7 (64-bit)
 -V:3.7-32        Python 3.7 (32-bit)
 -V:3.6           Python 3.6 (64-bit)
 -V:3.6-32        Python 3.6 (32-bit)
 -V:3.5
 -V:3.5-32
 -V:3.4
 -V:2.7
: 10:13:49.37 C:\Users\barry
: \\VM-WIN10\barry> type C:\Users\barry\AppData\Local\py.ini
[defaults]
python=3.10-64
python3=3.10-64

Your environment

CPython versions tested on: python 3.11.0-rc1
Operating system and architecture: windows 10

@pablogsal

This seems to be a behaviour change - the default of "3.10-64" is no longer recognised. It's documented as valid here:

Furthermore it is possible to specify if a 32 or 64 bit implementation shall be requested by adding “-32” or “-64”.

@pablogsal should this be marked as a release blocker?

I had a dig through the current "whatsnew" page and there's no mention of this change, fwiw. I had also run into this issue - the launcher seems to have dropped the -64 suffix, so if you have a version pinned in your py.ini file that uses the -64 syntax, you get into this state where there seem to be no versions. From memory, this changed at the rc1 release.

By all means drop the need for the -64, but I think it must be supported for existing installations.
The error messages gives the user no clue what is wrong either.

@pablogsal

@pablogsal should this be marked as a release blocker?

Hummmm, is not a bad crash but I think it may be important enough that should block the release. I will re-evaluate on release day if is not fixed, but for now it should be a release blocker.

Can someone bisect the problem to point to a specific commit?

@zooba

Can someone bisect the problem to point to a specific commit?

Unlikely, as the py.exe in 3.11 is a complete rewrite, so it's more probable that this behaviour simply wasn't included in the new version. @zooba any thoughts?

Note, this seems to only apply to configured defaults in py.ini. Using py -3.10-64 at the command line works fine. This would suggest to me that while this is a regression, its impact is manageable.

Currently the "-32" and "-64" suffixes are only implemented for the command line. They're not implemented for shebangs and not for the defaults in the "PY_*" environment variables or "py.ini" files. The "-32" suffix should work for newer 32-bit releases, which use the "-32" suffix in the registry key name. In this case, the launcher won't actually validate that the architecture is 32-bit, but that shouldn't be a problem.

In parseCommandLine() , suffixed version tags are handled by setting search->exclude32Bit ("-64") or search->only32Bit ("-32") and then ignoring the suffix by subtracting 3 from the tag length.

cpython/PC/launcher2.c

Lines 646 to 662 in 9e55685

    
           if (STARTSWITH(L"2") || STARTSWITH(L"3")) { 
        
               // All arguments starting with 2 or 3 are assumed to be version tags 
        
               search->tag = arg; 
        
               search->tagLength = argLen; 
        
               search->oldStyleTag = true; 
        
               search->restOfCmdLine = tail; 
        
               // If the tag ends with -64, we want to exclude 32-bit runtimes 
        
               // (If the tag ends with -32, it will be filtered later) 
        
               if (argLen > 3) { 
        
                   if (0 == _compareArgument(&arg[argLen - 3], 3, L"-64", 3)) { 
        
                       search->tagLength -= 3; 
        
                       search->exclude32Bit = true; 
        
                   } else if (0 == _compareArgument(&arg[argLen - 3], 3, L"-32", 3)) { 
        
                       search->tagLength -= 3; 
        
                       search->only32Bit = true; 
        
                   } 
        
               }

Handling the suffix could be factored out into a common function. Alternatively, I think it could be moved into performSearch() as a post-processing step for old-style tags, after parseCommandLine(), checkShebang(), and checkDefaults() have been called.

Defaults picked up from the config files should probably be treated as old-style arguments if they don't contain a slash.

The -64 was dropped because it's too confusing in the presence of ARM64. I opposed adding -64 as a suffix because it seemed likely to become an issue, and was never part of PEP 514 tags anyway (unless someone chose it explicitly).

In this case, it's just the oldStyleTag field that needs to be set. But people should be able to set their defaults to any runtime - not just PythonCore - so we can't just set it all the time.

We should fix this one up. It's a pretty significant takeback. I'll take a look tonight

pythongh-96559: Fixes Windows launcher handling of defaults using old…

gh-96559: Fixes Windows launcher handling of defaults using old-style…

pythongh-96559: Fixes Windows launcher handling of defaults using old…

Meta issue — are there any core devs besides Steve working on the codebase for Py? I wouldn’t even know how to find the source code. :-/

I’m keeping an eye on things as they come up but I’ve not really looked at the new code in depth.

The source is launcher2.c in the PC directory

@zooba

@zooba, why doesn't the new launcher implement the old launcher's check for 32-bit and 64-bit binaries via GetBinaryTypeW(), where possible at least? "SysArchitecture" is only defined for 3.6+. The fallback architecture may be wrong (e.g. KEY_WOW64_64KEY and KEY_WOW64_32KEY are ignored in 32-bit systems), and a fallback isn't possible for per-user installations in HKCU. Due to the latter, 32-bit releases prior to 3.6 can't be selected with the "-32" suffix if they're installed for the current user. Even py -3.5-32 fails in this case (e.g. debug: "Excluding PythonCore/3.5-32 because it doesn't look 32bit"). Why not at least try calling GetBinaryTypeW() on the executable before using the fallback, if any? Is this a conscious decision or an oversight?

gh-96559: Fixes Windows launcher handling of defaults using old-style…

Conscious decision. There are very few cases that will no longer be handled, and all of them in end-of-life interpreters released by us, as well as many more cases that will simply fail GetBinaryTypeW. The cost of the code isn't worth the few edge cases - I'd much rather hard-code the versions for the specific check (i.e. 2.x-32 for 0<=x<=7 and 3.x-32 for 0<=x<=5) than keep the additional complexity out of the code. (I also want to minimize file system access and rely on registration information.)

But there are vanishingly few 32-bit installs out there anymore, potentially more 32-bit 2.7 installs than any other version. I've never even seen the -32 options discussed outside of our own docs, other than the discussion that asked for -64 instead. I don't have real data on usage, but I find it hard to believe this is a widely used option.

I use a -32, but it's specifically for testing that something still works with a 32-bit Windows Python, not because I actually have a use for it myself :)

I use a -32, but it's specifically for testing that something still works with a 32-bit Windows Python

This is a valid use, though, so thanks for mentioning it. But are you testing that something still works on 3.5 or earlier 32-bit?

In our specific case we dropped 3.5 support a number of months ago, so no.

Conscious decision

Okay. I haven't used a 32-bit build of Windows in longer than I can remember, so the fallback works fine for me for all-users installations, based on whether the "HKLM\Software" hive is accessed with KEY_WOW64_64KEY or KEY_WOW64_32KEY.

Navigation Menu

Repository navigation

Py.exe not working after 3.11rc1 install #96559

Bug report

Your environment

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	if (STARTSWITH(L"2") \|\| STARTSWITH(L"3")) {
	// All arguments starting with 2 or 3 are assumed to be version tags
	search->tag = arg;
	search->tagLength = argLen;
	search->oldStyleTag = true;
	search->restOfCmdLine = tail;
	// If the tag ends with -64, we want to exclude 32-bit runtimes
	// (If the tag ends with -32, it will be filtered later)
	if (argLen > 3) {
	if (0 == _compareArgument(&arg[argLen - 3], 3, L"-64", 3)) {
	search->tagLength -= 3;
	search->exclude32Bit = true;
	} else if (0 == _compareArgument(&arg[argLen - 3], 3, L"-32", 3)) {
	search->tagLength -= 3;
	search->only32Bit = true;
	}
	}

Py.exe not working after 3.11rc1 install #96559

Description

Bug report

Your environment

Activity

pfmoore commented on Sep 4, 2022

mwichmann commented on Sep 4, 2022

barry-scott commented on Sep 4, 2022

pablogsal commented on Sep 4, 2022

pfmoore commented on Sep 4, 2022

pfmoore commented on Sep 4, 2022

eryksun commented on Sep 4, 2022

zooba commented on Sep 5, 2022

gvanrossum commented on Sep 5, 2022

pfmoore commented on Sep 5, 2022

pfmoore commented on Sep 5, 2022

eryksun commented on Sep 5, 2022

zooba commented on Sep 6, 2022

mwichmann commented on Sep 6, 2022

zooba commented on Sep 6, 2022

mwichmann commented on Sep 6, 2022

eryksun commented on Sep 6, 2022

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions