Repository navigation
sys.argv[0] and python -m package #52449
Description
| BPO | 8202 |
|---|---|
| Nosy | @brettcannon, @ncoghlan, @benjaminp, @voidspace |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/ncoghlan'
closed_at = <Date 2010-10-24.09:22:15.266>
created_at = <Date 2010-03-22.20:21:44.791>
labels = ['type-bug']
title = 'sys.argv[0] and python -m package'
updated_at = <Date 2010-10-24.09:22:15.265>
user = 'https://github.com/voidspace'bugs.python.org fields:
activity = <Date 2010-10-24.09:22:15.265>
actor = 'ncoghlan'
assignee = 'ncoghlan'
closed = True
closed_date = <Date 2010-10-24.09:22:15.266>
closer = 'ncoghlan'
components = []
creation = <Date 2010-03-22.20:21:44.791>
creator = 'michael.foord'
dependencies = []
files = []
hgrepos = []
issue_num = 8202
keywords = []
message_count = 11.0
messages = ['101531', '101538', '101542', '101544', '101546', '101582', '104434', '107651', '107720', '107723', '114116']
nosy_count = 4.0
nosy_names = ['brett.cannon', 'ncoghlan', 'benjamin.peterson', 'michael.foord']
pr_nums = []
priority = 'normal'
resolution = 'accepted'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue8202'
versions = ['Python 3.2']
Activity
voidspace commentedon Mar 22, 2010
When you execute "python -m package" the package is first imported with sys.argv[0] set to '-c' (and sys.modules['__main__'] exists but is empty. Then package.__main__.py is executed with the correct sys.argv[0].
This means module level code executed during the initial import that attempts to work out how the code is being executed can't use either sys.modules['__main__'] *or* sys.argv[0].
ncoghlan commentedon Mar 22, 2010
Hmm, the problem here is we don't *know* the module filename until we get hold of the loader for it (see _get_module_details() in runpy). And it is the process of finding the loader for the __main__ module that does the initial import of the package as a side effect.
Added Brett to the nosy list in case I've not thought of something obvious.
brettcannon commentedon Mar 22, 2010
Nick is right that importing package.__main__ requires importing package.__init__ first.
But it sounds like Michael really just wants some way to know when runpy is being used over something else. Could a special string token like "<runpy>" or "<-m>" be placed in sys.argv until it is back-patched to what it will eventually become? That should allow Michael to find out what whether runpy was being used or not.
voidspace commentedon Mar 22, 2010
I've already fixed my specific usecase (for unittest), but a special value in sys.argv[0] would indeed have met my needs.
voidspace commentedon Mar 22, 2010
'-c' isn't helpful because that is also the content of sys.argv[0] when a module is imported in response to "python -c '...'".
ncoghlan commentedon Mar 23, 2010
I realised today that this behaviour is actually the case for any execution of a module inside a package with -m (i.e. any __init__ modules execute before sys.argv and __main__ are fully configured).
As I recall, I used a bit of a hack to get main.c to implement -m correctly by piggybacking on the existing -c semantics. I'll find the hack and replace it with some proper '-c' or '-m' logic.
ncoghlan commentedon Apr 28, 2010
Committed for 2.7 in r80580 (as it turns out, the hack I remembered was probably from the original pre-runpy 2.4 implementation and has since been replaced by the proper runpy based system. This bug was likely just a lingering remnant of that original hackish approach).
Will forward port to 3.2 after the 2.7 beta is out.
ncoghlan commentedon Jun 12, 2010
Grr, in doing the Py3k forward port I found the hack based on the "-c" entry that I remembered (I had mentioned it *right there* in a comment in main.c, so I don't know how I missed it when first updating 2.7 - I suspect I got lost in the ifdef maze inside SetArgV and managed to convince myself it wasn't a problem).
Anyway, it turns out PySys_SetArgV() uses the "sys.argv[0] == '-c'" check to skip over checking the file system for a file by that name. By reusing that same value, the -m code was able to also skip that check without needing to add any additional checks in the sysmodule code.
With the change I made to the 2.x branch, 2.7 now reacts differently if someone creates a "-m" file in the launch directory:
System python is 2.6, local Python is SVN head. Note that the version from SVN changes behaviour after I create the oddly named file, while the system Python is unaffected.
My inclination is to fix this properly for 3.2 (including some extra command line tests to ensure that files named "-c" and "-m" don't confuse the sys.path configuration), but revert the change completely for 2.7.
Added Benjamin for an opinion, since 2.7 is in RC mode already.
benjaminp commentedon Jun 13, 2010
Revert seems sensible in this situation.
ncoghlan commentedon Jun 13, 2010
Reverted in r81964, marking as 3.2 only.
For 3.2, I'll get rid of the hack by having SetArgV treat both "-c" and "-m" as special values for sys.argv[0], and by adding some unit tests that make sure the presence of files with those names has no effect on the value of sys.path[0] when the corresponding command line switches are used.
ncoghlan commentedon Aug 17, 2010
Implemented in r84140.
Leaving issue open until the buildbots give it a clean bill of health (since the command line tests can be a little quirky when it comes to cross platform differences)