Repository navigation
Add sys.setasthook() to allow to use a custom AST optimizer #61717
Description
| BPO | 17515 |
|---|---|
| Nosy | @brettcannon, @vstinner |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2015-03-18.11:05:53.988>
created_at = <Date 2013-03-22.00:58:11.302>
labels = []
title = 'Add sys.setasthook() to allow to use a custom AST optimizer'
updated_at = <Date 2015-03-28.14:41:42.333>
user = 'https://github.com/vstinner'bugs.python.org fields:
activity = <Date 2015-03-28.14:41:42.333>
actor = 'brett.cannon'
assignee = 'none'
closed = True
closed_date = <Date 2015-03-18.11:05:53.988>
closer = 'vstinner'
components = []
creation = <Date 2013-03-22.00:58:11.302>
creator = 'vstinner'
dependencies = []
files = ['29539', '29540']
hgrepos = []
issue_num = 17515
keywords = ['patch']
message_count = 7.0
messages = ['184932', '184936', '184971', '184978', '184986', '238416', '239459']
nosy_count = 2.0
nosy_names = ['brett.cannon', 'vstinner']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue17515'
versions = ['Python 3.4']
Activity
vstinner commentedon Mar 22, 2013
I wrote an optimizer rewriting the AST to implement various optimizatoions:
https://bitbucket.org/haypo/astoptimizer
To be able to use it, or use any other kind of AST transformation, I propose to add a new sys.setasthook() function to set a custom AST hook.
Attached patch is a proof-of-concept. The patch should be improved: document the function and add write unit tests.
Prototype of the hook:
I don't know if we need to add a compiler option (like _ast.PyCF_ONLY_AST) to skip the hook.
vstinner commentedon Mar 22, 2013
enable_astoptimizer.patch: example of usage of the new function, you can use it to try astoptimizer.
brettcannon commentedon Mar 22, 2013
This is unnecessary. I added the source_to_code() method (http://docs.python.org/3.4/library/importlib.html#importlib.abc.SourceLoader.source_to_code) to loaders to explicitly handle this case.
vstinner commentedon Mar 22, 2013
Your change only concerns imports. eval() and compile() cannot be
hooked using your method.
2013/3/22 Brett Cannon <report@bugs.python.org>:
brettcannon commentedon Mar 22, 2013
And that's fine as it allows for more explicit control over what gets optimized. Basically if there is a global hook directly into the compiler there needs to be more control over detecting it's being optimized, what to do about .pyo files, etc. At least at the loader level it's much more modular and controlled as to what will and will not get the optimizations.
So if you want to pursue this I think there needs to be more thought than just a global sys hook for this sort of thing.
vstinner commentedon Mar 18, 2015
I'm not more interested to work on my astoptimizer project, and nobody looks to need sys.setasthook(), so I prefer to close the issue.
brettcannon commentedon Mar 28, 2015
The closing of this issue inspired me to think of a better way to do this, and I think providing something in importlib.util that accepted a NodeTransformer as an argument to a function that compiled source files to bytecode files would fill this issue thanks to PEP-488. It would mean that nothing is done dynamically and everything is done offline, but that also means startup costs are gone in regards to bytecode generation and it gets people to actually think about optimizations of Python instead of it just being an afterthought.
importlib.util.compile(path: Union[str, pathlib.Path], transformer: ast.NodeTransformer = None, *, optimization: Union[str, int] = None) -> pathlib.Path