Repository navigation
gh-139899: Introduce MetaPathFinder.discover and PathEntryFinder.discover #139900
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
8099249
gh-139899: Introduce MetaPathFinder.discover and PathEntryFinder.disc…
FFY00 6816705
Fix doc reference
FFY00 31d1a8f
Remove specific doc references
FFY00 051cd1e
Fix docstrings
FFY00 c343d32
Revert "Remove specific doc references"
FFY00 a324d96
Fix news references
FFY00 dc6faa2
Add docs warning
FFY00 3be757f
Raise ValueError on invalid parent
FFY00 0da477f
Dedupe __path__ in PathFinder.discover
FFY00 282bef7
Use context manager and add error handling to os.scandir
FFY00 469dc2a
Raise ValueError on invalid parent
FFY00 c09a12e
Dedupe when package exists with multiple suffixes
FFY00 41cd071
Apply suggestions from code review
FFY00 5488e93
Add tests
FFY00 fc1de5f
Merge branch 'main' into gh-139899
brettcannon File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Revert "Remove specific doc references"
This reverts commit 31d1a8f. Signed-off-by: Filipe Laíns <lains@riseup.net>
- Loading branch information
commit c343d328b90068df086b3482c498130fff305b82
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
would this be the first time we have a public API
yielding things from importlib, do we actually want to do that or should this return a list?callers consuming the results might be writing code that makes changes that could impact future results... yielding could get messy.
what are the intended use cases for the API? if it's something we expect callers to short circuit and stop iterating on after the first match maybe yield makes sense, but then we should probably just have an explicit direct discover_first API for that instead.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is something I considered.
The main use-case is finding a similar-named module to show as a hint on
ModuleNotFoundError(eg. "Did you meant numpy?", when trying to import numby), so I think it would make sense to make this new API a generator, or at least some kind of lazy container.For cases such as you describe, the user could just consume the full generator into a list to avoid any issue. Still leaving opportunity for the code that could leverage the benefit of this being a lazy API — scaning directories with a lot of files can take a while, not to mention the other exotic finders out there that may operate over the network or something like that.
I am not fundamentally opposed to make this method return a list, but I can't see the value in the trade-of if we document it properly.
While this is technically possible, I would find it extremely uncommon. And I think it should be reasonable to assume that people who are knowledgeable enough to do that, would probably be aware of the downsides of making changes to the import machinery, while consuming the API
And what would that look like? Would it take a predicate function and return the first entry that matches?
So, would have a warning in the documentation regarding your concern be a good enough compromise?
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
True, for the "nearest matching module name" error message case, you just want to keep the closest match, so even though all the names need to be checked, you only need to keep a reference to one of them (the closest so far).
Since the potential number of results may be absurdly high in some situations, and the iterable form provides more options for consumers to handle that appropriately, I do think it makes sense to make it an iterable. The warning in the docs should explain why it's an iterable, since consumers that unconditionally convert the result to a list can run into problems regardless.
As far as prior examples of iterable import APIs goes,
importlib.resources.contentswas the first example I found in a quick look (covering a very similar situation, just for non-module resources rather than submodules)