Document importlib.resources.abc.Traversable #91298

Activity

1. encukou commented on Mar 28, 2022

   

   encukou

   on Mar 28, 2022

   MemberAuthor

   More actions

   importlib.resources.files is documented 0 as returning importlib.resources.abc.Traversable, which is an undocumented class (dead link) now. It should be documented.
   The source has helpful docstrings, but is missing some details. I'd like to know these before writing some docs:

   • Can the argument to joinpath contain path separators?
   • Are the methods expected to raise specific exception types (e.g. if a resource is missing, or you call iterdir on a “file”)? This seems quite important for people who implement the protocol.
2. encukou

   assigned

   docspython

   on Mar 28, 2022
3. encukou

   added

   docsDocumentation in the Doc dirDocumentation in the Doc dir

   on Mar 28, 2022
4. encukou

   assigned

   docspython

   on Mar 28, 2022
5. encukou

   added

   docsDocumentation in the Doc dirDocumentation in the Doc dir

   on Mar 28, 2022
6. jaraco

   assigned

   jaraco

   and unassigned

   docspython

   on Mar 28, 2022
7. jaraco commented on Apr 3, 2022

   

   jaraco

   on Apr 3, 2022

   Member

   More actions

   Can the argument to joinpath contain path separators?

   Good question. Currently, I'm aware of three concrete implementations of Traversable.joinpath:

   • pathlib.Path.joinpath
   • zipfile.Path.joinpath
   • importlib.resources.simple.ResourceContainer.joinpath

   Traversable.joinpath was modeled after pathlib.Path.joinpath, which itself is itself not rigorous in what path separators are allowed. The docstring does indicate that each of the parameters will be combined with the existing path and acknowledges that absolute paths are possible. The curated docs only give examples, and curiously, those examples only use posixpath.sep, suggesting that it's recommended when passing compound paths to use posixpath separators as they're more widely supported.

   For zipfile.Path, I observe that joinpath does accept path separators but looking at the docs for zipfile, it's not even obvious to me what path separators are used in zipfiles, but I do observe there is coercion to posixpath.sep.

   ResourceContainer, on the other hand, will not accept path separators. This class, however, may not be used anywhere and probably could be extended to support path separators as well.

   Here's what I propose:

   • First, document that the interface is under-specified and that for maximum compatibility, a consumer should pass only relative paths using posixpath.sep or no separator at all. Additionally, explicitly declare that behavior with absolute paths is undefined and unsupported.
   • Second, expand the interface on Traversable.joinpath to stipulate that the interface should accept multiple relative paths.
   • Third, simple.ResourceContainer should be updated to meet that spec.
8. jaraco commented on Apr 3, 2022

   

   jaraco

   on Apr 3, 2022

   Member

   More actions

   Are the methods expected to raise specific exception types (e.g. if a resource is missing, or you call iterdir on a “file”)?

   The short answer is no. The protocol does not stipulate specific exceptions. Perhaps it should be documented that any exceptions that occur when accessing the backing resource will propagate. Although, I do believe that statement is true for all Python code unless stated otherwise.

   I agree it would be nice if the protocol could stipulate which exceptions might be raised by a given implementation, but I'm not confident that's possible, at least not without wrapping/adapting the pathlib.Path and zipfile.Path objects.

   Does that answer the question? Are you working on another provider that implements this protocol? I'd be happy to consult on that effort.
9. jaraco commented on Apr 3, 2022

   

   jaraco

   on Apr 3, 2022

   Member

   More actions

   Correction. In msg416618, link should have been:

   docs for zipfile
10. ezio-melotti

    transferred this issue fromon Apr 10, 2022
11. encukou commented on Apr 13, 2022

    

    encukou

    on Apr 13, 2022

    MemberAuthor

    More actions

    not rigorous in what path separators are allowed

    Right, that could be improved. IMO it could link to the constructor docs that have all the details.

    Forward slashes work on all supported systems. On Windows they're equivalent to backslashes (everywhere except in old CLI commands, AFAIK). So it makes sense to normalize to them.

    >>> PureWindowsPath('a/b\\c')
    PureWindowsPath('a/b/c')

    pathlib.Path.joinpath also allows .. and . segments in the relative paths. I assume those should not be allowed?

    any exceptions [...] will propagate
    Does that answer the question?

    Yes. Thank you!
    I was thinking about how legal code like this would be:

    try:
        return pkg_files.joinpath(f'templates/{user_supplied_name}').read_text()
    except (NoSuchFileError, IsADirectoryError):
        raise ...

    which suggests another question: should I expect resilience against path traversal attacks?

    Are you working on another provider that implements this protocol?

    No, I'm just trying to make sense of it all.
12. jaraco

    closed this as completedin #91623on Apr 17, 2022
13. jaraco

    added a commit that references this issue on Apr 17, 2022

    gh-91298: Refine traversable (apply changes from importlib_resources …

    ...

    7659681