Skip to content

bpo-41530: Handle unhandled IsADirectoryError and PermissionError in zoneinfo.ZoneInfo #21839

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
bijij wants to merge 13 commits into from
Closed

bpo-41530: Handle unhandled IsADirectoryError and PermissionError in zoneinfo.ZoneInfo #21839

Changes from 10 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
aa75c26
Add unhandled exceptions to zoneinfo._common.load_tzdata
bijij Aug 12, 2020
dbf37fd
Ensure changes follow PEP-8 specifications.
bijij Aug 12, 2020
65f2569
Remove erroneously added testing code.
bijij Aug 12, 2020
768bdb3
Remove whitespace on blank line
bijij Aug 12, 2020
4cbaa99
Remove ValueError from handled exceptions
bijij Aug 12, 2020
8ad2ec7
Fix whitespace for exception block
bijij Aug 12, 2020
0065b49
Rework to catch all OSError exceptions
bijij Aug 12, 2020
db83f0d
Remove unneccessary import guards from backport
bijij Aug 12, 2020
bbf42e3
Remove whitespace
bijij Aug 12, 2020
eafd623
Fix whitespace issue on line 34
bijij Aug 12, 2020
ff1d9ed
Restrict exceptions handled to IsADirectoryError & PermissionError
bijij Aug 13, 2020
c2e6fca
📜🤖 Added by blurb_it.
blurb-it[bot] Aug 13, 2020
814d98c
Simplify path determination
bijij Aug 13, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion Lib/zoneinfo/_common.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ def load_tzdata(key):
try:
return importlib.resources.open_binary(package_name, resource_name)
except (ImportError, FileNotFoundError, UnicodeEncodeError):
# There are three types of exception that can be raised that all amount
# There are some types of exception that can be raised that all amount
# to "we cannot find this key":
#
# ImportError: If package_name doesn't exist (e.g. if tzdata is not
Expand All @@ -22,6 +22,23 @@ def load_tzdata(key):
# UnicodeEncodeError: If package_name or resource_name are not UTF-8,
# such as keys containing a surrogate character.
raise ZoneInfoNotFoundError(f"No time zone found with key {key}")
except OSError as exc:
Comment thread
bijij marked this conversation as resolved.
Outdated
# Exceptions inherited from OSError can be raised in various scenarios:
#
# IsADirectoryError: If the resource_name links to a directory
# (e.g. Australia)
# PermissionError: If the resource_name links to a directory on Windows
# (e.g. Pacific)
import os
import importlib.util

package_spec = importlib.util.find_spec(package_name)
resource_path = os.path.join(os.path.dirname(package_spec.origin), key)

@vstinner vstinner Aug 12, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imports+two lines of code is a risk of raising a new exception. You may do that before and use the path to load the resource. I'm not sure.

Is it possible that tzdata is in a ZIP file, or in memory but not on disk?

Maybe it's overcomplicated and maybe it's better to convert any OSError into a ZoneInfoNotFoundError. I let @pganssle decides what is the best.

@pganssle pganssle Aug 12, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I don't like this at all. In importlib_resources there's a way to determine if something is a directory or not using the importlib.resources.files API.

That will complicate things a bit for the backport, though, which doesn't currently require importlib_resources in Python 3.8.

I'm inclined to go the simple way and catch all OSErrors instead of the more narrowly scoped ones. I realize this may frustrate some people debugging in unusual environments by disguising the source of some errors, but I don't love that the standard ZoneInfo constructor is exposing details about the specifics of the underlying file system.

@pganssle pganssle Aug 12, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jaraco Do you know if there's an equivalent of importlib.resources.files('my.package').joinpath('resource').isdir() that works for Python 3.8's version of importlib.resources?

@jaraco jaraco Aug 12, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Python 3.8 and earlier, importlib.resources didn't support directories as resources - all resources were required to be files directly in a package.

@pganssle pganssle Aug 12, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jaraco Well, we're not really concerned with using a directory as a resource, the problem is that it appears that importlib.resources raises PermissionError on Windows when a directory is specified, and we're trying to distinguish between the case where a purported resource exists but is a directory and the case of a legitimate permissions error.

Is there no way to do that in the Python 3.8 importlib.resources?

@bijij bijij Aug 13, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

swapped to using importlib.resources.files to locate the resource however inclined to agree that catching all OSError derivatives might prove less problematic.

@jaraco jaraco Mar 4, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pganssle Apologies for having dropped the ball on this one.

I don't believe there's a reliable in the older importlib.resources API to detect that a directory is present. It was meant to be the case that a directory was indistinguishable from 'not a resource'.

What I would recommend though is to use importlib.resources here, but in the zoneinfo backport, use importlib_resources, which implements the files() API even on Python 2.


if os.path.isdir(resource_path):
raise ZoneInfoNotFoundError(f"No time zone found with key {key}")
else:
raise


def load_data(fobj):
Expand Down