Repository navigation
tzinfo objects with sub-minute offsets are not supported (e.g. UTC+05:53:28) #49538
Description
| BPO | 5288 |
|---|---|
| Nosy | @gvanrossum, @tim-one, @mdickinson, @abalkin, @vstinner, @benjaminp |
| PRs | |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/abalkin'
closed_at = <Date 2017-07-31.14:27:55.922>
created_at = <Date 2009-02-17.04:33:24.503>
labels = ['extension-modules', 'easy', 'type-feature', '3.7']
title = 'tzinfo objects with sub-minute offsets are not supported (e.g. UTC+05:53:28)'
updated_at = <Date 2017-12-13.02:13:30.662>
user = 'https://bugs.python.org/jamesh'bugs.python.org fields:
activity = <Date 2017-12-13.02:13:30.662>
actor = 'p-ganssle'
assignee = 'belopolsky'
closed = True
closed_date = <Date 2017-07-31.14:27:55.922>
closer = 'belopolsky'
components = ['Extension Modules']
creation = <Date 2009-02-17.04:33:24.503>
creator = 'jamesh'
dependencies = []
files = ['17844', '17865', '17874', '17875', '17897']
hgrepos = []
issue_num = 5288
keywords = ['easy']
message_count = 23.0
messages = ['82299', '106374', '108606', '109151', '109262', '109347', '109348', '109504', '109508', '109510', '174494', '174742', '174760', '248468', '251870', '299240', '299254', '299330', '299377', '299389', '299394', '299395', '299557']
nosy_count = 8.0
nosy_names = ['gvanrossum', 'tim.peters', 'jamesh', 'mark.dickinson', 'belopolsky', 'vstinner', 'benjamin.peterson', 'Fergus.Noble']
pr_nums = ['2896', '4699']
priority = None
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue5288'
versions = ['Python 3.7']
Activity
jamesh commentedon Feb 17, 2009
The datetime module does not support time zones whose offset from UTC is
not an integer number of minutes.
The Olson time zone database (used by most UNIX systems and Mac OS X)
has a number of time zones with historic offsets that use second
resolution (from before those locations switched to a rounded offset
from GMT).
If you are working purely with the Python date time module, you can
round these offsets to whole numbers of minutes and get consistent
results (this is what the pytz module does), there are problems if you
want to interact with other programs using the Olson database.
As an example, PostgreSQL can return dates with sub-minute UTC offsets
and it would be nice to be able to represent them exactly.
The documentation doesn't explain why offsets need to be a whole number
of minutes, and the interface (returning timedeltas) doesn't look like
it'd need changing to support second offsets. I wouldn't expect any
more complexity in the code to support them either.
abalkin commentedon May 24, 2010
As far as I can tell, the TZ offset code can be simplified by eliminating conversion from timedelta to int and back in utcoffset() and dst() methods of time and datetime objects.
The only reason for the restriction that I can think of is that some text representation of datetime only provide 4 digits for timezone. The behavior of formatting functions, can be preserved while allowing fractional minute offsets elsewhere.
On the other hand, given that this issue did not see a single response in more than a year, it seems that there is little interest in fixing it.
abalkin commentedon Jun 25, 2010
This proved to require a lot of changes to C implementation because most of timezone arithmetics is done using integer operations with offset in minutes. It is easy, however to do this in pure python implementation which can be found at http://svn.python.org/view/*checkout*/sandbox/branches/py3k-datetime/datetime.py .
Marking this "easy" to attract volunteers who may want to do Python prototype.
abalkin commentedon Jul 2, 2010
I am attaching a patch against sandbox version of datetime.py. With this patch, there is a single place where subminute offset is rejected, _check_utc_offset() function. I have also added "whole minute" asserts in places where sub-minute part of the offset is discarded or assumed 0.
abalkin commentedon Jul 4, 2010
I am attaching a rough patch which removes timedelta -> int minutes -> timedelta round trips from utcoffset handling code. I think the result is an improvement, but needs more polishing.
Mark,
Do you think this is worth pursuing? I am not intending to add support for sub-minute offsets yet, just pass offsets around as timedeltas internally without unnecessary conversions to int and back.
abalkin commentedon Jul 5, 2010
I am attaching a test coverage file for the patched datetimemodule.c.
abalkin commentedon Jul 5, 2010
It looks like I attached coverage for the original file. See new-datetimemodule.c.gcov for coverage after the change.
abalkin commentedon Jul 7, 2010
Uploaded bpo-5288.diff to Rietveld:
http://codereview.appspot.com/1698050
abalkin commentedon Jul 7, 2010
issue5288a.diff addressed comments from Rietveld.
abalkin commentedon Jul 8, 2010
C code changes eliminating round-trips between timdelta and int offsets committed in r82642 and Python code changes committed to sandbox in r82641.
Now the requested behavior change is easy and I am about +0.5 on relaxing the offset checks to allow seconds in the offsets and about +0.1 on removing the checks altogether and allowing arbitrary timedeltas even over 1 day.
The only substantive question in my mind is how to handle printing in formats like %z which only provide 4 digits for UTC offset. Two options seem reasonable: a) truncate sub-minute offsets; and b) follow RFC 3339 recommendation and translate the time into the nearest whole-minute timezone.
I am slightly is favor of (a), but this whole problem may be a reason to reject this RFE.
8 remaining items
abalkin commentedon Jul 26, 2017
Most of the code supporting arbitrary offsets has already been committed. The only part left was to remove the checks and implement printing.
abalkin commentedon Jul 27, 2017
In PR 2896, I've modified %z formatting code to output sub-minute data if present. I think %z parsing should be also modified to accept sub-minute data, but I would like to do it in the context of bpo-24954. Thoughts?
vstinner commentedon Jul 28, 2017
James Henstridge:
Ok for increasing the resolution to seconds. But PR 2896 increases the resolution to microseconds. What is the rationale for supporting microseconds?
I would prefer to only accept microseconds=0.
abalkin commentedon Jul 28, 2017
Victor,
Tim called for removal of all restrictions on the offsets. See msg248468. I left the range restriction intact because we have some algorithms that rely on that, but in general I agree with Tim. There is nothing to be gained from restricting the offsets. It is just some extra code to enforce gratuitous limitations.
Note that Olson's database limits its precision to seconds for historical reasons. The mean solar time offsets that they record are known to subsecond precision.
I did add a few lines of code to support subsecond formatting, but at some point we should be able to unify timedelta and timezone formatting.
vstinner commentedon Jul 28, 2017
My concern is that it makes timestamp parsing more complex because we would have to handle the theorical case of timezone with microsecond precision.
abalkin commentedon Jul 28, 2017
To the contrary. The timezone field can now be parsed the same way as the time field plus the sign.
abalkin commentedon Jul 31, 2017
New changeset 018d353 by Alexander Belopolsky in branch 'master':
Closes issue bpo-5288: Allow tzinfo objects with sub-minute offsets. (bpo-2896)
018d353
Ensure Timestamp.ToDatetime(tz) has correct offset
Ensure Timestamp.ToDatetime(tz) has correct offset
Ensure Timestamp.ToDatetime(tz) has correct offset
Ensure Timestamp.ToDatetime(tz) has correct offset