datetime.rfcformat() for Date and Time on the Internet

@abalkin

BPO	7584
Nosy	@abalkin, @devdanzin, @merwok, @bitdancer, @durban, @mihaic
Dependencies	bpo-5094: datetime lacks concrete tzinfo implementation for UTC bpo-7989: Add pure Python implementation of datetime module to CPython
Files	issue7584.diff: Patch (py3k branch) issue7584_2.diff: New patch (py3k branch)

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/abalkin'
closed_at = <Date 2012-06-22.01:01:13.351>
created_at = <Date 2009-12-27.19:18:06.729>
labels = ['easy', 'type-feature', 'library']
title = 'datetime.rfcformat() for Date and Time on the Internet'
updated_at = <Date 2012-06-22.01:01:13.351>
user = 'https://bugs.python.org/techtonik'

bugs.python.org fields:

activity = <Date 2012-06-22.01:01:13.351>
actor = 'belopolsky'
assignee = 'belopolsky'
closed = True
closed_date = <Date 2012-06-22.01:01:13.351>
closer = 'belopolsky'
components = ['Library (Lib)']
creation = <Date 2009-12-27.19:18:06.729>
creator = 'techtonik'
dependencies = ['5094', '7989']
files = ['16909', '17275']
hgrepos = []
issue_num = 7584
keywords = ['patch', 'easy']
message_count = 33.0
messages = ['96917', '102670', '102671', '102672', '102700', '103051', '104417', '104418', '104420', '104442', '104463', '104465', '104578', '104615', '104631', '105394', '105399', '105405', '106527', '107000', '107066', '107069', '107139', '107140', '107144', '107160', '110240', '142687', '162729', '162731', '162786', '162787', '163135']
nosy_count = 9.0
nosy_names = ['belopolsky', 'techtonik', 'ajaksu2', 'eric.araujo', 'r.david.murray', 'daniel.urban', 'l0nwlf', 'mihaic', 'poolie']
pr_nums = []
priority = 'normal'
resolution = 'rejected'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue7584'
versions = ['Python 3.2']

RFC 3339 defines a standard for Date and Time on the Internet.
http://www.ietf.org/rfc/rfc3339.txt Given that Python is increasingly
popular on the Internet is should include convenience function to
generate RFC 3339 timestamps in standard library.

It is impossible to generate RFC 3339 timestamps (that are also valid
ISO 8601 timestamps) with datetime.strftime() alone, so the following
code should be used:

import datetime

def rfcformat(dt):
    """ Output datetime in RFC 3339 format that is also valid ISO 8601
        timestamp representation"""

    if dt.tzinfo is None:
        suffix = "-00:00"
    else:
        suffix = dt.strftime("%z")
        suffix = suffix[:-2] + ":" + suffix[-2:]
    return dt.strftime("%Y-%m-%dT%H:%M:%S") + suffix

It looks like feed generators need this feature[1]. See also bpo-5207 and a current implementation[2].

[1] http://validator.w3.org/feed/docs/error/InvalidRFC3339Date.html
[2] http://code.google.com/p/formattime/

Note that the Atom format requires a subset of RFC 3339 (which is in turn a subset of ISO 8601 if I remember correctly, and is the same as the W3C Datetime Format): https://www.tools.ietf.org/html/rfc4287#section-3.3

Summary: Atom requires using an uppercase T between date and time (the RFC permits lower and upper case), and using an uppercase Z for UTC timezone, not “-00:00”.

Regards

Should have checked before speaking. RFC 3339 is a subset of W3CDTF, as said on Daniel’s first link.

Atom spec - http://www.atomenabled.org/developers/syndication/atom-format-spec.php#date.constructs

I've made a patch.

This patch adds a datetime_rfcformat function to datetimemodule.c which is available from Python as the datetime.rfcformat method in the datetime module. This method returns the date in RFC 3339 format: YYYY-MM-DDTHH:MM:SS[.mmmmmm]+HH:MM. Documentation and tests are also added.

Is anyone still interested in this? Is there a problem with my patch?
Thanks.

Daniel : your patch is yet to be reviewed. Please have patience.

Thanks Daniel. I am still interested in this. My Python code as well as your patch doesn't specify that "Z" must be present when time zone offset is unknown or absent, but Atom specification mentions that and I believe this is that most users expect to see. In fact I needed such function for implementing Atom feed for Trac, so in my opinion Z is more clear than -00:00 or +00:00

I also was waiting for a final resolution of bpo-7582 which discusses the proper way to get UTC offset. The function used to correctly build RFC 3339 timestamp from file modification time evolved to the following code:

def isomtime(fname):
    """Return file modification time in RFC 3339 (ISO 8601 compliant) format"""
    stamp = time.localtime(os.stat(fname).st_mtime)

    # determine UTC offset rounded to minutes
    # (see http://bugs.python.org/issue7582 for discussion)
    #   true if file was modified during active DST
    isdst = stamp.tm_isdst
    utcoffset = -(time.altzone if (time.daylight and isdst) else time.timezone) // 60

suffix = "%+03d:%02d" % (utcoffset // 60, abs(utcoffset) % 60)
return time.strftime("%Y-%m-%dT%H:%M:%S", stamp) + suffix

My Python code as well as your patch doesn't specify that "Z" must be
present when time zone offset is unknown or absent,

Yes, that is because RFC 3339 explicitly says (in 4.3.) that -00:00 is "differs semantically from an offset of "Z" or "+00:00", which imply that UTC is the preferred reference point for the specified time". It also says that if "the offset to local time is unknown, this can be represented with an offset of "-00:00"". So I don't think we can write "Z" or "+00:00" if we don't know the UTC offset.

but Atom
specification mentions that

It says "an uppercase "Z" character MUST be present in the absence of a numeric time zone offset". Correct me if I'm wrong, but "-00:00" *is* a numeric time zone offset. So it isn't absent.

and I believe this is that most users
expect to see.

I think the RFC is clear (but correct me if I'm mistaken): we cannot replace "-00:00" with it. We of course could replace "+00:00" with "Z".

In fact I needed such function for implementing Atom
feed for Trac, so in my opinion Z is more clear than -00:00 or +00:00

It may be better than "+00:00", but according to the RFC it simply doesn't mean the same thing as "-00:00".

It also says that if "the offset to local time is unknown, this can
be
represented with an offset of "-00:00"". So I don't think we can write
"Z" or "+00:00" if we don't know the UTC offset.

> but Atom
> specification mentions that

It says "an uppercase "Z" character MUST be present in the absence of
a numeric time zone offset". Correct me if I'm wrong, but "-00:00"
*is* a numeric time zone offset. So it isn't absent.

As quoted earlier, -00:00 is used when time zone is unknown, and "the absence of a numeric time zone offset" or "the absence of offset" is only possible when this time zone is unknown.

I do not think that -00:00 or +00:00 will be invalid Atom timestamp, but to implement parser of rfc3339 timestamp, Z handling is still needed. I can easily imagine people making wrong assumption that parsing 00:00 at the end would be enough for proper round-tripping.

It seems to me that we should not try to produce an RFC 3339 compliant
date string from a naive datetime. It will be practical to accept
that restriction once bpo-5094 is resolved.

Does this mean that t.rfcformat() should fail if t is naive?

According to my reading of RFC 3339, it is not correct to produce 'Z' timestamps when local offset is not known. My understanding is that compliant producers must either know their local timezone and specify correct offset in the suffix or produce UTC timestamps with '-00:00'. Consumers receiving a '...Z' timestamp should be able to rely on that to conclude that the producer is in the UTC+0 timezone. Raising an exception when local offset is not known is OK, but I think generating '-00:00' would be more useful.

Note that overall I am -1 on this proposal. A naive application can probably get away with datetime.isoformat. A strictly compliant RFC 3339 implementation is beyond the scope of datetime module and belongs to a separate module which should probably support parsing of RFC 3339 timestamps as well.

Rather than adding more xxxformat() methods, I would rather see datetime getting a custom __format__ that would be flexible enough to make writing standard formats easy.

I see I didn't think it through far enough.

Given this, it seems that the Atom standard is saying, "if you don't know your actual UTC offset, you can't generate a valid ATOM timestamp". Which sorta makes sense, though you'd think they'd want to accept a -00:00 timestamp since then at least you know when the article was generated/modified, even if you don't know the local time of the poster. And maybe they do, since as someone pointed out -00:00 is a numeric offest...

I agree that generalizing the production of custom formats sounds like a better way forward long term. I'm not clear on why you think RFC3339 deserves its own module.

According to my reading of RFC 3339, it is not correct to produce 'Z'
timestamps when local offset is not known.

It is not said that 'Z' SHOULD NOT be produced if local offset is unknown. Even if offset is unknown, UTC still can be the preferred reference point for the specified time.

From the user point of view the assumption about "know or unknown offset" is beyond the judgement of low-level datetime library. For example, if I read timestamp of a local file, I assume that local offset is unknown and UTC is not preferred, but still do not have other choice than to write zero. It won't help me to find actual offset. For a network file I assume that preferred offset is actually UTC and although I do not have any means to check the offset I can write zero without any hesitation.

Solving philosophical -00:00 +00:00 problems require a deep knowledge of RFC3339. I doubt that pleasing 1% who need this kind of fine-grained semantic control with special API deserves suffering of other 99%. In the end there is nothing hard in replacing 'Z' with a flavor of your choice.

Of course, if smb. want to force developers to overload their brain for the sake of providing the most accurate semantic timestamps (that nobody uses anyway) - then the -00:00/+00:00 stuff is a way to go. But my opinion is that datetime API in Python is already complicated enough to negatively affect this language karma.

s/datetime/date\/time/ in the last sentence, as the complication comes from:
-- start offtopic --

the vast amount of various date and time modules/functions
the little amount of what they can do

I still hardly believe that there is no way to get current TZ offset given that every OS provides such information.
-- end offtopic --

On Sat, Jun 5, 2010 at 11:35 AM, anatoly techtonik
<report@bugs.python.org> wrote:
..

s/datetime/date\/time/ in the last sentence, as the complication comes from:
-- start offtopic --

the vast amount of various date and time modules/functions

"Vast" is a bit of overstatement. It is mostly the datetime and time
module with a little bit in a less used calendar module. I believe a
stated goal is to gradually remove the need for time module in
date/time manipulation and possibly fold it into posix module.

Can you be more specific? The biggest omission, IMO, is the lack of
time zone support, but this is being worked on. See bpo-5094.

I still hardly believe that there is no way to get current TZ offset given that every OS provides such information.

See bpo-1647654.

On Sat, Jun 5, 2010 at 7:00 PM, Alexander Belopolsky
<report@bugs.python.org> wrote:

On Sat, Jun 5, 2010 at 11:35 AM, anatoly techtonik
<report@bugs.python.org> wrote:
..
> s/datetime/date\/time/ in the last sentence, as the complication comes from:
> -- start offtopic --
> 1. the vast amount of various date and time modules/functions

"Vast" is a bit of overstatement. It is mostly the datetime and time
module with a little bit in a less used calendar module.

"Concise" Python datetime module API =)
(without any way to get current UTC offset from operating system)
-------------------
datetime.MINYEAR
datetime.MAXYEAR
class datetime.date
class datetime.time
class datetime.datetime
class datetime.timedelta
class datetime.tzinfo
timedelta.min
timedelta.max
timedelta.resolution
classmethod date.today()
classmethod date.fromtimestamp()
classmethod date.fromordinal()
date.min
date.max
date.resolution
date.year
date.month
date.day
date.replace()
date.timetuple()
date.toordinal()
date.weekday()
date.isoweekday()
date.isocalendar()
date.isoformat()
date.__str__()
date.ctime()
date.strftime(format)
classmethod datetime.today()
classmethod datetime.now()
classmethod datetime.utcnow()
classmethod datetime.fromtimestamp()
classmethod datetime.utcfromtimestamp()
classmethod datetime.fromordinal()
classmethod datetime.combine()
classmethod datetime.strptime()
datetime.min
datetime.max
datetime.resolution
datetime.year
datetime.month
datetime.day
datetime.hour
datetime.minute
datetime.second
datetime.microsecond
datetime.tzinfo
datetime.date()
datetime.time()
datetime.timetz()
datetime.replace()
datetime.astimezone()
datetime.utcoffset()
datetime.dst()
datetime.tzname()
datetime.timetuple()
datetime.utctimetuple()
datetime.toordinal()
datetime.weekday()
datetime.isoweekday()
datetime.isocalendar()
datetime.isoformat()
datetime.__str__()
datetime.ctime()
datetime.strftime()
time.min
time.max
time.resolution
time.hour
time.minute
time.second
time.microsecond
time.tzinfo
time.replace()
time.isoformat()
time.__str__()
time.strftime()
time.utcoffset()
time.dst()
time.tzname()
tzinfo.utcoffset()
tzinfo.dst()
tzinfo.tzname()
tzinfo.fromutc()

Raw doc with stripped markup is only 58161 bytes.
Perhaps the biggest problem that you need to read out the doc
thoroughly to understand that all UTC related stuff starts to work
only when somebody writes a tzinfo module. It is like "you are
developer, so its your headache when you get this data - we gave you
The API". =)

Can you be more specific? The biggest omission, IMO, is the lack of
time zone support, but this is being worked on. See bpo-5094.

Great, but it is Python 3.2 and I will be using 2.x Python for the
next two years at least.
It is not an only about omission - it is also about ineffective and
somehow bloated unpythonic C API that is not well suited for everyday
tasks.

> I still hardly believe that there is no way to get current TZ offset given that every OS provides such information.

See bpo-1647654.

Too long, and I've a little bit tired, and will be pretty busy next
week, but thanks for the pointers.

Adding bpo-7989 as a dependency because having pure python implementation in the CPython tree will make it much easier to experiment with new datetime features.

I am still -1 on adding specialized formatting methods to the datetime class. I think this should be done in specialized modules. On the other hand, datetime module should provide facilities to easily implement such methods and there does not seem to be a good solution for implementing locale independent formats. I would like to consider the idea of adding datetime.cstrftime() which provides formatting equivalent to datetime.strftime() in "C" locale. Another feature that will be needed to implement rfcformat, would be a GNU date style "%:z" code.

Z is well established as meaning "UTC time" <http://en.wikipedia.org/wiki/Coordinated_Universal_Time#Time_zones\> so shouldn't be used for "zone not known." rfc 3393 is clear that it's equivalent to +00:00.

So the questions seem to be:

should there be an included battery to do this format at all?
should it represent utc as '+00:00' or as 'Z' by default - applications should have the choice.

It's probably reasonable to assume correct Python application code using datetime objects will know whether they have a local, utc, or unknown time.

The current patch does not seem to have any way to format an object with a declared UTC tzinfo as having a 'Z' prefix, which would be useful.

David,

Isn't the requested feature now implemented as email.utils.format_datetime()? Also, what is the difference between RFC 3339 format and the one provided by datetime.isoformat?

>>> print(datetime(2000,1,1, tzinfo=timezone.utc).isoformat('T'))
2000-01-01T00:00:00+00:00

email.utils.format_datetime implements RFC 5322 date format, which is very different from RFC 3339. I don't remember enough about what I read in RFC 3339 to answer your question about isoformat.

On Wed, Jul 14, 2010 at 1:52 AM, Alexander Belopolsky
<report@bugs.python.org> wrote:

I am still -1 on adding specialized formatting methods to the datetime class. I think this should be done in specialized modules.

No problem - add "formats" module and I'll be happy.

On the other hand, datetime module should provide facilities to easily implement such methods and there does not seem to be a good solution for implementing locale independent formats. I would like to consider the idea of adding datetime.cstrftime() which provides formatting equivalent to datetime.strftime() in "C" locale. Another feature that will be needed to implement rfcformat, would be a GNU date style "%:z" code.

If there is no best solution then the best out worse solutions is much
much better - provided that an appropriate explanation is available
for the users.

I must add - many times better than no solution at all.

I have reviewed RFC 3339 and it looks like the following produces a fully compliant timestamp:
 
>>> print(datetime(2000,1,1, tzinfo=timezone.utc).isoformat('T'))
2000-01-01T00:00:00+00:00

I see the following remaining issues:

It is often desired to get RFC 3339 timestamp in local timezone instead of UTC. This will be addressed in bpo-9527.
If UTC timestamp is produced by a computer in non-UTC timezone, the offset should be specified as '-00:00'. If this is important, an application can replace '+' with '-', but most likely specifying the correct local offset is a better option.
RFC 3339 requires support for leap seconds. This limitation cannot be solved by adding a method to datetime.

Most importantly, given that there are several RFCs describing different date formats, a datetime.rfcformat() method will be ambiguous. (GNU date has --rfc-2822 and --rfc-3339 options and the later allows output of three different precisions.)

I am going to reject this RFE. I don't think adding datetime.rfcformat() method will solve any real deficiency and whatever limitations datetime has with respect to producing RFC compliant timestamps should be addressed in future specific proposals.

Navigation Menu

Repository navigation

datetime.rfcformat() for Date and Time on the Internet #51833

9 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

datetime.rfcformat() for Date and Time on the Internet #51833

Description

Activity

techtonik commented on Dec 27, 2009

devdanzin commented on Apr 9, 2010

merwok commented on Apr 9, 2010

merwok commented on Apr 9, 2010

techtonik commented on Apr 9, 2010

durban commented on Apr 13, 2010

durban commented on Apr 28, 2010

l0nwlf commented on Apr 28, 2010

techtonik commented on Apr 28, 2010

durban commented on Apr 28, 2010

techtonik commented on Apr 28, 2010

techtonik commented on Apr 28, 2010

9 remaining items

abalkin commented on Jun 4, 2010

bitdancer commented on Jun 4, 2010

techtonik commented on Jun 5, 2010

techtonik commented on Jun 5, 2010

abalkin commented on Jun 5, 2010

techtonik commented on Jun 5, 2010

abalkin commented on Jul 13, 2010

poolie commented on Aug 22, 2011

abalkin commented on Jun 13, 2012

bitdancer commented on Jun 13, 2012

techtonik commented on Jun 14, 2012

techtonik commented on Jun 14, 2012

abalkin commented on Jun 19, 2012

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions