Repository navigation
Change os.utime &c functions to use nanosecond precision where possible #57113
Description
| BPO | 12904 |
|---|---|
| Nosy | @loewis, @larryhastings |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/larryhastings'
closed_at = <Date 2012-02-26.21:50:32.693>
created_at = <Date 2011-09-05.21:08:02.180>
labels = ['extension-modules', 'type-feature']
title = 'Change os.utime &c functions to use nanosecond precision where possible'
updated_at = <Date 2012-02-26.21:50:32.691>
user = 'https://github.com/larryhastings'bugs.python.org fields:
activity = <Date 2012-02-26.21:50:32.691>
actor = 'larry'
assignee = 'larry'
closed = True
closed_date = <Date 2012-02-26.21:50:32.693>
closer = 'larry'
components = ['Extension Modules']
creation = <Date 2011-09-05.21:08:02.180>
creator = 'larry'
dependencies = []
files = ['23104']
hgrepos = []
issue_num = 12904
keywords = []
message_count = 7.0
messages = ['143563', '143566', '143568', '143571', '143587', '143750', '154399']
nosy_count = 4.0
nosy_names = ['loewis', 'larry', 'Arfrever', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue12904'
versions = ['Python 3.3']
Activity
larryhastings commentedon Sep 5, 2011
Since Linux 2.5 stat() has supported reading nanosecond resolution (1b/sec) for atime and mtime. However, the utime() family of functions could only write atime and mtime with microsecond resolution (1m/sec) until relatively recently.
On POSIX platforms Python reads atime and mtime at nanosecond resolution but only writes them at microsecond resolution. This impedance mismatch can cause undesirable behavior. Consider the following code:
When run under Python on any modern Linux system, the assert fails. (I think any Python since 2.5 will do; I tested with 2.7.1 and a fresh 3.3 trunk.)
The accompanying patch modifies Modules/posixmodule.c so all functions that write atime and mtime use nanosecond resolution when possible. With this patch applied, all the regression tests pass (except the ones for extension modules I didn't compile), and the above code runs to completion.
Happy to hoist the patch up on Rietveld if there's interest.
p.s. I have the commit bit set, so I'd like to be the one to check this in if we get that far.
larryhastings commentedon Sep 5, 2011
A small aside:
An IEEE 754 double is insufficient to store a modern timestamp with nanosecond accuracy. We're currently at just over 1.3 billion seconds since the epoch. This eats 28 bits of the mantissa. (The value is actually 31 bits to the left of the decimal point, but you get the leading 1 and the subsequent two 0s for free.) Precisely storing a billion values to the right of the decimal point would require another 30 bits. The mantissa of a double is sadly only 52 bits. So we're shy by 6 bits, give or take.
So this patch doesn't make Python 100% accurate with respect to atime/mtime. What it *does* do is makes Python's loss of accuracy consistent. I claim that's a definite improvement.
Arfrever commentedon Sep 5, 2011
It's a duplicate of issue bpo-11457.
(Python >=3.3 has os.utimensat() and os.futimens().)
larryhastings commentedon Sep 5, 2011
No. bpo-11457 is a how-many-angels-on-the-head-of-a-pin discussion proposing new representations of ctime/mtime/atime to preserve nanosecond accuracy. This patch changes the behavior of os.utime, os.futimes, os.lutimes, and os.futimesat so they conserve all of the existing representation's accuracy, making them consistent with os.stat.
How is that relevant?
I mean, sure, I leveraged some of the work from that support in my patch--it's heavily reliant on the preprocessor macros HAVE_UTIMENSAT and HAVE_FUTIMENS now generated by configure. (Thanks, Ross!) But the presence of those functions does not mitigate the loss of accuracy inside os.utime &c.
loewis commentedon Sep 6, 2011
The patch looks fine to me.
python-dev commentedon Sep 9, 2011
New changeset 1de6619733d9 by Larry Hastings in branch 'default':
Issue bpo-12904: os.utime, os.futimes, os.lutimes, and os.futimesat now write
http://hg.python.org/cpython/rev/1de6619733d9
larryhastings commentedon Feb 26, 2012
This was committed last September; I should have marked it closed then.