Repository navigation
add st_*time_ns fields to os.stat(), add ns keyword to os.*utime*(), os.*utimens*() expects a number of nanoseconds #58335
Description
| BPO | 14127 |
|---|---|
| Nosy | @gvanrossum, @loewis, @gpshead, @jcea, @ncoghlan, @pitrou, @vstinner, @larryhastings, @merwok, @bitdancer, @peterjc |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/larryhastings'
closed_at = <Date 2012-06-23.03:00:19.086>
created_at = <Date 2012-02-26.09:23:21.502>
labels = ['type-feature', 'library']
title = 'add st_*time_ns fields to os.stat(), add ns keyword to os.*utime*(), os.*utimens*() expects a number of nanoseconds'
updated_at = <Date 2018-01-18.02:59:27.721>
user = 'https://github.com/larryhastings'bugs.python.org fields:
activity = <Date 2018-01-18.02:59:27.721>
actor = 'jcea'
assignee = 'larry'
closed = True
closed_date = <Date 2012-06-23.03:00:19.086>
closer = 'larry'
components = ['Library (Lib)']
creation = <Date 2012-02-26.09:23:21.502>
creator = 'larry'
dependencies = []
files = ['24838', '24859', '24874', '24875', '24884', '24908', '24951', '24952', '24954', '25354', '25356', '25448', '25451', '25452']
hgrepos = []
issue_num = 14127
keywords = ['patch']
message_count = 86.0
messages = ['154316', '154319', '154334', '154355', '154362', '154363', '154365', '154366', '154367', '154368', '154377', '154378', '154383', '154389', '154863', '154864', '154865', '154866', '154868', '154869', '154870', '154871', '154872', '154873', '154874', '155138', '155605', '155608', '155620', '155632', '155635', '155724', '155740', '155747', '155773', '155774', '155784', '155868', '155919', '155938', '155940', '155941', '155942', '155944', '155946', '155962', '156000', '156181', '156364', '156366', '156368', '156369', '156375', '157423', '158764', '158771', '159219', '159221', '159222', '159239', '159361', '159410', '159416', '159637', '159836', '159837', '159840', '159841', '159857', '159871', '159891', '159904', '159909', '159911', '159912', '159913', '159914', '159919', '159920', '159921', '159922', '159924', '159925', '160050', '160054', '163526']
nosy_count = 16.0
nosy_names = ['gvanrossum', 'loewis', 'gregory.p.smith', 'jcea', 'ncoghlan', 'pitrou', 'vstinner', 'larry', 'eric.araujo', 'Arfrever', 'r.david.murray', 'maubp', 'shaurz', 'rosslagerwall', 'python-dev', 'sbt']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue14127'
versions = ['Python 3.3']
Activity
larryhastings commentedon Feb 26, 2012
Following on from Guido's rejection of PEP-410:
http://mail.python.org/pipermail/python-dev/2012-February/116837.html
This bug is the proposed hammering-out space for how to preserve exact metadata for st_atime / st_mtime between os.stat and os.utime.
(Yes, there's already bpo-11457 -- but that went pretty far down the rabbit hole of Decimal. I thought maybe a fresh start would be best.)
larryhastings commentedon Feb 26, 2012
Guido proposed st_atime_ns et al. I'll make an alternate proposal.
I'd like to avoid tying ourselves to ns resolution. As MvL said: "I don't want to deal with this issue *again* in my lifetime".
If all we want is to facilitate copying the exact metadata from os.stat to os.utime, then we don't really care about conveniently modifying the timestamp. So I propose we use MvL's suggestion of a tuple of ints. Either (numerator, denominator) or (seconds, fractional_numerator, fractional_denominator). (These are mentioned in PEP-410 under the heading "Tuple of ints" as options A and B respectively.) Name the fields with the suffix "_exact" (e.g. st_mtime_exact). os.stat and its ilk produce it; os.utime and its ilk consume it. If people want to monkey with the values and do math, let 'em--consenting adults.
loewis commentedon Feb 26, 2012
OTOH, it seems that Guido is very much in favor of hard-coding ns resolution in the API, claiming that expectations of even finer resolution are academic. While I still disagree (the *very* same argument was brought up for ms resolution ten years ago), I think that practicality beats purity here: people apparently can deal with an fixed scale much better than with a floating one. So it may be better to meet the apparent intuition of the majority than follow the purity route.
bitdancer commentedon Feb 26, 2012
Is it totally insane to think about using a float subclass with an additional attribute instead of a tuple?
gvanrossum commentedon Feb 26, 2012
@rdm: yes, that's insane for this purpose. The size of the solution doesn't match the size of the problem (the problem is non-existent for most people).
@larry: That seems unnecessarily general. I take full responsibility for fixing the precision at ns. And note that I *am* giving you a way out -- I'm establishing a clear pattern, and if someone really, desperately wants ps or fs, it's completely obvious how to change the API again. But I bet you a case of beer that won't happen in our lifetimes (or in Benjamin's). So the dynamic precision is just unnneeded complexity.
larryhastings commentedon Feb 26, 2012
I suggest that publishing nanoseconds as a plain int would be a nasty API. Consider what it would do to os.utime:
Have we ever published an API that treated a parameter as two wildly different numbers based solely on whether the parameter was an int or a float?
larryhastings commentedon Feb 26, 2012
(D'oh. My pseudo-code should have said
for that second assignment. Hopefully my point was still clear.)
pitrou commentedon Feb 26, 2012
How about a separate float attribute for the fractional part (in seconds)? This gives a femtosecond precision (assuming a 50 bits mantissa). The utime() could accept a (integral part, fractional part) tuple to set a timestamp without losing bits.
(PHP does something similar, but badly, with its microtime() function:
http://www.php.net/manual/en/function.microtime.php)
larryhastings commentedon Feb 26, 2012
@pitrou: What would that look like when passed in to os.utime?
pitrou commentedon Feb 26, 2012
Of course, the fact that utime takes a (atime, mtime) tuple makes this a bit cumbersome.
When passed a tuple, utime would ignore the fractional part of the first element.
loewis commentedon Feb 26, 2012
No, it wouldn't. Please re-read Guido's proposal. If you want to
specify nanoseconds, you have to pass the ns= parameter. My only
quibble with the specific spelling is that it invokes Godwin's law
(but I can live that that as a theoretical concern, also).
No, and Guido is on the record for objecting such APIs. Hence the
keyword parameter.
gvanrossum commentedon Feb 26, 2012
Thanks Martin. I'd be happy with nsec instead of ns.
larryhastings commentedon Feb 26, 2012
My mistake! That ought to teach me to bound out of bed Sunday morning with my "brilliant realization". (But it probably won't.)
I volunteer to implement this, with the new custom class for the os.stat object. I'll have it done no later than the PyCon sprints next month.
For the record, I actually prefer "ns". "s" is the SI standard abbreviation for second (as is "n" for nano-):
http://en.wikipedia.org/wiki/International_System_of_Units#Units_and_prefixes
If we're going to abbreviate, I think it's reasonable to go with their standard. But I'm willing to be overridden.
67 remaining items
sbt commentedon May 4, 2012
Without the check for RuntimeError
raises
because we have fallen through to the narrow path. The correct error is
larryhastings commentedon May 4, 2012
Let me recap, just to make sure I have it straight. There are two errors on Windows:
you get the wrong error: effectively it's complaining that the string
is not narrow, when it should be complaining about having both times
and ns.
For the former, obviously removing the ! is correct. But I like your idea of making the utime_read_time_argument() return value tell you what happened; that's what the Windows code needs to know.
So here it is! Please see the attached patch.
sbt commentedon May 4, 2012
That's right. The patch looks good and passes for me on Windows.
python-dev commentedon May 4, 2012
New changeset fc5d2f4291ac by Larry Hastings in branch 'default':
Issue bpo-14127: Fix two bugs with the Windows implementation.
http://hg.python.org/cpython/rev/fc5d2f4291ac
sbt commentedon May 4, 2012
There is another problem causing a fatal error in test_posix on Unix.
The attached patch fixes it: *ua->path should be decrefed not ua->path.
larryhastings commentedon May 4, 2012
Looks good to me. You're a core contributor, yes? If not let me know and I'll commit it.
Though I must admit I'm baffled how I haven't seen that crash. I've run the unit tests a zillion times on this patch.
sbt commentedon May 4, 2012
I will commit.
Were you running test_posix or only test_os?
larryhastings commentedon May 4, 2012
By "the unit tests" I meant I ran the whole suite: Lib/test/regrtest.py. I know that runs test_os, and I assume it runs test_posix too.
I just ran test_posix by hand and it passed.
I'm developing on Linux (64-bit) in case that helps.
sbt commentedon May 4, 2012
I tested it on 32 bit Linux.
I have committed it, but I forgot to put the issue number in the commit message.
loewis commentedon May 4, 2012
Revision 4deb7c1f49b9
python-dev commentedon May 6, 2012
New changeset 709850f1ec67 by Larry Hastings in branch 'default':
Update Misc/NEWS for issues bpo-14127 and bpo-14705. (And, technically, bpo-10148.)
http://hg.python.org/cpython/rev/709850f1ec67
python-dev commentedon May 6, 2012
New changeset 05274ab06182 by Larry Hastings in branch 'default':
Update Misc/NEWS for issues bpo-14127 and bpo-14705. (And, technically, bpo-10148.)
http://hg.python.org/cpython/rev/05274ab06182
larryhastings commentedon Jun 23, 2012
Closing this, as I have now removed os.utimensat and os.futimesat. As well as os.futimens, os.futimes, and os.lutimes. And in fact retooled os.utime in a pretty major way. (See bpo-14626.)