Repository navigation
threading.Lock.acquire(timeout) should use sem_clockwait(CLOCK_MONOTONIC) #85876
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2021-10-01.16:52:29.583>
created_at = <Date 2020-09-04.08:34:54.835>
labels = ['interpreter-core', 'type-bug', '3.9', '3.10', '3.11']
title = 'threading.Lock.acquire(timeout) should use sem_clockwait(CLOCK_MONOTONIC)'
updated_at = <Date 2021-12-06.13:27:11.081>
user = 'https://bugs.python.org/wocket'bugs.python.org fields:
activity = <Date 2021-12-06.13:27:11.081>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2021-10-01.16:52:29.583>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2020-09-04.08:34:54.835>
creator = 'wocket'
dependencies = []
files = ['49440']
hgrepos = []
issue_num = 41710
keywords = ['patch']
message_count = 22.0
messages = ['376338', '376340', '376394', '399405', '402925', '402926', '402931', '402939', '402942', '402987', '402995', '402997', '403001', '403003', '403009', '403011', '403012', '403021', '403022', '403023', '403962', '407802']
nosy_count = 5.0
nosy_names = ['vstinner', 'methane', 'miss-islington', 'wocket', 'mikecrowe']
pr_nums = ['28629', '28642', '28643', '28644', '28647', '28662', '28671', '28672', '28673', '28674', '28676', '28683', '28962']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue41710'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']
Activity
wocket commentedon Sep 4, 2020
The timeout for threading.Lock, threading.Condition, etc, is not using a monotonic clock — it is affected if the system time (realtime clock) is set.
The attached program can be used to show the problem. It is expected to print "Took 2.000 s" repeatedly, but if run with permissions to set the system time, it prints:
$ sudo ./time_test.py
Took 2.400 s
Took 1.657 s
Took 2.044 s
Took 2.401 s
...
(the 1.6 s time can be explained by NTP correcting the clock)
There are already a number of closed bugs for this and related issues: bpo-23428, bpo-31267, bpo-35747.
This happens in Python 3.7.7 (ARM32, Yocto Warrior), Python 3.8.2 (AMD64, Ubuntu Linux 20.04) and Python v3.9.0rc1 (AMD64, Ubuntu Linux 20.04).
vstinner commentedon Sep 4, 2020
bpo-23428 modified the pthread implementation of conditional variable to use pthread_condattr_setclock(&ca, CLOCK_MONOTONIC) is available: commit 001fee1. The change should be part of Python 3.8.
What is your sys.thread_info value? Example on Fedora 32 with Python 3.8.5:
Sadly, the semaphore implementation doesn't use monotonic clock. See glibc issues:
wocket commentedon Sep 4, 2020
In my tests it works as expected on a Macintosh (3.8.5 with lock='mutex+cond') and also if I force a Linux build to use the mutex+cond implementation by defining HAVE_BROKEN_POSIX_SEMAPHORES.
Doesn't look like those glibc and Linux bug reports will get any attention anytime soon. I will find a workaround instead :-/
mikecrowe commentedon Aug 11, 2021
glibc v2.30 onwards provides sem_clockwait which can wait on either CLOCK_MONOTONIC or CLOCK_REALTIME. I failed to notice that https://sourceware.org/bugzilla/show_bug.cgi?id=14717 existed until today. :(
[-]Timeout is affected by jumps in system time[/-][+]acquire(timeout) of threading.Lock and threading.Condition is affected by jumps in system time: Python should use sem_clockwait(CLOCK_MONOTONIC)[/+][-]Timeout is affected by jumps in system time[/-][+]acquire(timeout) of threading.Lock and threading.Condition is affected by jumps in system time: Python should use sem_clockwait(CLOCK_MONOTONIC)[/+]vstinner commentedon Sep 30, 2021
New changeset 09796f2 by Victor Stinner in branch 'main':
bpo-41710: Add _PyTime_AsTimespec_clamp() (GH-28629)
09796f2
vstinner commentedon Sep 30, 2021
New changeset d62d925 by Victor Stinner in branch 'main':
bpo-41710: Add pytime_add() and pytime_mul() (GH-28642)
d62d925
vstinner commentedon Sep 30, 2021
New changeset 0231b6d by Victor Stinner in branch 'main':
bpo-41710: Fix building pytime.c on Windows (GH-28644)
0231b6d
9 remaining items
vstinner commentedon Oct 1, 2021
Fixed bpo-12822 modified threading.Condition.wait(timeout) to use a monotonic clock: use pthread_condattr_setclock(CLOCK_MONOTONIC). bpo-23428, bpo-31267 and bpo-35747 are duplicates of bpo-12822.
This issue is about threading.Lock.acquire(timeout) which has a different implementation.
vstinner commentedon Oct 1, 2021
New changeset 98d2827 by Victor Stinner in branch 'main':
bpo-41710: Fix PY_TIMEOUT_MAX on Windows (GH-28673)
98d2827
vstinner commentedon Oct 1, 2021
New changeset 54957f1 by Victor Stinner in branch 'main':
bpo-41710: gc_collect_main() uses _PyTime_GetPerfCounter() (GH-28676)
54957f1
vstinner commentedon Oct 1, 2021
New changeset 833fdf1 by Victor Stinner in branch 'main':
bpo-41710: Add private _PyDeadline_Get() function (GH-28674)
833fdf1
vstinner commentedon Oct 1, 2021
New changeset 6df8c32 by Victor Stinner in branch '3.10':
bpo-41710: PyThread_acquire_lock_timed() uses sem_clockwait() (GH-28671)
6df8c32
vstinner commentedon Oct 1, 2021
New changeset 0e1aeab by Miss Islington (bot) in branch '3.9':
bpo-41710: PyThread_acquire_lock_timed() uses sem_clockwait() (GH-28671) (GH-28683)
0e1aeab
vstinner commentedon Oct 1, 2021
Jonas Norling: Thanks for the bug report! It's now fixed in 3.9, 3.10 and main branches.
vstinner commentedon Oct 14, 2021
New changeset 03bbc60 by Victor Stinner in branch 'main':
bpo-41710: Fix What's New Entry credit (GH-28962)
03bbc60
vstinner commentedon Dec 6, 2021
bpo-33632 "undefined behaviour: signed integer overflow in threadmodule.c" has been fixed by this change:
New changeset 833fdf1 by Victor Stinner in branch 'main':
bpo-41710: Add private _PyDeadline_Get() function (GH-28674)
833fdf1
pythongh-108724: Use _PyTime_GetMonotonicClock() in parking_lot.c