Repository navigation
Use the monotonic clock for thread conditions on POSIX platforms #67616
Description
| BPO | 23428 |
|---|---|
| Nosy | @vstinner, @stratakis, @pablogsal, @anikey-m |
| PRs | |
| Superseder | |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2019-02-20.01:01:03.341>
created_at = <Date 2015-02-09.21:34:27.880>
labels = ['interpreter-core', '3.8']
title = 'Use the monotonic clock for thread conditions on POSIX platforms'
updated_at = <Date 2021-10-01.08:48:32.784>
user = 'https://github.com/vstinner'bugs.python.org fields:
activity = <Date 2021-10-01.08:48:32.784>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2019-02-20.01:01:03.341>
closer = 'methane'
components = ['Interpreter Core']
creation = <Date 2015-02-09.21:34:27.880>
creator = 'vstinner'
dependencies = []
files = ['38069']
hgrepos = []
issue_num = 23428
keywords = ['patch']
message_count = 12.0
messages = ['235637', '235639', '235640', '236156', '273565', '333841', '333844', '333845', '333846', '333851', '336060', '376341']
nosy_count = 5.0
nosy_names = ['vstinner', 'neologix', 'cstratak', 'pablogsal', 'anikey']
pr_nums = ['4964']
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '12822'
type = None
url = 'https://bugs.python.org/issue23428'
versions = ['Python 3.8']
Activity
vstinner commentedon Feb 9, 2015
Python 3.5 now requires a monotonic clock to start and has the C function _PyTime_monotonic().
Python/condvar.h and Python/thread_pthread.h should use the monotonic clock CLOCK_MONOTONIC, not the system clock CLOCK_REALTIME. See the PEP-418 for the rationale.
Most platforms support pthread_condattr_setclock(CLOCK_MONOTONIC), except Mac OS X and old versions of Android.
The glib looks to use pthread_cond_timedwait_relative_np() for Mac OS X:
https://mail.gnome.org/archives/commits-list/2014-February/msg07782.html
Note: Android had non-standard pthread_cond_timedwait_monotonic() and pthread_cond_timedwait_monotonic_np() functions. Android is not a official supported platform, and newer Android version now support pthread_condattr_setclock(). I prefer to not support old Android versions (yet).
https://android-review.googlesource.com/#/c/83881/
--
For Windows, SleepConditionVariableSRW() is used on Windows 7 and newer, otherwise WaitForSingleObjectEx() is used. By the way, the check looks to be done during the compilation. I should check which Windows version is used to build Python...
SleepConditionVariableSRW() and WaitForSingleObjectEx() both take a relative timeout, so they don't use (directly) the system clock. I don't see any required change for Windows.
According to the PEP-418: "WaitForSingleObject() uses the same timer as GetTickCount() with the same precision."
Hum, it is not possible to interrupt such wait() with CTRL+c? time.sleep() is implemented with WaitForSingleObjectEx() with _PyOS_SigintEvent(). It doesn't look to be the case here.
vstinner commentedon Feb 9, 2015
Oh, I forgot that Python/thread_pthread.h only uses pthread_cond_timedwait() if semaphores are emulated with mutexes+conditional variables.
On most platforms, PyThread_acquire_lock_timed() is implemented with sem_timedwait(). Problem: sem_timedwait() requires an absolute time using the CLOCK_REALTIME clock and the clock is not yet configurable on Linux :-(
See the feature request in the glibc: "Bug 14717 - Allow choice of clock source for calls to sem_timedwait() and pthread_mutex_timedwait()" opened in 2012:
https://sourceware.org/bugzilla/show_bug.cgi?id=14717
Note: QNX provides sem_timedwait_monotonic().
vstinner commentedon Feb 9, 2015
cond_timedwait_monotonic.patch: Work-in-progress patch. It doesn't change configure.ac yet to check if pthread_condattr_setclock() is supported (with CLOCK_MONOTONIC).
vstinner commentedon Feb 17, 2015
Oh, I missed the issue bpo-12822 which looks to fix similar bugs.
stratakis commentedon Aug 24, 2016
Hello,
Is there any progress on the issue? Should someone take over?
vstinner commentedon Jan 17, 2019
I marked bpo-31267 "threading.Timer object is affected by changes to system time: Python locks should use a monotonic clock if available" as a duplicate of this issue.
vstinner commentedon Jan 17, 2019
I closed bpo-35747 "Python threading event wait influenced by date change" as a duplicate of the issue.
vstinner commentedon Jan 17, 2019
It's a limitation of the libc, not directly of Python.
The problem is that the sem_timedwait() function of the glibc doesn't allow to specify which clock is used:
https://sourceware.org/bugzilla/show_bug.cgi?id=14717
Someone has to contribute to the glibc to add an option to sem_init() or sem_timedwait() to allow to use a different clock than CLOCK_REALTIME.
One workaround is to use Python to use the mutex+cond implementation of pthread locks, since this one is already able to use CLOCK_MONOTONIC:
https://bugs.python.org/issue31267#msg302257
vstinner commentedon Jan 17, 2019
See also bpo-12822: "NewGIL should use CLOCK_MONOTONIC if possible".
pablogsal commentedon Jan 17, 2019
Does this have any drawbacks?
vstinner commentedon Feb 20, 2019
INADA-san fixed bpo-12822 with:
New changeset 001fee1 by Inada Naoki in branch 'master':
bpo-12822: use monotonic clock for condvar if possible (GH-11723)
001fee1
vstinner commentedon Sep 4, 2020
See bpo-41710 "Timeout is affected by jumps in system time".