Repository navigation
Replace context enter and exit events with single "context switched" event #124872
Description
Feature or enhancement
Proposal:
(split off of #99633 (comment) to keep related pull requests and discussion better organized)
Starting in v3.14, C code can subscribe to "context enter" and "context exit" events:
cpython/Include/cpython/context.h
Lines 30 to 60 in 120729d
See gh-119333 for some background.
I would like to replace the just-added "context enter" and "context exit" events with a single "context switched" event:
diff --git a/Include/cpython/context.h b/Include/cpython/context.h
index ec72966e82c..f9638b0abe5 100644
--- a/Include/cpython/context.h
+++ b/Include/cpython/context.h
@@ -28,8 +28,12 @@ PyAPI_FUNC(int) PyContext_Enter(PyObject *);
PyAPI_FUNC(int) PyContext_Exit(PyObject *);
typedef enum {
- Py_CONTEXT_EVENT_ENTER,
- Py_CONTEXT_EVENT_EXIT,
+ /*
+ * The current context has switched to a different context. The object
+ * passed to the watch callback is the now-current contextvars.Context
+ * object.
+ */
+ Py_CONTEXT_SWITCHED = 1,
} PyContextEvent;
/*
diff --git a/Python/context.c b/Python/context.c
index ddb03555f9e..6c9f68583c2 100644
--- a/Python/context.c
+++ b/Python/context.c
@@ -192,7 +192,7 @@ _PyContext_Enter(PyThreadState *ts, PyObject *octx)
ts->context = Py_NewRef(ctx);
ts->context_ver++;
- notify_context_watchers(Py_CONTEXT_EVENT_ENTER, ctx, ts);
+ notify_context_watchers(Py_CONTEXT_SWITCHED, ctx, ts);
return 0;
}
@@ -226,13 +226,13 @@ _PyContext_Exit(PyThreadState *ts, PyObject *octx)
return -1;
}
- notify_context_watchers(Py_CONTEXT_EVENT_EXIT, ctx, ts);
Py_SETREF(ts->context, (PyObject *)ctx->ctx_prev);
ts->context_ver++;
ctx->ctx_prev = NULL;
ctx->ctx_entered = 0;
+ notify_context_watchers(Py_CONTEXT_SWITCHED, (PyContext *)ts->context, ts);
return 0;
}
Rationale: Users want to know when the current context switches to a different context object. Right now this happens when and only when a context is entered or exited, so the enter and exit events are synonymous with "switched". However, if the changes proposed for gh-99633 are implemented, the current context will also switch for reasons other than context enter or exit. Since users actually care about context switches and not enter or exit, replacing the enter and exit events with a single switched event would make the new feature compatible with gh-99633.
The current "context exit" event is emitted just before exiting the context. The new "context switched" event would be emitted after the context is exited to match the semantics users expect of an event with a past-tense name. If users need the ability to clean up before the switch takes effect, another "context is about to switch" event can be added in the future. I am not proposing to add such an event for this issue because no users have requested it yet (YAGNI).
I would love to get this resolved before v3.14 is released so that we don't have to worry about breaking backwards compatibility.
Has this already been discussed elsewhere?
I have already discussed this feature proposal on another bug report.
Links to previous discussion of this feature:
Linked PRs
- gh-124872: Move PyThreadState to first argument for consistency #124774
- gh-124872: Refine contextvars documentation #124773
- gh-124872: Back up exception before calling PyContext_WatchCallback #124741
- gh-124872: Change PyContext_WatchCallback to take PyObject #124737
- gh-124872: Replace enter/exit events with "switched" #124776
- [3.13] gh-124872: Refine contextvars documentation (GH-124773) #125233
- gh-124872: Rename blurb file to reference the correct issue #125285
- gh-124872: Replace enter/exit events with "switched" #125532
- gh-124872: Mark the thread's default context as entered #125638
Activity
pythongh-124872: Replace enter/exit events with "switched"
ZeroIntensity commentedon Oct 2, 2024
Generally, it's a bad idea to create several different PRs for the same issue -- you can put all the needed changes in the same PR.
pythongh-124872: Replace enter/exit events with "switched"
rhansen commentedon Oct 2, 2024
Apologies—I was under the impression that this project squashes pull requests when merging, which I don't want (the commits are intended to be separate). I can combine them into a single PR if that would be preferred.
ZeroIntensity commentedon Oct 2, 2024
We do squash, but why do you want the commits to be separate? It's harder to review and backport across multiple PRs
rhansen commentedon Oct 2, 2024
I'm following the Python Developer's Guide:
Each of these PRs is focused on only one thing, and they are independent from each other (but all support the overall goal of this issue).
In general, it's bad practice to have large monolithic commits that do multiple separable things. Small, focused, and independent commits have several advantages:
I can put all of the commits in a single pull request, but if they are squashed then much will be lost.
pythongh-124872: Replace enter/exit events with "switched"
ZeroIntensity commentedon Oct 2, 2024
In that case, it's better to make a new issue for each PR (I don't think it's worth doing so at this point, but this is just for future reference). If they really are all "independent", then having all the PRs under one issue makes this more difficult for us. If, for example, one PR gets merged while another gets shot down, what do we do about this issue? Does it stay closed or open?
Eh, looking through a tree of "related yet independent" PRs is arguably more difficult for people trying to understand design decisions. With one PR, all the reviews are in one place. For example, #124774 looks like a cosmetic change -- it would honestly fit better in a larger PR, where there's lots of other refactoring going on (and also easier to review without being brushed off as cosmetic).
GitHub saves the commits for us :)
In terms of bisecting, we can always checkout the branch that got merged and bisect to individual commits on it. But in my experience, we haven't ever needed to do that. Once we've found a bad commit, it's typically easy to figure out what went wrong without doing any extra micro-bisections.
pythongh-124872: Replace enter/exit events with "switched"
rhansen commentedon Oct 2, 2024
Creating a separate issue per PR seems like a heavyweight process to me. Why have two places to discuss the same thing? Though I understand if it makes it easier for the project to track done/pending status.
Also, I think it's easier to understand relationships between changes if there is one issue with N associated pull requests vs. N separate issues with one corresponding pull request each.
How about: "If there are any open PRs associated with an issue, then the issue is not yet resolved."
I think the ideal solution would be to require contributors to indicate whether their PR is intended to be squashed vs. rebased-then-merged (aka semi-linear history):
mainthen merged with a merge commit (git merge --no-ff). Contributors would be expected to frequently rebase and force-push their PR branch to address review comments, resolve conflicts withmain, etc.Rebase-then-merge PRs would be easier to review/manage/understand vs. N separate PRs, but would still have the advantages of N small and focused commits. The N commits would be grouped together in the Git history, linked to
mainwith a merge commit that references the PR. This makes it easy to see that they are related and all part of a particular pull request. Also, Git's--first-parentoption can be used to bisect/examine the history at PR granularity.Unfortunately, the review experience is still not great when force pushes are involved. Also, GitHub has poor/no support for semi-linear history (GitLab is superior in this regard). It can still be done manually, but this is error-prone and tedious. Maybe bots can help. Maybe someone has already written one.
git blamedoesn't walk GitHub PR commits.This doesn't work for PRs written with the expectation that the commits will be squashed. Such PRs often have intermediate commits that are works-in-progress (they don't compile, don't pass tests, etc.).
True.
ZeroIntensity commentedon Oct 2, 2024
All in all, this isn't common practice for CPython. Let's try to stick with tradition :)
rhansen commentedon Oct 2, 2024
On a related note, it would be nice if this tradition was documented in the devguide. :)
ZeroIntensity commentedon Oct 2, 2024
It is, sort of. There's supposed to be an implied 1:1 correspondence between issues and PRs (with the exception of backports).
pythongh-124872: Replace enter/exit events with "switched"
31 remaining items
pythongh-124872: Mark the thread's initial context as entered
pythongh-124872: Mark the thread's default context as entered
pythongh-124872: Mark the thread's default context as entered
pythongh-124872: Mark the thread's default context as entered
pythongh-124872: Mark the thread's default context as entered
pythongh-124872: Mark the thread's default context as entered
Merge branch '3.13' into pythongh-124872-docs
[3.13] gh-124872: Refine contextvars documentation (GH-124773) (#125233)
Merge branch 'main' into pythongh-124872-enter-initial
fixup! pythongh-124872: Mark the thread's default context as entered
fixup! pythongh-124872: Mark the thread's default context as entered
fixup! pythongh-124872: Mark the thread's default context as entered
fixup! pythongh-124872: Change PyContext_WatchCallback return type to…
fixup! pythongh-124872: Change PyContext_WatchCallback return type to…
pythongh-124872: Replace enter/exit events with "switched" (python#12…