Skip to content

Inefficient signal handling in multithreaded applications #84191

Closed
Closed
Inefficient signal handling in multithreaded applications#84191
Labels
3.9 (EOL)end of lifeinterpreter-core(Objects, Python, Grammar, and Parser dirs)performancePerformance or resource usage

Description

@vstinner
vstinner
opened on Mar 19, 2020
BPO 40010
Nosy @vstinner, @ericsnowcurrently
PRs
  • bpo-40010: Optimize signal handling in multithreaded applications #19067
  • bpo-40010: Pass tstate to ceval GIL functions #19077
  • bpo-39877: 4th take_gil() fix for daemon threads #19080
  • bpo-40010: COMPUTE_EVAL_BREAKER() checks for subinterpreter #19087
  • bpo-40010: Optimize pending calls in multithreaded applications #19091
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2020-03-25.17:33:17.981>
created_at = <Date 2020-03-19.00:57:19.125>
labels = ['interpreter-core', '3.9', 'performance']
title = 'Inefficient signal handling in multithreaded applications'
updated_at = <Date 2020-05-05.15:27:40.795>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2020-05-05.15:27:40.795>
actor = 'eric.snow'
assignee = 'none'
closed = True
closed_date = <Date 2020-03-25.17:33:17.981>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2020-03-19.00:57:19.125>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 40010
keywords = ['patch']
message_count = 8.0
messages = ['364580', '364583', '364611', '364628', '364659', '364666', '364667', '368171']
nosy_count = 2.0
nosy_names = ['vstinner', 'eric.snow']
pr_nums = ['19067', '19077', '19080', '19087', '19091']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue40010'
versions = ['Python 3.9']

    Activity

    1. vstinner commented on Mar 19, 2020

      @vstinner
      vstinner
      on Mar 19, 2020
      MemberAuthor

      When a thread gets a signal, SIGNAL_PENDING_SIGNALS() sets signals_pending to 1 and eval_breaker to 1. In this case, _PyEval_EvalFrameDefault() calls handle_signals(), but since it's not the main thread, it does nothing and signals_pending value remains 1. Moreover, eval_breaker value remains 1 which means that the following code will be called before executing *each* bytecode instruction.

          if (_Py_atomic_load_relaxed(eval_breaker)) {
        (...)
        opcode = _Py_OPCODE(*next_instr);
        if (opcode == SETUP_FINALLY || ...) {
            ...
        }
        if (_Py_atomic_load_relaxed(&ceval->signals_pending)) {
            if (handle_signals(tstate) != 0) {
                goto error;
            }
        }
        if (_Py_atomic_load_relaxed(&ceval->pending.calls_to_do)) {
            ...
        }

        if (_Py_atomic_load_relaxed(&ceval->gil_drop_request)) {
            ...
        }
        if (tstate->async_exc != NULL) {
            ...
        }
    }

      This is inefficient.

      I'm working on a PR modifying SIGNAL_PENDING_SIGNALS() to not set eval_breaker to 1 if the current thread is not the main thread.

    2. vstinner
      added
      3.9 (EOL)end of life
      interpreter-core(Objects, Python, Grammar, and Parser dirs)
      performancePerformance or resource usage
      on Mar 19, 2020

    3. vstinner commented on Mar 19, 2020

      @vstinner
      vstinner
      on Mar 19, 2020
      MemberAuthor

      I found this issue while working on PR 19066 of bpo-39984.

    4. vstinner commented on Mar 19, 2020

      @vstinner
      vstinner
      on Mar 19, 2020
      MemberAuthor

      New changeset 5a3a71d by Victor Stinner in branch 'master':
      bpo-40010: Optimize signal handling in multithreaded applications (GH-19067)
      5a3a71d

    5. vstinner commented on Mar 19, 2020

      @vstinner
      vstinner
      on Mar 19, 2020
      MemberAuthor

      New changeset a36adfa by Victor Stinner in branch 'master':
      bpo-39877: 4th take_gil() fix for daemon threads (GH-19080)
      a36adfa

    6. vstinner commented on Mar 20, 2020

      @vstinner
      vstinner
      on Mar 20, 2020
      MemberAuthor

      New changeset da2914d by Victor Stinner in branch 'master':
      bpo-40010: Pass tstate to ceval GIL functions (GH-19077)
      da2914d

    7. vstinner commented on Mar 20, 2020

      @vstinner
      vstinner
      on Mar 20, 2020
      MemberAuthor

      New changeset d2a8e5b by Victor Stinner in branch 'master':
      bpo-40010: COMPUTE_EVAL_BREAKER() checks for subinterpreter (GH-19087)
      d2a8e5b

    8. vstinner commented on Mar 20, 2020

      @vstinner
      vstinner
      on Mar 20, 2020
      MemberAuthor

      New changeset d831688 by Victor Stinner in branch 'master':
      bpo-40010: Optimize pending calls in multithreaded applications (GH-19091)
      d831688

    9. vstinner
      changed the title [-]Inefficient sigal handling in multithreaded applications[/-] [+]Inefficient signal handling in multithreaded applications[/+] on Mar 20, 2020
    10. vstinner
      changed the title [-]Inefficient sigal handling in multithreaded applications[/-] [+]Inefficient signal handling in multithreaded applications[/+] on Mar 20, 2020
    11. vstinner
      closed this as completedon Mar 25, 2020
    12. vstinner
      closed this as completedon Mar 25, 2020

    13. ericsnowcurrently commented on May 5, 2020

      @ericsnowcurrently
      ericsnowcurrently
      on May 5, 2020
      Member

      Good catch on this, Victor. Thanks for doing it.

    14. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.9 (EOL)end of lifeinterpreter-core(Objects, Python, Grammar, and Parser dirs)performancePerformance or resource usage

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions