Skip to content

gh-89279: In ceval.c, redefine some macros for speed #32387

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gvanrossum merged 27 commits into from
Apr 22, 2022
Merged

gh-89279: In ceval.c, redefine some macros for speed #32387

Changes from 1 commit
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
0be9d62
TEMPORARY hacks to show inline decisions
gvanrossum Apr 6, 2022
abd9118
In ceval.c, redefine some macros for speed
gvanrossum Apr 7, 2022
1724056
Merge remote-tracking branch 'origin/main' into inline
gvanrossum Apr 7, 2022
50e3d7b
Victor's review
gvanrossum Apr 7, 2022
59da3bf
Merge branch 'python:main' into inline
gvanrossum Apr 7, 2022
ab6660e
Add _Py_atomic_load_32bit_impl and use cast macro
gvanrossum Apr 7, 2022
ae6b53c
Move _Py_atomic_load_32bit_impl out of '#if'
gvanrossum Apr 7, 2022
01cff81
Go back to full PGO test suite
gvanrossum Apr 8, 2022
69da380
Longer comment; improve _Py_atomic_load_relaxed
gvanrossum Apr 8, 2022
2894f6e
Only specialize atomic-load-relaxed if MSC
gvanrossum Apr 9, 2022
4b70976
Try _Py_atomic_load_relaxed_int32
gvanrossum Apr 9, 2022
5617fcd
MSC is better off with the default Py_UNREACHABLE()
gvanrossum Apr 11, 2022
a901b91
Update bpo numbers to GH issue numbers
gvanrossum Apr 11, 2022
cceb531
Split accidentally merged lines
gvanrossum Apr 13, 2022
26e8107
Delete temporary /d2inlinelogfull option
gvanrossum Apr 13, 2022
11084c0
Merge branch 'main' into inline
gvanrossum Apr 13, 2022
b7dfcdc
📜🤖 Added by blurb_it.
blurb-it[bot] Apr 13, 2022
9a5c57c
Use static_assert() in _Py_atomic_load_relaxed_int32
gvanrossum Apr 19, 2022
a8cba6e
Revert "MSC is better off with the default Py_UNREACHABLE()"
gvanrossum Apr 19, 2022
c43ec56
Add comment referring to GH-89279 for is_method()
gvanrossum Apr 19, 2022
9a15194
static_assert() is only a C++ feature
gvanrossum Apr 19, 2022
5c992d2
Merge remote-tracking branch 'origin/main' into inline
gvanrossum Apr 20, 2022
7b342e4
Also redefine _Py_DECREF_SPECIALIZED as a macro
gvanrossum Apr 20, 2022
3b21c52
Reformat macros for readability
gvanrossum Apr 21, 2022
41cb067
Try to silence warnings
gvanrossum Apr 21, 2022
87aaf81
Update Python/ceval.c
vstinner Apr 21, 2022
3508f45
Update Python/ceval.c
vstinner Apr 21, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Longer comment; improve _Py_atomic_load_relaxed
  • Loading branch information
@gvanrossum
gvanrossum committed Apr 8, 2022
commit 69da380ab4c46c3878def4156912a1d894ddf2a9
15 changes: 11 additions & 4 deletions Python/ceval.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,9 +44,12 @@
# error "ceval.c must be build with Py_BUILD_CORE define for best performance"
#endif

#if !defined(Py_DEBUG)
// bpo-45116: The MSVC compiler fails to inline these in PGO build,
// and they're kind of important for performance.
#ifndef Py_DEBUG

@vstinner vstinner Apr 19, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if only MSVC is affected, only redefine these macros for MSVC, no?

Suggested change
#ifndef Py_DEBUG
#if defined(_MSC_VER) && !defined(Py_DEBUG)

@gvanrossum gvanrossum Apr 19, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer us running the same code on all platforms if possible, so that it's less likely that someone accidentally breaks the Windows version by want appears to be an innocent change on Linux.

(I have to make an exception for load_relaxed because it is heavily platform-specific.)

@vstinner vstinner Apr 20, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be extra safe, you may also check Py_REF_DEBUG, even if Py_REF_DEBUG should not be defined if Py_DEBUG is defined.

Suggested change
#ifndef Py_DEBUG
#if !defined(Py_DEBUG) && !defined(Py_REF_DEBUG)

Or maybe object.h should fail with #error with Py_REF_DEBUG is defined whereas Py_DEBUG is not.

// bpo-45116: The MSVC compiler does not inline these static inline functions
// in PGO build in _PyEval_EvalFrameDefault(), because this function is over
// the limit of PGO, and that limit cannot be configured.
// Define them as macros to make sure that they are always inlined by the
// preprocessor.

#undef Py_DECREF
#define Py_DECREF(arg) do { PyObject *op = _PyObject_CAST(arg); if (--op->ob_refcnt == 0) { destructor d = Py_TYPE(op)->tp_dealloc; (*d)(op); } } while (0)

@vstinner vstinner Apr 20, 2022 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Such long line is one of the reason why I wrote PEP 670 :-) I propose to make the code more readable by putting one statement per line, as done in static inline functions:

#ifndef Py_DEBUG
// GH-89279: The MSVC compiler does not inline these static inline functions
// in PGO build in _PyEval_EvalFrameDefault(), because this function is over
// the limit of PGO, and that limit cannot be configured.
// Define them as macros to make sure that they are always inlined by the
// preprocessor.

#undef Py_DECREF
#define Py_DECREF(arg) \
    do { \
        PyObject *op = _PyObject_CAST(arg); \
        if (--op->ob_refcnt == 0) { \
            destructor dealloc = Py_TYPE(op)->tp_dealloc; \
            (*dealloc)(op); \
        } \
    } while (0)

#undef Py_XDECREF
#define Py_XDECREF(arg) \
    do { \
        PyObject *xop = _PyObject_CAST(arg); \
        if (xop != NULL) { \
            Py_DECREF(xop); \
        } \
    } while (0)

#undef Py_IS_TYPE
#define Py_IS_TYPE(ob, type) \
    (_PyObject_CAST_CONST(ob)->ob_type == (type))

#endif

// GH-89279: Similar to above, force inlining by using a macro.
#if defined(_MSC_VER) && SIZEOF_INT == 4
#define _Py_atomic_load_relaxed_int32(ATOMIC_VAL) \
    (assert(sizeof((ATOMIC_VAL)->_value) == 4), \
     *((volatile int*)&((ATOMIC_VAL)->_value)))
#else
#define _Py_atomic_load_relaxed_int32(ATOMIC_VAL) \
    _Py_atomic_load_relaxed(ATOMIC_VAL)
#endif

I declared Py_XDECREF just after Py_DECREF. I also renamed "op1" to "xop" in Py_XDECREF.

UPDATE: Oh, and to copy/paste Py_IS_TYPE(), I used _PyObject_CAST_CONST().

Expand All @@ -57,9 +60,13 @@
#undef Py_XDECREF
#define Py_XDECREF(arg) do { PyObject *op1 = _PyObject_CAST(arg); if (op1 != NULL) { Py_DECREF(op1); } } while (0)

#if SIZEOF_INT == 4
#undef _Py_atomic_load_relaxed
#define _Py_atomic_load_relaxed(ATOMIC_VAL) (*((volatile int*)&((ATOMIC_VAL)->_value)))
#endif
Comment thread
gvanrossum marked this conversation as resolved.

#endif

#define _Py_atomic_load_32bit_impl(value, order) (assert((order) == _Py_memory_order_relaxed), *(value))

/* Forward declarations */
static PyObject *trace_call_function(
Expand Down