Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
8eeb4fe
Add _PyDictKeys_StringLookupAndVersion
mpage Dec 10, 2024
fcd05a0
Pass shared keys version to specialization
mpage Dec 10, 2024
5c03db0
Add support for enabling each of the instance attribute kinds
mpage Dec 10, 2024
a576748
Only cache deferred descriptors
mpage Dec 10, 2024
8475fd6
Make analyze_descriptor thread-safe
mpage Dec 11, 2024
29c3356
Use an atomic load for GUARD_TYPE_VERSION
mpage Dec 11, 2024
4f9eeb3
Use atomics to load valid bit for inline values in _GUARD_DORV_VALUES…
mpage Dec 11, 2024
e9050fc
Use atomic to load keys version in _GUARD_KEYS_VERSION
mpage Dec 11, 2024
ade57f2
Use an atomic load for managed dict in `_CHECK_ATTR_METHOD_LAZY_DICT`
mpage Dec 11, 2024
0a87264
Enable specialization of method loads
mpage Dec 11, 2024
816e22f
Use atomics for fetching type flags
mpage Dec 11, 2024
ca1e232
Get a strong reference to dict in instance_has_key
mpage Dec 11, 2024
945b61c
Split specialize_dict_access
mpage Dec 11, 2024
feb7d34
Pass type version to specialize_dict_access
mpage Dec 12, 2024
ecfd199
Take a critical section around dict specialization
mpage Dec 12, 2024
9fda5db
Use thread-safe version of _PyDictKeys_StringLookup
mpage Dec 12, 2024
3b2c220
Use atomic load in _CHECK_MANAGED_OBJECT_HAS_VALUES
mpage Dec 12, 2024
408e44b
Make _LOAD_ATTR_INSTANCE_VALUE thread-safe
mpage Dec 12, 2024
16aab70
Make _LOAD_ATTR_WITH_HINT thread-safe
mpage Dec 12, 2024
d0920ea
Specialize instance accesses
mpage Dec 12, 2024
e7cea82
Specialize LOAD_ATTR_SLOT
mpage Dec 12, 2024
8dac8c4
Enable LOAD_ATTR_PROPERTY
mpage Dec 13, 2024
d29b3aa
Checkpoint LOAD_ATTR_GETATTRIBUTE_OVERRIDEN
mpage Dec 13, 2024
b0b8102
Lock dict in instance_has_key
mpage Dec 13, 2024
85dab0d
Lock dict instead of owner when specializing for dict access
mpage Dec 13, 2024
581869e
Use atomic load for valid bit
mpage Dec 13, 2024
96be738
Use _PyDictKeys_StringLookup and lock around it, rather than wasting …
mpage Dec 13, 2024
9afe052
Fix cases_generator bug
mpage Dec 13, 2024
e190a0d
Fix load
mpage Dec 13, 2024
8c78369
Remove FT_UNIMPLEMENTED
mpage Dec 13, 2024
cdf8eb5
Specialize class attribute loads
mpage Dec 13, 2024
1398699
Fix test_type_lookup_mro_reference
mpage Dec 14, 2024
3fbe18b
Remove TYPE_CHANGED
mpage Dec 14, 2024
5aa68cc
Misc clean ups
mpage Dec 14, 2024
e1bf1f4
Check that inline values still valid after taking CS
mpage Dec 14, 2024
396edb1
Enable test_opcache tests
mpage Dec 18, 2024
e5e2508
Skip test that triggers gh-127773 in refleak tests
mpage Dec 20, 2024
fa02260
Merge branch 'main' into gh-115999-load-attr
mpage Dec 20, 2024
47c794f
Add test for LOAD_ATTR_CLASS_WITH_METACLASS_CHECK
mpage Dec 20, 2024
3876bc7
Undo workaround for cases generator bug
mpage Dec 20, 2024
08d14d0
Fix whitespace after merge
mpage Dec 20, 2024
56e9a8a
Double check that dict hasn't changed after locking it
mpage Dec 20, 2024
8ca405b
Use correct type for loading type version
mpage Dec 20, 2024
4c7a4b9
Remove unnecessary comma
mpage Dec 20, 2024
1b787b3
Use atomics when loading oparg
mpage Dec 20, 2024
b868363
Fix formatting
mpage Dec 20, 2024
d6d4c73
Always return type version from analyze_descriptor_load
mpage Dec 20, 2024
9755562
Merge branch 'main' into gh-115999-load-attr-instance-merged
mpage Dec 23, 2024
9673f78
Combine incref/steal into new stackref
mpage Dec 23, 2024
6c3041f
Fix formatting
mpage Dec 23, 2024
a20a4a4
Remove unnecessary error check for PyUnicode_Type.tp_hash
mpage Dec 23, 2024
35b31c6
Check keys kind explicitly
mpage Dec 23, 2024
e5a7ae9
Pass dict from `_CHECK_ATTR_WITH_HINT` to `_LOAD_ATTR_WITH_HINT`
mpage Dec 23, 2024
bb00f5a
Fix unused variable warning
mpage Dec 23, 2024
b6ae487
Update number of specialization failure kinds
mpage Dec 24, 2024
11a351d
Clarify construction of deferred object
mpage Dec 24, 2024
6f8aebf
Add suppression for _PyUnicode_CheckConsistency
mpage Dec 24, 2024
07b7c20
Merge branch 'main' into gh-115999-load-attr-instance-merged
mpage Jan 2, 2025
1870885
Fix compiler warning
mpage Jan 2, 2025
f1fdcc6
Make it easier to reason about thread-safety of instance_has_key
mpage Jan 2, 2025
8b71951
Restore refactor lost in merge
mpage Jan 2, 2025
7190a2e
Merge branch 'main' into gh-115999-load-attr-instance-merged
mpage Jan 2, 2025
8b96368
Revert "Add suppression for _PyUnicode_CheckConsistency"
mpage Jan 2, 2025
db02869
Merge branch 'main' into gh-115999-load-attr-instance-merged
mpage Jan 8, 2025
dc3ba7f
Merge branch 'main' into gh-115999-load-attr-instance-merged
mpage Jan 9, 2025
e07fb62
Split check_keys_and_hash
mpage Jan 9, 2025
a3f89b7
Clear inline values upon invalidation
mpage Jan 9, 2025
3baa840
Replace kill/pop_dead_inputs with pop_input
mpage Jan 10, 2025
9250e4e
Fix formatting
mpage Jan 10, 2025
675eb82
Fix visibility
mpage Jan 10, 2025
d24dc3a
Merge branch 'main' into gh-115999-load-attr-instance-merged
mpage Jan 10, 2025
10d693d
Use release instead of seq_cst
mpage Jan 10, 2025
7ab3ec6
Make POP_INPUT take the name of the TOS
mpage Jan 10, 2025
8c70ea4
Merge branch 'main' into gh-115999-load-attr-instance-merged
mpage Jan 14, 2025
3616eab
Use DECREF_INPUTS() instead of manually closing inputs
mpage Jan 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Make _LOAD_ATTR_INSTANCE_VALUE thread-safe
- Use atomic load for value
- Use _Py_TryIncrefCompareStackRef for incref
  • Loading branch information
mpage committed Dec 18, 2024
commit 408e44b23b1c3dcb9586400e71adc09ac345522e
12 changes: 9 additions & 3 deletions Python/bytecodes.c
Original file line number Diff line number Diff line change
Expand Up @@ -2142,12 +2142,18 @@ dummy_func(
split op(_LOAD_ATTR_INSTANCE_VALUE, (offset/1, owner -- attr, null if (oparg & 1))) {
PyObject *owner_o = PyStackRef_AsPyObjectBorrow(owner);
PyObject **value_ptr = (PyObject**)(((char *)owner_o) + offset);
PyObject *attr_o = *value_ptr;
PyObject *attr_o = FT_ATOMIC_LOAD_PTR_ACQUIRE(*value_ptr);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do we know that the values array is still valid at this point?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a case that we are not handling correctly here: if, after loading value_ptr, another thread invalidates the inline values, modifies an attribute, and reallocates another object at the same memory address as the previous attribute, we can incorrectly return a value that wasn't the attribute at that offset.

We need to either:

  • Re-check _PyObject_InlineValues(owner_o)->valid at the end of this handler and clean-up and deopt if it's not still true.
  • Set the contents of the inlines values array to NULL when we mark it as invalid so that the _Py_TryIncrefCompareStackRef handles this case and correctly deopts.

I think the second options is probably simpler to implement and more efficient given that _LOAD_ATTR_INSTANCE_VALUE is common and invalidating inline values is relatively rare.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Invalidating inline values might not be that rare.
I suggest gathering some stats before deciding how to handle this.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From
https://github.com/faster-cpython/benchmarking-public/blob/main/results/bm-20250104-3.14.0a3%2B-0cafa97/bm-20250104-azure-x86_64-python-0cafa97932c6574734bb-3.14.0a3%2B-0cafa97-pystats.md:

LOAD_ATTR_INSTANCE_VALUE: 4,922,474,147
Materialize dict (on request) | 4,444,396 | 2.4%
Materialize dict (new key) | 476,375 | 0.3%
Materialize dict (too big) | 4,884 | 0.0%
Materialize dict (str subclass) | 0 | 0.0%

So, LOAD_ATTR_INSTANCE_VALUE is somewhere between 1,000x-10,000x more frequent than invalidating inline values. (Invalidating inline values requires first materializing the dictionary, but materializing the dictionary doesn't always invalidate the inline values).

DEOPT_IF(attr_o == NULL);
STAT_INC(LOAD_ATTR, hit);
#ifdef Py_GIL_DISABLED
if (!_Py_TryIncrefCompareStackRef(value_ptr, attr_o, &attr)) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does _Py_TryIncrefCompareStackRef do?
Function names need to be self explanatory. The result of this function does not depend on a comparison, but on whether something has been modified. Maybe _Py_TryIncrefIfPointersConsistentStackRef as it only works if the `PyObject ** and PyObject *pointers are consistent.

I know that _Py_TryIncrefCompareStackRef was not introduced in this PR, but it is critical to the understanding of this code.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is an atomic compare-and-incref function, similar to how _Py_atomic_compare_exchange is an atomic compare-and-swap function. If the comparison succeeds, the value is incref'd and 1 is returned. 0 is returned on failure.

"Compare" is a more appropriate term than "consistent" here.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think "Compare" is misleading. The name should say what the function does, not how it does it.

DEOPT_IF(true);
}
#else
Py_INCREF(attr_o);
null = PyStackRef_NULL;
attr = PyStackRef_FromPyObjectSteal(attr_o);
#endif
STAT_INC(LOAD_ATTR, hit);
null = PyStackRef_NULL;
DECREF_INPUTS();
}

Expand Down
30 changes: 24 additions & 6 deletions Python/executor_cases.c.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 9 additions & 3 deletions Python/generated_cases.c.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.