Keep module cache entries present during import reordering - #8948

Summary

Keep exact dict entries in sys.modules present while importlib updates shutdown order. The old pop() / assignment pair exposes a temporary absence to other importing threads. This is independently based on current Python 3.14-targeting main; it does not switch the target or the cached importer callback.

• Add a private _imp._dict_move_to_end operation for exact dictionaries, with lookup callbacks outside the storage guard and relocation under one write guard
• Preserve the current value, stored key and hash; invalidate layout caches and retain iterator/compaction bookkeeping
• Revalidate both matched candidates and previously probed prefixes after reentrant equality callbacks, including nested deletion/compaction and insertion into a visited dummy bucket
• Use the helper at all four current importlib reorder sites; retain the original pop / assignment behavior for custom mappings, dict subclasses and CPython
• Add deterministic frozen/source bootstrap regressions and native cache/iterator/compaction tests

Semantic scope

This is a true move rather than a universally equivalent pop / assignment pair. It hashes once, preserves the stored key object, pins one dictionary and lookup key, and makes an already-last entry a no-op. Exact-dict callback side effects that depended on a second hash, key replacement, or rebinding sys.modules between the two Python operations intentionally differ. The custom-mapping fallback preserves those original Python operations and is not atomic.

Deliberate module removal/replacement and whole-map rebinding remain possible; this eliminates the artificial gap caused by shutdown-order maintenance. Identity hits in the first bucket allocate no probe-prefix storage. Pathological reentrant colliding keys can require O(k²) prefix validation and O(k) retained references.

This is a prerequisite for the separately prepared Python 3.15 bootstrap and canonical importer callback migration. It does not claim to fix the pre-existing Python 3.14 hierarchical-import deadlock.

Validation

The standalone release was built from this branch, and its frozen/source bootstrap, private helper, target 3.14 and source-Lib provenance were checked.

• Native regression snippet and the independently identified false-equality/prefix regression: pass
• test_importlib test_import test_site test_support test_pkgutil test_dict test_ordered_dict: 2,096 tests run, 131 skipped, no failures
• Full bounded Python snippets: 469 passed, 13 failures; 10 AF_UNIX sandbox failures, two chown errno differences, and one bounded CPython GC/import stress timeout. The new native regression passes; this is not a green full-snippet claim
• Fresh workspace Rust tests: 1,348 passed, 18 ignored, including all three new dict_inner::tests::move_to_end_* tests
• Fresh separate C-API tests: 115 passed, 4 ignored
• Workspace/C-API Clippy: both passed without warnings
• Final regression snippet: native RustPython, CPython 3.14.7 and CPython 3.15.0rc2 all pass. Legacy cases remain enabled whenever the bootstrap exposes them; modern load/exec cases always run

Rust/Lib implementation and aggregate checks are at 2e30cbf. The final head only adds the independently rerun snippet availability guard; production source is byte-identical.

A shared-target stale VM test binary was detected because it lacked the new tests. That first aggregate run is discarded as validation. The reported fresh runs rebuilt the affected sources, verified the new test names, and checked the resulting binary provenance. No source content was changed to repair that cache issue.

AI assistance

Implemented, reviewed and tested with OpenAI Codex under the requesting RustPython maintainer's direction. No human code review or maintainer privilege is implied for this account. The runtime does not expose an exact model version; commits use Assisted-by: Codex:model-version-unavailable rather than inventing one.