RuntimeErrors upon interpreter shutdown due to garbage collection #762

Describe the bug
There are various runtime errors when a snapshot is being garbage collected during shutdown.

To Reproduce

import sys
import pynbody

def main():
    snapshot_path = sys.argv[1]
    snapshot = pynbody.load(snapshot_path)

if __name__ == "__main__":
    main()

I get this when opening certain Ramses snapshots, for example in the pynbody testdata, opening ramses_new_format_partial_output_00001:
Expected behaviour or result
The expected behaviour is no warnings or errors when simply loading a snapshot and exiting. Instead I get

Exception ignored in: <function _get_auto_closing_shared_memory_buffer.<locals>.clearup_callback at 0x7f57579696c0>
Traceback (most recent call last):
  File "/home/pavy/micromamba/envs/phd/lib/python3.12/site-packages/pynbody/array/shared.py", line 169, in clearup_callback
    t.start()
  File "/home/pavy/micromamba/envs/phd/lib/python3.12/threading.py", line 992, in start
    _start_new_thread(self._bootstrap, ())
RuntimeError: can't create new thread at interpreter shutdown

which appears multiple times (every time a shared memory array is getting garbage collected when python is shutting down).

Additionally, there are issues with the multiprocessing resource tracker as well:

/home/pavy/micromamba/envs/phd/lib/python3.12/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 16 leaked shared_memory objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

Setup:

• Debian on Windows Subsystem for Linux
• python 3.12.1
• numpy 1.26.3
• pynbody 1.5.2

Additional context
The first issue seems to be a new issue with python 3.12 (see python/cpython#104826). They started emitting errors when threads are spawned during interpreter finalisation. This error can be somewhat avoided by forcing the garbage collector to clean up the arrays before exiting by adding:

import gc
del snapshot
gc.collect()

before main() exits.

From what I read from the pynbody code is that these errors come from using the cleanup_callback approach to close and unlink the shared memory?

The second issue I don't fully understand but it seems related to this python issue (see https://bugs.python.org/issue39959) where multiple processes are trying to close the same shared memory. It seems to have a fix in the upcoming python 3.13 (see python/cpython#82300), so it might not be actionable until then.