Repository navigation
Resource tracker fails to track filenames with colons on Linux #98896
Description
Bug report
multiprocessing.resource_tracker fails to track files with a colon (':') in the filename due to a parsing error, allowing resource leaks. I originally encountered this while working with filenames that include ISO8601 timestamps with colons as time separators.
Minimal example:
$ python -c "from multiprocessing.shared_memory import SharedMemory; shm = SharedMemory(create=True, size=1, name='a:b')"
$ Traceback (most recent call last):
File "/usr/lib/python3.10/multiprocessing/resource_tracker.py", line 199, in main
cmd, name, rtype = line.strip().decode('ascii').split(':')
ValueError: too many values to unpack (expected 3)
$ ls /dev/shm/a\:b
/dev/shm/a:bWithout the colon, the file is correctly tracked and removed:
$ python -c "from multiprocessing.shared_memory import SharedMemory; shm = SharedMemory(create=True, size=1, name='ab')"
$ /usr/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
$ ls /dev/shm/ab
ls: cannot access '/dev/shm/ab': No such file or directoryYour environment
- CPython versions tested on: 3.10.7, 3.11.0, 3.12.0a1
- Operating system and architecture: Fedora 35 x86_64 (3.10.7 from Fedora, others with Docker images)
Linked PRs
- gh-98896: Fix parsing issue in resource_tracker to allow shared memory names containing colons #138473
- [3.14] gh-98896: resource_tracker: use json&base64 to allow arbitrary shared memory names (GH-138473) #141922
- [3.13] gh-98896: resource_tracker: use json&base64 to allow arbitrary shared memory names (GH-138473) #142014
Activity
saito828koki commentedon Nov 1, 2022
@maaleske
Thanks for your report.
In my opinion, making colons unusable in resource names is a reasonable fix for this issue. (We should also check if other parts of
multiprocessingusingresource_trackerlikemultiprocessing.synchronize.SemLockwill cause the same problem. )Are there better ways to fix this issue?
maaleske commentedon Nov 1, 2022
@saito828koki
That is poor solution, since the issue is still present if the resource is not created by the Python process, but just opened through
SharedMemory(as was the case in my original encounter with it). Disallowing colons would makeSharedMemoryunusable in situations where Python is not in charge of the filenames, which I imagine to be a common use of shared memory resources.I also investigated a bit more and colons are not the only problem; if the filenames contain newlines (again, valid on Linux) and colons, you can trigger other errors from the same code path, and I think there may even be a way to induce the resource tracker to unlink the wrong files by injecting an extra
\ncmd:name:rtype\nin to the file through the filename, though I didn't yet find a working example.Given that Python might not be the one responsible for the filenames, the real fix is to not parse them and store them in a way that does not require it. If you want to be rigorous, I think you would have to store each filename in a separate file, and manage the related cmd/rtype in some way.
However, there is also an issue related to incorrect behaviour of the resource tracker (#82300), with one fix proposing to remove it altogether for shared memory resources, which would make this a non-issue for SHM, but still probably relevant for semaphores.
pythongh-98896: Fix parsing of registered or unregistered resources
pythongh-98896: Fix parsing of registered or unregistered resources
gh-98896: resource_tracker: use json&base64 to allow arbitrary shared…
[3.14] pythongh-98896: resource_tracker: use json&base64 to allow arb…
[3.14] gh-98896: resource_tracker: use json&base64 to allow arbitrary…
[3.14] pythongh-98896: resource_tracker: use json&base64 to allow arb…
[3.13] gh-98896: resource_tracker: use json&base64 to allow arbitrary…
pythongh-98896: resource_tracker: use json&base64 to allow arbitrary …
flucasgabriel58-eng commentedon Jan 13, 2026