Repository navigation
Increased memory usage with mimalloc #135153
Description
Activity
Some additional information about this. Using the
MIMALLOC_PURGE_DELAYsetting makes a big difference on the RSS when mimalloc is used. This controls how quickly mimalloc will release unused memory back to the OS. On Linux, by default, it will useMADV_DONTNEEDon the memory-mapped pages it no longer needs. By default, mimalloc will wait in case the program needs more memory. This is a kind of classic memory use vs speed tradeoff. When using the malloc built-in to glibc combined with pymalloc, unused memory is generally returned to the OS quickly.Comparisons of RSS with different options running the bench_tomllib_mem.py.txt script:
Config RSS [MB] Command-line default, pymalloc 70 uv run --python=3.14 python default, mimalloc 94 PYTHONMALLOC=mimalloc uv run --python=3.14 python defauilt, mimalloc, purge 72 PYTHONMALLOC=mimalloc MIMALLOC_PURGE_DELAY=0 uv run --python=3.14 python free-threaded 106 uv run --python=3.14t python free-threaded, purge 88 MIMALLOC_PURGE_DELAY=0 uv run --python=3.14t python May be largely unavoidable? I don't know mimalloc internals, but my understanding is that mimalloc's "ssgments" are substantially larger than pymalloc's "arenas", and that's the granularity at which they request/free RAM from/to the OS.
pymslloc used to return an arena to the OS as soon as it became empty. Then it changed to save at most one empty arena on an internal free list, to prevent pathological cases of thrashing. in loops that systematically used and freed one arena per iteration. That increased "committed memory", but only by the size of one arena (currently 1 MiB). With larger segments, and/or less eager freeing, it would get worse.
As @nascheme said above, this is probably mostly the outcome of mimalloc not returning memory to the OS eagerly. This is the behavior of most modern allocators, because returning memory to the OS (sometimes only to ask for more memory microseconds later) is expensive.
In general, "memory usage" has become a tricky thing to diagnose and understand, due to the multiple layers of caching that can exist (for good reasons).
Reacted by Tim PetersPerhaps we should just close this? It seems to me there is nothing really to fix here. mimalloc being less eager to return memory to the OS is most of the issue. There are some additional factors involved with the free-threaded build. Biased reference counting and QSBR (essentially a delayed memory free mechanism) makes it so that with the free-threaded build it's possible there are even more delaying in freeing memory.
There as an issue with QSBR that was fixed. You could set
MIMALLOC_PURGE_DELAY=0but I don't think that should be the default.Should we do something about realloc? mimalloc currently only downsizes if the new size is <= 50% of the old. If we don't change that, perhaps the logic on objects like dicts and lists needs to be adjusted. E.g. no use resizing a list down to 70% of it's original size if mimalloc gives you the same block back.
Reacted by Tim Peters and SurajAgreed that
MIMALLOC_PURGE_DELAY=0would be a bad idea - and at least for the same reasons that pymalloc returning empty arenas ASAP turned out to be a bad idea 😉.I don't think complicating list/dict/set resizing would help either. It's already the case that lists don't shrink unless the new size is strictly less than half of list's idea of the current maximum capacity. Don't recall dict/set details, but they always request room for a power of 2 number of slots, So it requires "big shrinkage" for them too to try a realloc.
I'd be happier if mimalloc's delay triggered on a measure of how much RAM could be freed rather than on elapsed time. But not happier enough to suggest changing that for real 😉,
In the absence of something "actionable", ya, closing this would be best. Thanks for digging into it!
This was originally going to be titled "with free-threaded build". However, based on some investigation, the increased memory usage is seemingly largely due to the use of mimalloc rather than free-threading specific behaviors. This issue will focus on mimalloc causing increased memory usage.
Here are some techniques I used to investigate this, which might be useful to others. Note that these instructions are specific for running on Linux.
Include/internal/mimalloc/mimalloc/types.h, uncomment theMI_TRACK_VALGRINDdefine. This allows the "massif" tool of Valgrind to trace the mimalloc allocations and deallocations. Also note that you must compile with GCC, not Clang.Objects/mimalloc/alloc.c, comment out the line containing the "reallocation still fits and not more than 50% waste". This makes mimalloc always resize down an object, even if the decrease is less than 50% of the original size. This is not a major cause of extra memory usage but reduces the gap between pymalloc and mimalloc. pymalloc always resizes downwards.valgrind --tool=massif ./python <script>. This will create amassif.out.NNNfile as a result. That is a text file report of the allocations and where they happened.As an alternative to "massif", you can also use memray. To make it run with the free-threaded build, you need to uncomment one line (the definition of Py_LIMITED_API in
src/memray/_memray/inject.cpp). You do have to rely on--trace-python-allocatorsas well as--nativebecause mimalloc allocates a 1Gb region right at process startup, but you really want to see the results of tracing python allocations anyway.Here is a run of pyperformance done by Thomas Wouters, showing memory usage of benchmarks compared to a baseline.
pyperformance results
Based on the pyperformance memory usage statistics, the
bm_tomli_loadsbenchmark is a benchmark that uses quite a lot of extra memory under the free-threaded build vs the default build. To make it easier to run, I wrote a simpler version of a benchmark that uses the builtintomllibmodule instead. It also shows a similar amount of extra memory usage.bench_tomllib_mem.py.txt
Some additional scripts, to dump out memory usage stats:
memory_stats.py.txt
count_used_pages.py.txt
Statistics on RSS (resident set size) when running this script under various Python builds:
Note that to run the default build with mimalloc, you must set
PYTHONMALLOC=mimallocin the environment. Otherwise, pymalloc is used, even if mimalloc is available in the build. The FT build always uses mimalloc.Using the massif reports, we can examine where the memory is being allocated from. Below is a summary for three different builds.
Based on these results, my suspicion is that mimalloc can waste quite a lot of memory compared to pymalloc when many small allocations are made. I suspect so because it seems most of the extra memory usage is due to the
unicode_subscriptcalls. For thepymallocbuild, that memory must be accounted under the "PyUnicode_New, others" category (massif doesn't show theunicode_subscriptcall explicitly). More work is needed to confirm this theory. Another theory is that mimalloc is not releasing the freed memory back to the OS as aggressively as pymalloc does.