Skip to content

Valgrind lists memory leaks: dlopen() called without dlclose() #121096

Description

@vstinner

Example on Python 3.12 when loading _crypt extension module:

$ PYTHONMALLOC=malloc valgrind --leak-check=full --show-leak-kinds=all --num-callers=50 ./python -c 'import _crypt'
==2444751== Memcheck, a memory error detector
==2444751== Copyright (C) 2002-2024, and GNU GPL'd, by Julian Seward et al.
==2444751== Using Valgrind-3.23.0 and LibVEX; rerun with -h for copyright info
==2444751== Command: ./python -c import\ _crypt
==2444751== 
==2444751== 
==2444751== HEAP SUMMARY:
==2444751==     in use at exit: 3,253 bytes in 8 blocks
==2444751==   total heap usage: 44,037 allocs, 44,029 frees, 6,464,804 bytes allocated
==2444751== 
==2444751== 21 bytes in 1 blocks are still reachable in loss record 1 of 7
==2444751==    at 0x484282F: malloc (vg_replace_malloc.c:446)
==2444751==    by 0x402587F: malloc (rtld-malloc.h:56)
==2444751==    by 0x402587F: strdup (strdup.c:42)
==2444751==    by 0x4014B58: _dl_load_cache_lookup (dl-cache.c:515)
==2444751==    by 0x4008F1F: _dl_map_object (dl-load.c:2116)
==2444751==    by 0x400287C: openaux (dl-deps.c:64)
==2444751==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==2444751==    by 0x4002CDF: _dl_map_object_deps (dl-deps.c:232)
==2444751==    by 0x400CA34: dl_open_worker_begin (dl-open.c:638)
==2444751==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==2444751==    by 0x400C10F: dl_open_worker (dl-open.c:803)
==2444751==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==2444751==    by 0x400C563: _dl_open (dl-open.c:905)
==2444751==    by 0x49E9E23: dlopen_doit (dlopen.c:56)
==2444751==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==2444751==    by 0x4001678: _dl_catch_error (dl-catch.c:256)
==2444751==    by 0x49E9912: _dlerror_run (dlerror.c:138)
==2444751==    by 0x49E9EDE: dlopen_implementation (dlopen.c:71)
==2444751==    by 0x49E9EDE: dlopen@@GLIBC_2.34 (dlopen.c:81)
==2444751==    (...)

Valgrind reports leaks on memory allocated by dlopen() in Python/dynload_shlib.c, because Python doesn't call dlclose().

I don't think that it's big deal to not call dlclose(), since there is little memory and it's not common to unload a dynamic library at runtime. Also, I don't see where/how dlclose() can be called.

Instead, I propose to suppress these warnings in the Valgrind suppressions file, to help spotting more important leaks.

Linked PRs

Activity

  1. added a commit that references this issue on Jun 27, 2024
  2. picnixz commented on Jun 28, 2024

    @picnixz
    Member

    Also, I don't see where/how dlclose() can be called

    AFAICT, dlopen() is used for the init function of modules. I don't know if it's possible, but I would assume that dlclose should be called once the module is unloaded or if there is not reference anymore. In addition, dlclose() does not decessarily unload the shared library, it only decrements the number of references (like Py_DECREF).

    Is there a callback mechanism for (extension) modules that could be called in order to close the handle when needed? As for "how", you need to carry over the handle in the module objects probably...

  3. vstinner commented on Jun 28, 2024

    @vstinner
    MemberAuthor

    Is there a callback mechanism for (extension) modules that could be called in order to close the handle when needed?

    I don't think that it does exist, and it would require to rewrite a lot of things. For now, I prefer to simply ignore ("suppress") the "leak" in Valgrind.

  4. added a commit that references this issue on Jun 28, 2024
  5. added 2 commits that reference this issue on Jun 28, 2024
  6. added 2 commits that reference this issue on Jun 28, 2024
  7. added a commit that references this issue on Jun 30, 2024
  8. added a commit that references this issue on Jul 11, 2024
  9. added a commit that references this issue on Jul 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions