Captured exceptions are keeping user objects alive unnecessarily in the stdlib #81001

Activity

1. mariocj89 commented on May 6, 2019

   

   mariocj89mannequin

   on May 6, 2019

   MannequinAuthor

   More actions

   There are multiple places in the standard library where the code captures an exception and reraises it later (outside of the original except).

   This is known to cause a cycle as saving the exception has a traceback that eventually points back to the exception, but it is especially problematic as it keeps alive the objects that the user has as well.

   This can be reproduced with the following example:

   import weakref
   
   class A: pass
   
   ref = None
   
   def x():
       global ref
       cool_var = A()
       ref = weakref.ref(cool_var)
       assert ref()
       try:
           1/0
       except Exception as e:
           ee = e
   
   try:
       x()
   except Exception:
       pass
   
   print(ref())
   assert ref() is None
   

   There are places in the standard library that try to get around this. See acb9fa7 as an example. This change did not include the exception path though, when the err variable is raised, the issue is still present.

   This issue is to fix the instances found in socket.py, codeop.py and dyld.py. By either setting the variable to None to remove the name or if it needs to be reraised by using a finally to delete it. This is basically the same that a try except would do, which automagically adds a finally to delete the local name used in the except X as y.
   There was a discussion with twouters,pablogsal,barry about potentially adding something extra to the exception handling to try to clean this up but it was suggested the best approach is for the user to handle it and maybe a linter to potentially flag it, as there might also be multiple references to the exception being captured. It was also proposed to just change the situations where this happens in the standard library to remove the name if possible.

   Note that eventually those objects will just be collected by the gc, it is just an improvement.

   I'm preparing a PR for those three modules.
2. mariocj89

   added

   3.8 (EOL)end of lifeend of life

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   type-featureA feature request or enhancementA feature request or enhancement

   on May 6, 2019
3. miss-islington commented on Dec 6, 2019

   

   miss-islington

   on Dec 6, 2019

   Contributor

   More actions

   New changeset b64334c by Miss Islington (bot) (Mario Corchero) in branch 'master':
   bpo-36820: Break unnecessary cycle in socket.py, codeop.py and dyld.py (GH-13135)
   b64334c
4. pablogsal

   closed this as completedon Dec 6, 2019
5. miss-islington commented on Dec 6, 2019

   

   miss-islington

   on Dec 6, 2019

   Contributor

   More actions

   New changeset 681285d by Miss Islington (bot) in branch '3.8':
   bpo-36820: Break unnecessary cycle in socket.py, codeop.py and dyld.py (GH-13135)
   681285d
6. miss-islington commented on Dec 6, 2019

   

   miss-islington

   on Dec 6, 2019

   Contributor

   More actions

   New changeset 5ba591f by Miss Islington (bot) in branch '3.7':
   bpo-36820: Break unnecessary cycle in socket.py, codeop.py and dyld.py (GH-13135)
   5ba591f
7. ezio-melotti

   transferred this issue fromon Apr 10, 2022