Object allocation overallocates by 8 bytes for instances of tuple subclasses  #100659

Activity

1. mdickinson

   added

   performancePerformance or resource usagePerformance or resource usage

   on Jan 1, 2023
2. mdickinson commented on Jan 1, 2023

   

   mdickinson

   on Jan 1, 2023

   Last edited by mdickinson

   MemberAuthor

   More actions

   Here's an ad-hoc reproducer (not exactly a proof of the overallocation, but at least strong evidence), on main. Let's start with a 3-element namedtuple called Point:

   Python 3.12.0a3+ (heads/main:d52d4942cf, Jan  1 2023, 14:45:48) [Clang 14.0.0 (clang-1400.0.29.202)] on darwin
   Type "help", "copyright", "credits" or "license" for more information.
   >>> from collections import namedtuple, Counter
   >>> Point = namedtuple("Point", "x y z")

   sys.getsizeof reports 64 bytes per Point instance, which sounds about right to me: 8 fields of 8 bytes each (2 for gc-tracking, 1 for the type pointer, 1 for the refcount, 1 for the tuple size, 3 for the three item pointers, no __dict__ (managed or otherwise), no __weakref__):

   >>> import sys
   >>> sys.getsizeof(Point(1, 2, 3))
   64
   >>> Point.__basicsize__, Point.__itemsize__
   (24, 8)

   But if we create a large number of Point instances (with shared-integer items, so that we're not also allocating memory for the items), their ids never differ by less than 80:

   >>> points = [Point(1, 2, 3) for _ in range(10**6)]
   >>> ids = sorted(map(id, points))
   >>> Counter(id1 - id0 for id1, id0 in zip(ids[1:], ids))
   Counter({80: 995084, 144: 4823, 16528: 77, 480: 2, 160: 1, 184848: 1, 880: 1, 960: 1, 1360: 1, 800: 1, 2720: 1, 720: 1, 904640: 1, 207072: 1, 6640: 1, 180368: 1, 82064: 1})
3. ionite34 commented on Jan 2, 2023

   

   ionite34

   on Jan 2, 2023

   Last edited by ionite34

   Contributor

   More actions

   Similar results here for main and 3.11.1, macos M1

   namedtuple
   sys.getsizeof: 64
   [(80, 995094), (144, 4900), (160, 1), (3200, 1), (82064, 1), (136896, 1), (7585936, 1)]

   Using eval("(1, 2, 3)") to create a bunch of new tuples shows the expected 64 byte allocation

   tuple
   sys.getsizeof: 64
   [(64, 996067), (128, 3918), (192, 2), (576, 2), (768, 1), (832, 1), (1024, 1), (1536, 1), (6144, 1), (7808, 1), (16832, 1), (49280, 1), (82048, 1), (7585920, 1)]

   Subclasses of int and dict do not appear to be affected

   class UserInt(int):
       __slots__ = ()
      
   UserInt(10**50)

   user-int
   sys.getsizeof: 64
   [(64, 996055), (128, 3924), (192, 3), (256, 1), (320, 1), (384, 1), (448, 2), (896, 1), (1088, 1), (1344, 1), (1664, 1), (2240, 1), (2880, 1), (7360, 1), (16512, 1), (51264, 1), (66304, 1), (1127040, 1), (7700608, 1)]

   class UserDict(dict):
       __slots__ = ()
       
   UserDict(x=10)

   user-dict
   basicsize: 48
   basicsize (+ gc): 64
   [(64, 996055), (128, 8), (192, 1), (256, 1), (320, 1), (448, 2), (640, 1), (896, 1), (1664, 1), (1728, 1), (2240, 1), (3328, 1), (7360, 1), (32896, 3888), (49280, 32), (51264, 1), (66304, 1), (1127040, 1), (7733376, 1)]
4. corona10 commented on Jan 10, 2023

   

   corona10

   on Jan 10, 2023

   Member

   More actions

   #81381

   This issue looks relevant, we might need to investigate which case could be reduced.
5. mdickinson commented on Jan 10, 2023

   

   mdickinson

   on Jan 10, 2023

   MemberAuthor

   More actions

   @corona10 Thanks; that looks like the exact same issue. I'll close this as a duplicate.

   👍React with 👍1Reacted by Donghee Na
6. mdickinson

   closed this as not plannedon Jan 10, 2023