Skip to content

gh-107089: Improve Shelf.clear method performance #107090

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
corona10 merged 4 commits into from
Jul 29, 2023
Merged

gh-107089: Improve Shelf.clear method performance #107090

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
2b5cc11
gh-107089: Improve Shelf.clear method performance
jtcave Jul 23, 2023
0e3f387
gh-107089: call dbm.clear method
jtcave Jul 24, 2023
0f41369
fix whitespace
jtcave Jul 24, 2023
d90b207
fix whitespace
jtcave Jul 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
gh-107089: call dbm.clear method 
Because gh-107089 is peculiar to implementation details of dbm objects,
it would be less disruptive to implement it in the DbfilenameShelf
class, which is used for calls to shelve.open. Since it is known that
the backing object is specifically one of the dbm objects, its clear
method (see gh-107122) can be used with no fallback code.
  • Loading branch information
@jtcave
jtcave committed Jul 28, 2023
commit 0e3f387ca1f5c25b0b59dbb81064bd1e4b57df55
27 changes: 7 additions & 20 deletions Lib/shelve.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -171,26 +171,6 @@ def sync(self):
if hasattr(self.dict, 'sync'):
self.dict.sync()

def clear(self):
"""Remove all items from the shelf."""
self.cache.clear()
try:
self.dict.clear()
except AttributeError:
# dbm objects don't have a clear method, so we need to do
# the clearing here.
keys = self.dict.keys()
if not isinstance(keys, list):
# The keys method on dbm objects returns a list.
# Typically, the keys method on a mapping returns a
# lazy iterator, so we need to watch out for that in
# case someone passes in a backing object that behaves
# that way.
keys = list(keys)
for k in keys:
del self.dict[k]


class BsdDbShelf(Shelf):
Comment thread
jtcave marked this conversation as resolved.
"""Shelf implementation using the "BSD" db interface.

Expand Down Expand Up @@ -244,6 +224,13 @@ class DbfilenameShelf(Shelf):
def __init__(self, filename, flag='c', protocol=None, writeback=False):
import dbm
Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)

def clear(self):
"""Remove all items from the shelf."""
# Call through to the clear method on dbm-backed shelves.
# see https://github.com/python/cpython/issues/107089
self.cache.clear()
self.dict.clear()


def open(filename, flag='c', protocol=None, writeback=False):
Expand Down
4 changes: 2 additions & 2 deletions Misc/NEWS.d/next/Library/2023-07-22-21-57-34.gh-issue-107089.Dnget2.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
The :meth:`Shelf.clear` method in the :mod:`shelve` module is much faster.
Patch by James Cave.
Shelves opened with :func:`shelve.open` have a much faster :meth:`clear`
method. Patch by James Cave.