Skip to content
Open
Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
193cc2e
feat: add direct async iteration on Dataset and KeyValueStore
Pijukatel Oct 1, 2026
2deace2
feat: make KeyValueStore record iteration overridable by storage clients
Pijukatel Oct 1, 2026
3c03eae
perf: read key-value store entries in bulk in the SQL and Redis clients
Pijukatel Oct 1, 2026
125bba2
test: drop the test for records deleted during key-value store iteration
Pijukatel Oct 2, 2026
668c170
refactor: decode None values in one place in the Redis key-value stor…
Pijukatel Oct 2, 2026
0e255a5
docs: describe the bulk iterate_entries overrides in docstrings
Pijukatel Oct 2, 2026
6c0cc3f
refactor: reject a decoding Redis client instead of suppressing the t…
Pijukatel Oct 2, 2026
95a5154
refactor: narrow Redis replies without a cast
Pijukatel Oct 2, 2026
a07f81a
docs: scope the raw-bytes requirement to the Redis key-value store cl…
Pijukatel Oct 2, 2026
1f32875
fix: bound memory of the SQL key-value store iteration with keyset pages
Pijukatel Oct 2, 2026
f809fe7
fix: retry the batched Redis value read and test the batching helper
Pijukatel Oct 2, 2026
c0c334e
test: give the batching helper test module a unique basename
Pijukatel Oct 2, 2026
f7db136
fix: skip Redis records deleted mid-iteration without a missing-value…
vdusek Oct 6, 2026
5918899
docs: correct memory and round-trip claims in KVS iteration docstrings
vdusek Oct 6, 2026
eb6a2f3
test: cover SQL iterate_entries limit ending inside a later page
vdusek Oct 6, 2026
a967540
test: rename batching test helpers and add missing test docstrings
vdusek Oct 6, 2026
db0339a
feat: make KeyValueStore async iteration yield keys like a dict
Pijukatel Oct 6, 2026
0b9af81
refactor: return the key iterator from KeyValueStore.__aiter__ like D…
vdusek Oct 6, 2026
4905199
docs: fix RedisStorageClient and KVS iteration docstrings
vdusek Oct 6, 2026
1c66c3b
test: record Redis and SQL batch calls with wrapping mocks
vdusek Oct 6, 2026
2c3dd78
test: cover records deleted mid-iteration in memory and SQL KVS clients
vdusek Oct 6, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
test: drop the test for records deleted during key-value store iteration
Whether a record deleted mid-iteration is yielded depends on how the
storage client reads values, which is not part of the contract.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wg6jZgyQp7XkvVvxuueJyV
  • Loading branch information
Pijukatel and claude committed Oct 2, 2026
commit 125bba2a7fb1f441c338f1485c11c946a60b21f6
27 changes: 0 additions & 27 deletions tests/unit/storages/test_key_value_store.py
Original file line number Diff line number Diff line change
Expand Up @@ -317,33 +317,6 @@ async def test_iterate_entries_empty_kvs(kvs: KeyValueStore) -> None:
assert collected_entries == []


async def test_iterate_entries_skips_records_deleted_during_iteration(
kvs: KeyValueStore, storage_client: StorageClient
) -> None:
"""Test that a record deleted after its key was listed but before its value was read is skipped.

This only holds for storage clients using the default key-by-key `iterate_entries` implementation. Clients that
read the values in batches or in a single query may have already read the record when it gets deleted.
"""
if not isinstance(storage_client, (MemoryStorageClient, FileSystemStorageClient)):
pytest.skip('Storage client reads values in batches, so a deleted record may already be read.')

for i in range(5):
await kvs.set_value(f'key{i}', f'value{i}')

all_keys = [metadata.key for metadata in await kvs.list_keys()]
deleted_key = all_keys[-1]

collected_entries = []
async for key, value in kvs.iterate_entries():
if key == all_keys[0]:
await kvs.delete_value(deleted_key)
collected_entries.append((key, value))

assert len(collected_entries) == 4
assert deleted_key not in dict(collected_entries)


async def test_iterate_entries_uses_storage_client_implementation() -> None:
"""Test that `iterate_entries` and `iterate_values` go through the storage client's `iterate_entries`.

Expand Down
Loading