You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Expose RecordBatch batch_size through XarrayContext.from_dataset聽#250
馃 read_xarray_table(..., batch_size=...) exposes the source RecordBatch size, but XarrayContext.from_dataset() and _from_dataset() currently do not pass that option through.
This matters for dense-grid consumers that use the convenient context API. A quarter-degree (lat=720, lon=1440) time plane has 1,036,800 rows, so the current 65,536-row default emits about 16 source batches per time partition. A larger batch may reduce Python/Arrow crossings and downstream batch-coalescing overhead; smaller batches may be preferable under memory pressure.
It should be threaded through heterogeneous dimension groups as well. The default should remain unchanged; this issue is about making the existing reader control available from the primary API and documenting the latency/memory tradeoff.
Activity