feat(gax): add RewindableStreamBuffer in prep for chunk upload recovery - #14423
Merged
Merged
Conversation
whowes
force-pushed
the
whowes/resumable-upload-buffer-window
branch
from
September 17, 2026 22:08
ba8840b to
6d89325
Compare
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
whowes
added this pull request to stack #14429
September 17, 2026 22:16
whowes
force-pushed
the
whowes/resumable-upload-buffer-window
branch
from
September 18, 2026 02:23
6d89325 to
da1275c
Compare
whowes
force-pushed
the
whowes/resumable-upload-buffer-window
branch
2 times, most recently
from
September 18, 2026 03:21
589ef8e to
f130ccc
Compare
whowes
force-pushed
the
whowes/resumable-upload-buffer-window
branch
from
September 18, 2026 15:05
f130ccc to
0e94df8
Compare
whowes
force-pushed
the
whowes/resumable-upload-buffer-window
branch
from
September 19, 2026 01:36
0e94df8 to
00f7932
Compare
whowes
force-pushed
the
whowes/resumable-upload-buffer-window
branch
from
September 19, 2026 21:12
00f7932 to
c6c28b1
Compare
whowes
force-pushed
the
whowes/resumable-upload-buffer-window
branch
from
September 19, 2026 22:57
c6c28b1 to
4a4b890
Compare
whowes
force-pushed
the
whowes/resumable-upload-buffer-window
branch
from
September 19, 2026 23:17
4a4b890 to
14f6204
Compare
whowes
force-pushed
the
whowes/resumable-upload-buffer-window
branch
from
September 20, 2026 00:19
14f6204 to
0a389c4
Compare
whowes
force-pushed
the
whowes/resumable-upload-buffer-window
branch
from
September 20, 2026 05:23
0a389c4 to
7d192ff
Compare
whowes
changed the title
whowes
force-pushed
the
whowes/resumable-upload-buffer-window
branch
from
September 20, 2026 06:51
7d192ff to
33b9862
Compare
whowes
removed this pull request from stack #14429
September 20, 2026 07:20
whowes
force-pushed
the
whowes/resumable-upload-buffer-window
branch
from
September 22, 2026 08:52
2f7c89c to
715822d
Compare
whowes
force-pushed
the
whowes/resumable-upload-buffer-window
branch
2 times, most recently
from
September 22, 2026 16:07
0925229 to
11ab9b0
Compare
whowes
removed this pull request from stack #14454
September 22, 2026 16:21
whowes
force-pushed
the
whowes/resumable-upload-buffer-window
branch
2 times, most recently
from
September 22, 2026 16:45
3ae0f6e to
97fb372
Compare
whowes
added this pull request to stack #14476
September 22, 2026 17:47
whowes
force-pushed
the
whowes/resumable-upload-buffer-window
branch
from
September 22, 2026 18:52
97fb372 to
3258c60
Compare
whowes
force-pushed
the
whowes/resumable-upload-buffer-window
branch
from
September 22, 2026 19:56
3258c60 to
074e557
Compare
blakeli0
reviewed
Sep 23, 2026
| public abstract byte[] getPayload(); | ||
|
|
||
| /** The number of bytes within {@link #getPayload()} to upload. */ | ||
| public abstract int getPayloadLength(); |
Contributor
There was a problem hiding this comment.
Do we have to expose getPayloadLength in the request? I think we can do the slicing in RewindableStreamBuffer and getPayload should always return the byte[] that we can upload directly.
blakeli0
reviewed
Sep 23, 2026
| * offset or beyond the current buffer window | ||
| * @throws IOException if reading from the stream fails | ||
| */ | ||
| void realignTo(long committedOffset) throws IOException { |
Contributor
There was a problem hiding this comment.
Is this method used in this PR?
Contributor
Author
There was a problem hiding this comment.
No, it's used in recovery (not implemented until #14424). This PR just introduces the buffer and wires it up.
whowes
changed the title
whowes
force-pushed
the
whowes/resumable-upload-buffer-window
branch
from
September 24, 2026 00:45
074e557 to
c69d6b1
Compare
whowes
force-pushed
the
whowes/resumable-upload-buffer-window
branch
from
September 24, 2026 01:06
c69d6b1 to
7431c49
Compare
whowes
force-pushed
the
whowes/resumable-upload-buffer-window
branch
from
September 24, 2026 05:52
7431c49 to
1ba5e07
Compare
whowes
force-pushed
the
whowes/resumable-upload-buffer-window
branch
2 times, most recently
from
September 24, 2026 17:40
95a562e to
4c0fd44
Compare
blakeli0
reviewed
Sep 24, 2026
| * @throws IOException if reading from the stream fails | ||
| */ | ||
| void fill(long targetOffset) throws IOException { | ||
| this.bufferBaseOffset = targetOffset; |
Contributor
There was a problem hiding this comment.
Looks like this offset is always calculated from the info within the buffer and passed into here in follow up PR. I think we can get rid of this parameter and let RewindableStreamBuffer manage the targetOffset.
blakeli0
approved these changes
Sep 25, 2026
Introduce RewindableStreamBuffer managing a single-chunk buffer over an InputStream, supporting forward compaction and topping up upon recovery realignment without mark()/reset(). Enforces boundaries by throwing IllegalStateException when a server offset is below the base offset or beyond the current buffer window.
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduces
RewindableStreamBufferto manage a single-chunk buffer for the user-providedInputStream.Recovery is not implemented in this PR - the buffer is wired up in the chunk coordinator here and the query-command-based recovery process will be implemented in the next.