Bug: S3Object raises InvalidRange when reading at the end of an object or reading an empty object #8510

Expected Behaviour

S3Object should behave like a regular seekable file at the end of the data. Reading at or past the end returns b"", an empty object reads as b"" and iterates as nothing, and closing never needs another request.

Current Behaviour

_S3SeekableIO.raw_stream opens every stream with Range: bytes=<position>-. S3 rejects a range that starts at or past the end of the object with 416 InvalidRange, and that includes any range on an empty object. The error isn't handled, so:

• read() after seek(0, io.SEEK_END), or after seeking past the end, raises ClientError (InvalidRange).
• read() or iterating an empty object raises InvalidRange. Zero-byte objects are common: the S3 console's "Create folder" makes one (photos/), and it fires the same S3 event notifications as any upload.
• close() goes through the raw_stream property, so closing an object that was never read opens a new GetObject just to close it. If the position is at the end, close() itself raises. That means a with S3Object(...) block that ends at EOF fails on exit.

Code snippet

import io

import boto3
from moto import mock_aws

from aws_lambda_powertools.utilities.streaming import S3Object


@mock_aws
def main():
    s3 = boto3.client("s3", region_name="us-east-1")
    s3.create_bucket(Bucket="my-bucket")
    s3.put_object(Bucket="my-bucket", Key="data.txt", Body=b"hello")
    s3.put_object(Bucket="my-bucket", Key="photos/", Body=b"")

    def attempt(name, fn):
        try:
            print(f"{name}: {fn()!r}")
        except Exception as exc:
            print(f"{name}: {type(exc).__name__}: {exc}")

    def read_at_end():
        obj = S3Object(bucket="my-bucket", key="data.txt", boto3_client=s3)
        obj.seek(0, io.SEEK_END)
        return obj.read()

    def read_empty_object():
        return S3Object(bucket="my-bucket", key="photos/", boto3_client=s3).read()

    def with_block_ending_at_eof():
        with S3Object(bucket="my-bucket", key="data.txt", boto3_client=s3) as obj:
            obj.seek(0, io.SEEK_END)
        return "ok"

    attempt("read at end", read_at_end)
    attempt("read empty object", read_empty_object)
    attempt("with block ending at EOF", with_block_ending_at_eof)


main()

Output:

read at end: ClientError: An error occurred (InvalidRange) when calling the GetObject operation: The requested range is not satisfiable
read empty object: ClientError: An error occurred (InvalidRange) when calling the GetObject operation: The requested range is not satisfiable
with block ending at EOF: ClientError: An error occurred (InvalidRange) when calling the GetObject operation: The requested range is not satisfiable

io.BytesIO(b"hello") and io.BytesIO(b"") return b'', b'' and 'ok' for the same three operations.

Possible Solution

• In _S3SeekableIO.raw_stream, catch a ClientError whose code is InvalidRange and use an empty PowertoolsStreamingBody. This keeps the single request per read, with no extra HeadObject to check the size first. Any other error is still raised.
• In _S3SeekableIO.close(), only close _raw_stream if a stream is open.

I have this change with tests ready and will open a PR for it.

Steps to Reproduce

1. Upload an object to S3, or create an empty one, e.g. a folder via the S3 console.
2. Open it with S3Object and either seek(0, io.SEEK_END) then read(), or read() the empty object.
3. ClientError (InvalidRange) is raised instead of b"".

Powertools for AWS Lambda (Python) version

latest (3.35.0)

AWS Lambda function runtime

3.10

Packaging format used

PyPi

Debugging logs

N/A: reproduced locally with the snippet above.