Skip to content

Record streaming attempt and front-end metrics when PartialResultSet.last=True is observed #18656

Open

Description

@olavloite

Description

In google-cloud-spanner, streaming queries that receive PartialResultSet.last = True hand off the underlying response iterator to _GLOBAL_STREAM_DRAINER to drain trailing metadata to EOF asynchronously on a background worker thread (in snapshot.py).

However, _StreamingResponseWrapper in metrics_interceptor.py currently defers recording attempt completion (attempt_latencies) and front-end timing metrics (gfe_latencies, afe_latencies) until StopIteration is raised:
https://github.com/googleapis/google-cloud-python/blob/main/packages/google-cloud-spanner/google/cloud/spanner_v1/metrics/metrics_interceptor.py

This causes two issues:

  1. Metric Timing Inaccuracy: Attempt latency should measure when the operation actually completes from an outside observer's perspective (i.e. when PartialResultSet.last = True is received by the caller), rather than including time spent waiting in the background drainer queue or consuming trailing stream metadata.
  2. Race Condition with Metrics Readers: Callers that complete a query and immediately collect or inspect metrics observe missing attempt_latencies, gfe_latencies, and afe_latencies because the background drainer thread has not yet reached EOF.

Proposed Solution

The client should measure attempt completion and record front-end metrics as soon as PartialResultSet.last == True is observed on the stream, rather than waiting for StopIteration during background stream draining.

Activity

  1. self-assigned this
    on Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

api: spannerIssues related to the Spanner API.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions