Skip to content

feat(bigquery-jdbc): infer undeclared fields in PreparedStatement - #14400

Merged
Neenu1995 merged 21 commits into
mainfrom
add-undeclared-query-parameters
Sep 29, 2026
Merged

Neenu1995 merged 21 commits into
mainfrom
add-undeclared-query-parameters

Conversation

@Neenu1995

@Neenu1995 Neenu1995 commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

Fixes b/557150710

  • Infers types for undeclared positional parameters by issuing a POSITIONAL-mode
    dry run when a PreparedStatement is prepared.
  • Derives the parameter count from the dry run's undeclared parameter list instead
    of counting ? characters, so placeholders inside string literals
    (SELECT 'Hello, ?World!', ?) no longer inflate the count.
  • Adds a parameter provenance state machine (UNBOUND / TYPED / BOUND) so an
    inferred type never overwrites a value the caller set, and so batch copies and
    clearParameters() preserve that distinction.
  • Sends typed nulls: setNull and setObject(i, null) resolve the type as
    caller-declared, then dry-run inferred, then STRING.
  • Populates getParameterMetaData() before any value is bound, so callers can
    discover parameter types rather than receiving Types.OTHER.
  • Populates getMetaData() with a SELECT's result schema before execution, and
    returns null for statements that produce no ResultSet, per the JDBC contract.
  • Captures the target table and schema of single-table INSERTs for the Storage
    Write API, retried during executeBatch so tables created after prepare time
    still qualify.
  • Degrades safely: a dry run that fails is logged and swallowed, leaving types to
    be derived from caller-supplied values and the count from the query text.

@Neenu1995
Neenu1995 requested review from a team as code owners September 16, 2026 14:11
@Neenu1995
Neenu1995 marked this pull request as draft September 16, 2026 14:11

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces parameter type inference for BigQueryPreparedStatement by executing dry-run queries to retrieve undeclared query parameters. It also adds validation to ensure all parameters are set before execution. The review feedback highlights a compilation error in BigQueryConnection where clear() was replaced with git(), recommends handling potential exceptions when parsing SQL type names to prevent loop termination, advises removing leftover debugging print statements in production code, and suggests correcting tautological assertions in the integration tests to verify actual expected JDBC types.

@Neenu1995
Neenu1995 changed the base branch from main to fix-undeclared-query-parameter September 16, 2026 17:08
Base automatically changed from fix-undeclared-query-parameter to main September 17, 2026 13:21
@Neenu1995
Neenu1995 marked this pull request as ready for review September 18, 2026 01:14
QueryJobConfiguration dryRunConfig =
getJobConfig(query).setDryRun(true).setParameterMode("POSITIONAL").build();
Job dryRunJob = this.bigQuery.create((JobInfo.of(dryRunConfig)));
QueryStatistics jobStatistics = dryRunJob.getStatistics();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it wait for it to complete?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think dryRun jobs don't need wait-t-finish since thay are not actually executing a job. I will double check.

}
}
} catch (Exception ex) {
LOG.warning("Could not infer parameter types via dryRun: " + ex.getMessage());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't look like just a warning, it likely indicates there is an issue with the query.
Also can we do this only in case when this information is needed? e.g. if customer code calls directly setString, do we need to validate the type or just allow it to fail at the 'execute' time?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dryRun could be failing due to a transient error. Should the statement creation fail because of that? Since the Prepared Statement can work even without the dryRun, doesn't it make more sense to fail at execute.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic is incorrect. E.g. SELECT "data??" as s will return 2 as parameter count. We should leverage dry-run data.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. I changed the code to update paramCount from the dryRun stats. But i am keeping this as a fallback in case dryRun fails due to a transient issue or if stats don't get correctly populated for some reason.

@Neenu1995
Neenu1995 requested a review from logachev September 24, 2026 13:14
for (int i = 1; i <= this.parametersArraySize; i++) {

int arrayIndex = i - 1;
if (this.parametersList.size() <= arrayIndex

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this check should happen as a part of getParameter(i).
That way it can distinguish between "unset parameter" and "parameter set to null"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getParameter(i) only looks at 1 parameter. This if block is about "Are there any missing parameters in the PreparedStatement?". In addition, Callable Statement calls getParameter as well. So I don't think it should be in getParameter.

@Neenu1995
Neenu1995 merged commit e989a72 into main Sep 29, 2026
202 checks passed
@Neenu1995
Neenu1995 deleted the add-undeclared-query-parameters branch September 29, 2026 18:04
blakeli0 pushed a commit that referenced this pull request Sep 30, 2026
🤖 I have created a release *beep* *boop*
---


<details><summary>1.93.0</summary>

##
[1.93.0](v1.92.0...v1.93.0)
(2026-09-30)


### ⚠ BREAKING CHANGES

* **automl:** remove java-automl library
([#14540](#14540))

### Features

* **automl:** remove java-automl library
([#14540](#14540))
([6f13a1d](6f13a1d))
* **bigquery-jdbc:** infer undeclared fields in PreparedStatement
([#14400](#14400))
([e989a72](e989a72))
* **gax,gapic-generator-java:** retry start upload call and rename
ResumableUploadCallSettings to ResumableUploadOptions
([#14511](#14511))
([98c2073](98c2073))
* **gax:** add chunk upload recovery loop for resumable uploads
([#14424](#14424))
([8ecd1e5](8ecd1e5))
* **gax:** add ResumableUploadProgressListener and related
infrastructure
([#14426](#14426))
([c7f10d8](c7f10d8))
* **gax:** add RewindableStreamBuffer in prep for chunk upload recovery
([#14423](#14423))
([349c315](349c315))
* **gax:** enforce global timeout for resumable uploads
([#14425](#14425))
([3cb9805](3cb9805))
* **gax:** include upload-status header in resumable upload command
response objects
([#14420](#14420))
([23e49c1](23e49c1))
* **gax:** replace InputStream with InputStreamSupplier in resumable
upload public surfaces
([#14521](#14521))
([3c79730](3c79730))
* **gax:** retry chunk upload on transient errors
([#14422](#14422))
([e72365e](e72365e))
* **gax:** treat resumable upload server rejection as terminal
([#14516](#14516))
([b288788](b288788))
* **gax:** wire up ResumableUploadProgressTracker
([#14427](#14427))
([e7ba3e8](e7ba3e8))
* **grpc-gcp:** Add shared fallback state and probing recovery options
to GcpFallbackChannel
([#14013](#14013))
([675f639](675f639))
* **pubsub:** add publish start time to client telemetry header
([#14496](#14496))
([4b6a671](4b6a671))


### Bug Fixes

* **auth:** exclude javax.annotation-api from api-common dependency
([#14538](#14538))
([668fd18](668fd18)),
refs
[#12363](#12363)
* **bigquery-jdbc:** abort session when connection is closed
([#14273](#14273))
([5bd4520](5bd4520)),
refs
[#13922](#13922)
* **storage:** add App Hub storage.googleapis.com prefix to
destination.id
([#14527](#14527))
([0f6f9b5](0f6f9b5))


### Documentation

* Add gRPC Post-Quantum Cryptography Guide
([#14245](#14245))
([340a239](340a239))
* **samples:** add zonal bucket pre-warmed writer pool sample
([#14517](#14517))
([32e6f87](32e6f87))
* **samples:** pre-warm writer pool channels with flush() after open()
([#14537](#14537))
([df06290](df06290))
* update PQC guide to follow standard template format
([#14333](#14333))
([0b969d5](0b969d5))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants