FEAT: TVP support for async queries by subrata-ms · Pull Request #814 · microsoft/mssql-python

Subrata (subrata-ms) · 2026-09-24T10:08:29Z

Work Item / Issue Reference

AB#47998,48435

GitHub Issue: #<ISSUE_NUMBER>

Summary

This pull request introduces several improvements and enhancements to the mssql_python.async_query async API, focusing on better native type integration, more robust and consistent cursor behavior, and improved handling of parameter iterables in executemany. It also expands the test suite to cover these changes and edge cases.

Key highlights include:

Lazy exposure of native SQL type hints and the table-valued parameter constructor.
Improved type and error handling for parameter iterables in executemany.
New and clarified properties and behaviors for async cursors, including a closed property.
Expanded and refined test coverage for new and existing features.

Native type integration and exports

Added lazy exports for native SQL type hints (SQL_MONEY, SQL_SMALLMONEY, SQL_XML, SQL_JSON, SQL_VECTOR) and the internal _TableValuedParameter constructor, with dynamic loading and error reporting if the native dependency is missing. These are now included in __all__ and dir() as appropriate.
Added tests to verify that these native exports are correctly exposed, loaded only when accessed, and report missing features properly.

AsyncCursor API and behavior improvements

Added a closed property to async cursors to reflect both wrapper and parent connection state, with tests for idempotency and parent connection closure propagation.
Clarified and enhanced docstrings for nextset, close, description, and rowcount to document async-specific behaviors and differences from the synchronous API.
Added tests to ensure nextset correctly tracks rowcount and description per result, including edge cases.

Executemany and parameter iterable handling

Changed executemany signatures to accept any synchronous iterable (not just sequences), improved error handling for non-iterables, and ensured iterables are consumed only once. Streaming/asynchronous iterables are explicitly not supported.
Added tests for handling of empty generators, iterator failures, and correct consumption of parameter iterables.

Logging and test adjustments

Updated logging to remove reliance on batch_count for executemany and adjusted test expectations accordingly.
Minor imports and test harness updates to support new features.

These changes make the async query API more robust, Pythonic, and consistent with both native and DB-API expectations.

Expose native TVP and SQL type-hint helpers lazily. Support iterable executemany batches and preserve result state on iteration failure. Add cursor closed-state reporting, document async result contracts, and cover these behaviors with async integration tests.

github-actions · 2026-09-24T10:08:53Z

PR Performance Report

⛔ Performance unavailable

Performance could not be assessed because no environment produced a complete result.

0 IMPROVEMENTS 0 SLOWDOWNS 0/2 ENVIRONMENTS

Coverage: 0 of 2 environments completed. Advisory result; does not block merging.

Unavailable: Unix / SQL Server 2022 (missing); Unix / SQL Server 2025 (missing).

Performance diagnostics

Phase times are inclusive diagnostics and must not be added together. They identify where measured time changed, not why it changed.

No affected phases or call-count changes were recorded.

All database tasks and timings

Build and measurement details

ADO build 177976

PR head: 0579d4b55365745d13360c0bd1421679c4bd223f

A consistent change requires more than 20% median paired movement, at least 1 ms between the median runtimes, and at least 80% of pairs exceeding the relative threshold in the same direction. A slowdown without enough pair agreement is reported as inconsistent.

The displayed change is the median of paired before-and-after ratios. It is not recalculated from the two displayed median runtimes.

Unavailable or rejected data: Linux-SQL2022 (missing), Linux-SQL2025 (missing)

Both revisions use profiling-enabled builds on the same agent and database, with alternating order and discarded warmups. Results are diagnostic and do not represent production-wheel latency.

Raw samples and logs are attached to the ADO run as profiler-* artifacts.

Copilot

Copilot review overview

🟡 Changes recommended

Guard the optional native dependency test and complete the required PR metadata.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1

Open (1)

Skip loader test when mssql_py_core is unavailable · New

What changed in this PR

Adds preview async TVP/type-hint support, iterable executemany, cursor state reporting, and expanded integration tests.

Changes:

Adds lazy native TVP and SQL type-hint exports.
Supports synchronous iterable batches and failure-result preservation.
Documents and tests async cursor/result behavior.

File	Summary
`tests/AsyncTest/test_006_async_execute.py`	Tests iterable batches, TVPs, and type hints.
`tests/AsyncTest/test_005_async_cursor.py`	Tests cursor state and result behavior.
`tests/AsyncTest/test_004_async_logging.py`	Updates execution logging expectations.
`tests/AsyncTest/test_001_async_query_native.py`	Tests native exports. Finding: moderate, 2 votes—add an `importorskip` guard for the optional dependency at lines 55 and 65.
`mssql_python/async_query/async_execute.py`	Adds iterable batch execution and failure handling.
`mssql_python/async_query/async_cursor.py`	Adds cursor state and result-contract documentation.
`mssql_python/async_query/__init__.py`	Adds lazy native exports. Finding: nit, 1 vote—replace PR placeholders and empty summary with valid metadata.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Copilot review overview

🟡 Changes recommended

Fix the indentation error that prevents the native async test module from being collected.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1

Open (1)

Fix indentation causing test module parse failure · New

Resolved since last review (1)

Skip loader test when mssql_py_core is unavailable

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Copilot review overview

🔵 Needs a closer look

Fix the indentation error that prevents the native async test module from being collected.

Review effort: Lite
Findings: None

Resolved since last review (1)

Fix indentation causing test module parse failure

Copilot

Copilot review overview

🔵 Needs a closer look

Cancellation handling and version-specific collation test behavior need correction.

Review effort: Lite
Findings: None

github-actions · 2026-09-24T11:15:54Z

📊 Code Coverage Report

🔥 Diff Coverage 83%	🎯 Overall Coverage 84%	📈 Total Lines Covered: `9381` out of `11070` 📁 Project: `mssql-python`

Diff Coverage

Diff: main...HEAD, staged and unstaged changes

mssql_python/async_query/init.py (100%)
mssql_python/async_query/async_cursor.py (100%)
mssql_python/async_query/async_execute.py (50.0%): Missing lines 102-106

Summary

Total: 31 lines
Missing: 5 lines
Coverage: 83%

mssql_python/async_query/async_execute.py

Lines 98-110

   98     iteration_failed = False
   99 
  100     def parameter_rows():
  101         nonlocal iteration_failed
! 102         try:
! 103             yield from seq_of_parameters
! 104         except BaseException:
! 105             iteration_failed = True
! 106             raise
  107 
  108     logger.debug(
  109         "AsyncCursor.executemany: starting; use_prepare=%s",
  110         use_prepare,

📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)

mssql_python.pybind.performance_counter.hpp: 0.7%
mssql_python.pybind.logger_bridge.cpp: 57.9%
mssql_python.pybind.ddbc_bindings.h: 62.6%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.ddbc_bindings.cpp: 79.1%
mssql_python.pybind.connection.connection_pool.cpp: 82.3%
mssql_python.pybind.connection.connection.cpp: 83.1%
mssql_python.logging.py: 86.2%
mssql_python.pooling.py: 90.1%
mssql_python.async_query.async_execute.py: 91.5%

🔗 Quick Links

⚙️ Build Summary	📋 Coverage Details
View Azure DevOps Build	Browse Full Coverage Report

Copilot

Copilot review overview

🔵 Needs a closer look

The pinned native core dependency must be updated or the new exports gated before approval.

Review effort: Lite
Findings: None

Copilot

Copilot review overview

🟡 Changes recommended

The UTF-8 collation test must skip servers that do not support that version-specific collation.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1

Open (1)

Guard UTF-8 collation test for older SQL Server versions · New

Skip test if SQL Server does not support the specified collation. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Copilot review overview

🔵 Needs a closer look

Correctness depends on external Rust-core behavior and live SQL Server integration that could not be exercised here.

Review effort: Balanced
Findings: None

Resolved since last review (1)

Guard UTF-8 collation test for older SQL Server versions

Copilot

Copilot review overview

🔵 Needs a closer look

Native-backed TVP behavior and the extensive live-SQL execution matrix require final human validation.

Review effort: Balanced
Findings: None

Copilot

Copilot review overview

🟡 Changes recommended

A new fetch test relies on undefined SQL row ordering and may fail across execution plans or server versions.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1

Open (1)

Add explicit ordering to stabilize UNION ALL result assertions · New

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Copilot review overview

🔵 Needs a closer look

Correctness depends heavily on external native bindings and live SQL Server behavior requiring final human validation.

Review effort: Balanced
Findings: None

Resolved since last review (1)

Add explicit ordering to stabilize UNION ALL result assertions

Copilot AI lite review requested due to automatic review settings September 24, 2026 10:08

Copilot started reviewing on behalf of Subrata (subrata-ms) September 24, 2026 10:09 View session

Subrata (subrata-ms) changed the title ~~FEAT: Add preview TVP support for async queries~~ FEAT: Add TVP support for async queries Sep 24, 2026

github-actions Bot added the pr-size: medium Moderate update size label Sep 24, 2026

Copilot AI reviewed Sep 24, 2026

View reviewed changes

Comment thread tests/AsyncTest/test_001_async_query_native.py Outdated

Change loading of py_core to use pytest importorskip

d3fd82d

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings September 24, 2026 10:14

Merge branch 'main' into subrata-ms/AQEIntegration

19bd060

Copilot started reviewing on behalf of Subrata (subrata-ms) September 24, 2026 10:15 View session

Copilot AI reviewed Sep 24, 2026

View reviewed changes

Comment thread tests/AsyncTest/test_001_async_query_native.py Outdated

enhancing String test cases and coverage

23dd23c

Copilot AI review requested due to automatic review settings September 24, 2026 11:05

github-actions Bot added pr-size: large Substantial code update and removed pr-size: medium Moderate update size labels Sep 24, 2026

Copilot started reviewing on behalf of Subrata (subrata-ms) September 24, 2026 11:05 View session

Subrata (subrata-ms) and others added 2 commits September 24, 2026 16:35

Fix indentation in async type hints test

94ba027

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Merge branch 'main' into subrata-ms/AQEIntegration

c44930b

Copilot AI reviewed Sep 24, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings September 24, 2026 11:08

Copilot started reviewing on behalf of Subrata (subrata-ms) September 24, 2026 11:09 View session

Copilot AI reviewed Sep 24, 2026

View reviewed changes

CHORE: Expand async iterator failure integration coverage

5dd4a19

Copilot AI review requested due to automatic review settings September 24, 2026 11:34

Copilot started reviewing on behalf of Subrata (subrata-ms) September 24, 2026 11:35 View session

Copilot AI reviewed Sep 24, 2026

View reviewed changes

Subrata (subrata-ms) requested a balanced review from Copilot September 24, 2026 11:40

Copilot started reviewing on behalf of Subrata (subrata-ms) September 24, 2026 11:40 View session

Copilot AI reviewed Sep 24, 2026

View reviewed changes

Comment thread tests/AsyncTest/test_007_async_fetch.py

Add collation support check in async fetch test

59aeacf

Skip test if SQL Server does not support the specified collation. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings September 24, 2026 11:52

Copilot started reviewing on behalf of Subrata (subrata-ms) September 24, 2026 11:52 View session

Subrata (subrata-ms) changed the title ~~FEAT: Add TVP support for async queries~~ FEAT: TVP support for async queries Sep 24, 2026

Copilot AI reviewed Sep 24, 2026

View reviewed changes

Subrata (subrata-ms) marked this pull request as ready for review September 24, 2026 11:56

Subrata (subrata-ms) requested review from Gaurav Sharma (bewithgaurav), gargsaumya, Jahnvi Thakkar (jahnvi480), Saurabh Singh (saurabh500) and Sumit Sarabhai (sumitmsft) September 24, 2026 11:58

FIX: Remove privileged DBCC calls from async truncation test

7e0d70c

Copilot AI review requested due to automatic review settings September 24, 2026 13:58

Copilot started reviewing on behalf of Subrata (subrata-ms) September 24, 2026 13:59 View session

Copilot AI reviewed Sep 24, 2026

View reviewed changes

Merge branch 'main' into subrata-ms/AQEIntegration

fca873c

Copilot AI review requested due to automatic review settings September 24, 2026 15:30

Copilot started reviewing on behalf of Subrata (subrata-ms) September 24, 2026 15:31 View session

Copilot AI reviewed Sep 24, 2026

View reviewed changes

Comment thread tests/AsyncTest/test_007_async_fetch.py Outdated

Refactor SQL query to handle empty values

0579d4b

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings September 24, 2026 15:54

Copilot started reviewing on behalf of Subrata (subrata-ms) September 24, 2026 15:55 View session

Copilot AI reviewed Sep 24, 2026

View reviewed changes

Conversation

Subrata (subrata-ms) commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Work Item / Issue Reference

Summary

Native type integration and exports

AsyncCursor API and behavior improvements

Executemany and parameter iterable handling

Logging and test adjustments

Uh oh!

github-actions Bot commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Performance Report

⛔ Performance unavailable

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

github-actions Bot commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📊 Code Coverage Report

🔥 Diff Coverage

83%

🎯 Overall Coverage

84%

Diff Coverage

Diff: main...HEAD, staged and unstaged changes

Summary

mssql_python/async_query/async_execute.py

📋 Files Needing Attention

🔗 Quick Links

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Subrata (subrata-ms) commented Sep 24, 2026 •

edited

Loading

github-actions Bot commented Sep 24, 2026 •

edited

Loading

github-actions Bot commented Sep 24, 2026 •

edited

Loading