Skip to content

Support ClickHouse transformer STRICT modifier and EXCEPT regex patterns - #2737

Merged
manticore-projects merged 1 commit into
JSQLParser:masterfrom
fudianchn:feat/transformers-strict-regex-2636
Oct 2, 2026
Merged

manticore-projects merged 1 commit into
JSQLParser:masterfrom
fudianchn:feat/transformers-strict-regex-2636

Conversation

@fudianchn

@fudianchn fudianchn commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

AI disclosure: this change was prepared with AI coding agents, reviewed and revised line by line by me.

What

Support ClickHouse EXCEPT STRICT (a, b), REPLACE STRICT(1 AS a, 2 AS b) and EXCEPT '^tmp_' after *, t.* and COLUMNS('m'), including transformer chains. Switching an EXCEPT transformer between its pattern and column-list forms now replaces the previous payload.

Why

The STRICT and regex forms reported in #2636 fail to parse on the base commit. After changing a parsed EXCEPT pattern to a column list, the previous PR head still prints the old pattern. Changing a column list to a pattern also leaves the discarded columns visible to expression visitors.

How

  1. Keep the transformer production and its four lookahead gates aligned when recognizing optional STRICT and regex payloads.
  2. Store STRICT per transformer and represent the EXCEPT regex as a StringValue.
  3. Render the new forms through the existing output paths, preserving non-STRICT formatting and the REPLACE(...) spacing convention. Include the pattern in COLUMNS-side expression traversal.
  4. Both non-null EXCEPT payload setters clear the alternative payload. Setting either field to null preserves the alternative, so clearing an inactive field does not discard the active payload.

Root cause

The transformer grammar only recognized parenthesized column lists and replacements without STRICT. Adding the regex payload also allowed both EXCEPT fields to remain populated: rendering preferred the pattern while expression traversal collected both payloads.

Testing

  • All nine position × form combinations from the third example group in [BUG] 5.4-SNAPSHOT : ClickHouse : asterisk column modifiers fail or mis-parse (APPLY as alias, docs chain, STRICT, EXCEPT regex) #2636 reject on base bb55bb9d and parse and round-trip with this change. The existing STRICT, pattern, alias and invalid-form tests remain green.
  • ColumnsTransformerTest adds eight cases. Both switching directions are tested after *, t.* and COLUMNS('m'), including toString(), StatementDeParser, reparsing and COLUMNS expression traversal. These six cases fail on previous PR head 97985bc and pass with the fix; the two null-setter guards pass on both sides.
  • JDK 17: ./gradlew --offline --no-daemon spotlessApply check passed with 9417 test cases, zero failures/errors and 25 skipped. Grammar ambiguity, formatting, Checkstyle, PMD, SpotBugs and applicable coverage checks passed.
  • No ClickHouse server validation or new performance benchmark was run in this update.

Behavior notes

  • STRICT belongs to each individual EXCEPT or REPLACE transformer. APPLY and EXCLUDE do not gain a STRICT form. Both non-null EXCEPT payload setters maintain mutual exclusion.
  • EXCEPT a and EXCEPT ('^p') remain unsupported. Regex patterns serialize without parentheses.
  • The existing wildcard visitor/deparser paths do not dispatch transformer children. Pattern traversal and custom child rewriting work through COLUMNS; the inherited wildcard limitation is outside this payload-switching fix.

Verification of the original issue

The follow-up scope is the third example group of #2636, already closed by #2653:

SELECT * EXCEPT STRICT (a, b) FROM t;
SELECT * REPLACE STRICT (1 AS a, 2 AS b) FROM t;
SELECT * EXCEPT '^tmp_' FROM t;

Each example rejects on base bb55bb9d and parses and round-trips with this change, also after t.* and COLUMNS('m'). REPLACE output follows the existing REPLACE STRICT(...) spacing.

Fixes #2636

…PT regex patterns

Signed-off-by: 付典 <fudianchn@gmail.com>
@fudianchn
fudianchn force-pushed the feat/transformers-strict-regex-2636 branch from 97985bc to 4f161e3 Compare October 1, 2026 16:47
@manticore-projects
manticore-projects merged commit 6d1ec1d into JSQLParser:master Oct 2, 2026
10 checks passed
@manticore-projects

Copy link
Copy Markdown
Contributor

Why only Regex? I think they should have implement Lua scripting there. :-D

@fudianchn

Copy link
Copy Markdown
Contributor Author

I'll forward it and threaten to rewrite ClickHouse in Java myself if they refuse. That'll show 'em. ;-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] 5.4-SNAPSHOT : ClickHouse : asterisk column modifiers fail or mis-parse (APPLY as alias, docs chain, STRICT, EXCEPT regex)

2 participants