Skip to content

Tags: sqlc-dev/darkwing

Tags

v0.1.0

Toggle v0.1.0's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Milestone 6: hardening — error fidelity, difftest, benchmarks, fuzzing (

#5)

* Match the pinned build's parser error messages over the negative corpus

Milestone 6, error fidelity: every must-reject corpus case now renders
the oracle's recorded first line verbatim, gated by the new
TestCorpusErrorMessages in errors_test.go alongside hand-written
position (Offset/Line/Column) pins.

Message fixes to reach zero divergence:
- Whole-query UTF-8 validation before parsing ("Invalid UTF-8 in query
  (byte sequence mismatch)"), via a new port of Utf8Proc::Analyze that
  classifies byte mismatches vs invalid codepoints and, like upstream,
  accepts overlong encodings; escape-string literals reuse it for the
  "byte mismatch"/"invalid unicode" suffixes.
- Column-without-type errors quote the name per
  KeywordHelper::WriteOptionallyQuoted (keywords and non-plain
  identifiers quoted, plain ones bare).
- Chained-comparison ban transforms its operands first so deeper errors
  (e.g. Empty subscript '[]') win, and uses upstream's wording.
- Window frames validate the start bound before the end bound.
- Interval range specifiers name the range (YEAR TO MONTH is not
  supported) instead of a generic message.
- RESET options name the last offending option; PIVOT-on-subquery
  renders the expression's source text; plus upstream wording for
  CREATE TABLE AS, OR REPLACE/IF NOT EXISTS, OR REPLACE|IGNORE with
  ON CONFLICT, COMMENT ON qualification depth, and NEAREST counts
  (which now also reject non-positive literals, retiring the last
  nearest_errors todo).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QQHouvcnCnGT1MuHvwPJrj

* Add milestone 6 benchmarks, fuzz target and parallel-parse race gate

- bench_test.go: tokenize + parse throughput on simple/complex/mixed
  statement workloads, plus upstream's pathological unmatched-parens
  packrat case as both a benchmark and a fast-failure regression test
  (19 and 40 parens complete in milliseconds thanks to memoization).
- fuzz_test.go: FuzzParse holds Parse to its public invariants on any
  input - errors are *parser.Error with in-range offset and 1-based
  line/column, successful parses return non-nil statements whose spans
  tile the input. Seeds cover the dialect's corners (FROM-first, PIVOT,
  list comprehensions, dollar quotes, invalid UTF-8, chained
  comparisons); a plain test keeps the seeds honest in ordinary runs
  (3-minute local fuzz run: 2.4M execs, no failures).
- TestParallelParse hammers the shared compiled engine from concurrent
  goroutines so the -race dev loop and CI gate the engine's immutability
  and per-call packrat state.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QQHouvcnCnGT1MuHvwPJrj

* Add cmd/difftest, the mutation differential fuzzer, and fix what it found

Milestone 6, differential loop: cmd/difftest mutates corpus seed
statements at token-span boundaries (delete, duplicate, swap adjacent,
cross-seed replace, truncate, splice) and compares darkwing's verdict -
and, for syntax errors, the message first line - against the pinned
DuckDB CLI. Deterministic per -seed, parallel oracle workers, bounded by
-n or -duration; a nightly GitHub Actions lane runs it (plus a native
fuzz pass) while the DuckDB nightly artifact still matches the pin, and
skips green once the artifact drifts.

Known oracle imprecisions are filtered, not reported: nested-SQL
functions whose bind-time inner parse raises Parser Error, bind-time
ParserExceptions outside the parse pipeline (the corpus todo category),
statements the CLI shell intercepts (column-0 '#' comments, dot
commands), and multi-statement mutants where a post-parse error aborts
the batch CLI before later statements parse - duckdbsrc.Verdict now
records that with PostParseError.

Real divergences it caught, all fixed with regression tests:

- SyntaxError rendered token spellings through Go %q escaping; upstream
  puts the raw spelling between plain quotes (embedded quotes and
  backslashes verbatim).
- COLLATE rejected constant collation names; upstream accepts constants
  via Value::ToString ('x' COLLATE 'nocase', even COLLATE 3) and only
  fails lookup at bind time, and raises NotImplementedException (post-
  parse) for other expression shapes, so darkwing now accepts those too.
- Number tokens whose text has no value ("2.03.0") raised a parser
  error; upstream accepts the token and throws InvalidInputException
  while constructing the value, so darkwing records the raw text as a
  cast and accepts.
- Dollar-quoted string tokens record the rewritten literal's length
  (upstream behavior pinned by query_location_length in the serialize
  goldens), which could push a statement span past the end of the
  input; consumedEnd now clamps so statement spans keep their tiling
  contract.

Validated with ~24k mutants across seven RNG seeds (zero disagreements)
and a corpus-wide serialize sweep against the live pinned CLI (25,982
statements, zero mismatches).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QQHouvcnCnGT1MuHvwPJrj

---------

Co-authored-by: Claude <noreply@anthropic.com>