Skip to content

JS: experimental query for a multi-character pattern applied to a single stream chunk - #22738

Open
ninadphalak wants to merge 1 commit into
github:mainfrom
ninadphalak:js/split-boundary-chunk-filter
Open

ninadphalak wants to merge 1 commit into
github:mainfrom
ninadphalak:js/split-boundary-chunk-filter

Conversation

@ninadphalak

Copy link
Copy Markdown

Summary

Adds javascript/ql/src/experimental/Security/CWE-693/SplitBoundaryChunkFilter.ql
(js/split-boundary-chunk-filter, path-problem, precision medium) with query help, examples
and a test.

A stream arrives in chunks whose boundaries are chosen by the sender or transport. A filter
that applies a regular expression to each chunk on its own never sees a value that straddles
two chunks, so a redaction filter forwards it and a blocking filter lets it through. Whole-string
tests pass, which is why the defect survives review. It was found and fixed independently in
several streaming guardrail frameworks this year and reappears in general stream replacers and
secret scanners; the mechanism is the one described for network intrusion detection in 1998.

What the query does

  • Sources: the chunk parameter of a per-chunk callback: Node Transform/Writable
    (_transform, _write, or the transform/write option), web TransformStream
    transformers, through2-style callbacks, 'data' listeners, and for await loop variables;
    plus the part of a Mastra processOutputStream processor as one framework example.
  • Flow: data flow from the chunk through property reads, toString/trim/decode/String
    and across calls.
  • Sinks: replace, replaceAll, match, matchAll, search, split with a regular
    expression that can match two or more characters (resolved through RegExp::getRegExpFromNode),
    and test/exec; for test/exec with an unresolvable receiver the alert says the pattern
    width is unknown.
  • Not flagged: the pattern applied to the chunk joined with carried text (tail + chunk), or to
    an accumulated buffer, because data flow does not pass through string concatenation. That is
    the fix shape the help recommends.

Evaluation

  • Test corpus: 8 positive shapes flagged, 5 negative shapes (buffer-to-end, bounded hold-back,
    single-character pattern, non-stream code, pattern on an unrelated string) not flagged.
  • Real code: run on three revisions of Mastra's pii-detector.ts. No results on any, and that
    is correct: the original per-chunk version used an LLM call rather than a regular expression,
    and the later revisions carry a tail. I have no confirmed open-source hit for the regex form
    yet; the query encodes the defect class, and I would welcome pointers to projects to run it on.

Checklist

  • query help with examples
  • test with .expected generated by codeql test run
  • codeql query format
  • change note: not added, on the understanding that experimental queries do not need one;
    happy to add if required

… single stream chunk

A regular expression that can match two or more characters, applied to one chunk of a
stream with nothing carried from the previous chunk, never sees a value that straddles
two chunks. Flags per-chunk callbacks (Transform, TransformStream, 'data' listeners,
for-await loops) where the chunk flows to such a pattern. Includes query help, examples
and a test.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant