Skip to content

catastrophic regex backtracking in sqlite inline UNIQUE reflection (ReDoS on get_unique_constraints) #13419

Description

@dxbjavid

Describe the bug

get_unique_constraints on the SQLite dialect scans the stored CREATE TABLE
text (taken verbatim from sqlite_master.sql) with an inline-UNIQUE regex
whose tail is [\t ]+[a-z0-9_ ]+?[\t ]+UNIQUE. The lazy middle class itself
contains a space, so all three quantifiers can lay claim to the same space
character. When a column definition holds a run of whitespace that isn't
followed by UNIQUE, the engine tries every way of splitting that run across
the three quantifiers, which is cubic in the length of the run.

SQLite preserves the original whitespace of a CREATE TABLE statement in
sqlite_master, so any account that can create a table can leave a long gap
in a column definition and make later reflection of that schema hang. This is
a denial-of-service on schema reflection.

SQLAlchemy Version in Use

2.1.0b3 (also present on 2.0)

DBAPI (i.e. the database driver)

pysqlite (stdlib sqlite3)

Database Vendor and Major Version

SQLite

Python Version

3.11

Operating system

Linux / macOS

To Reproduce

from sqlalchemy import create_engine, inspect

e = create_engine("sqlite://")
with e.begin() as c:
    c.exec_driver_sql(
        "CREATE TABLE t (x INTEGER" + " " * 1000 + "NOT NULL, "
        "y INTEGER NOT NULL UNIQUE)"
    )

inspect(e).get_unique_constraints("t")   # hangs ~11s before, instant after

Error

No exception; get_unique_constraints just spends cubic time backtracking
before returning. Timing the tail pattern directly on the run of whitespace:

200 spaces:   93 ms
400 spaces:  702 ms
800 spaces: 5580 ms

roughly 8x per doubling, i.e. O(n^3). At 1000 spaces the full reflection call
takes about 11 seconds.

Additional context

Fix is to tokenise the inter-keyword whitespace with disjoint classes so a
space only ever belongs to a separator and never to a token. PR #13398.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingreflectionreflection of tables, columns, constraints, defaults, sequences, views, everything else

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions