Describe the bug
get_unique_constraints on the SQLite dialect scans the stored CREATE TABLE
text (taken verbatim from sqlite_master.sql) with an inline-UNIQUE regex
whose tail is [\t ]+[a-z0-9_ ]+?[\t ]+UNIQUE. The lazy middle class itself
contains a space, so all three quantifiers can lay claim to the same space
character. When a column definition holds a run of whitespace that isn't
followed by UNIQUE, the engine tries every way of splitting that run across
the three quantifiers, which is cubic in the length of the run.
SQLite preserves the original whitespace of a CREATE TABLE statement in
sqlite_master, so any account that can create a table can leave a long gap
in a column definition and make later reflection of that schema hang. This is
a denial-of-service on schema reflection.
SQLAlchemy Version in Use
2.1.0b3 (also present on 2.0)
DBAPI (i.e. the database driver)
pysqlite (stdlib sqlite3)
Database Vendor and Major Version
SQLite
Python Version
3.11
Operating system
Linux / macOS
To Reproduce
from sqlalchemy import create_engine, inspect
e = create_engine("sqlite://")
with e.begin() as c:
c.exec_driver_sql(
"CREATE TABLE t (x INTEGER" + " " * 1000 + "NOT NULL, "
"y INTEGER NOT NULL UNIQUE)"
)
inspect(e).get_unique_constraints("t") # hangs ~11s before, instant after
Error
No exception; get_unique_constraints just spends cubic time backtracking
before returning. Timing the tail pattern directly on the run of whitespace:
200 spaces: 93 ms
400 spaces: 702 ms
800 spaces: 5580 ms
roughly 8x per doubling, i.e. O(n^3). At 1000 spaces the full reflection call
takes about 11 seconds.
Additional context
Fix is to tokenise the inter-keyword whitespace with disjoint classes so a
space only ever belongs to a separator and never to a token. PR #13398.
Describe the bug
get_unique_constraintson the SQLite dialect scans the storedCREATE TABLEtext (taken verbatim from
sqlite_master.sql) with an inline-UNIQUEregexwhose tail is
[\t ]+[a-z0-9_ ]+?[\t ]+UNIQUE. The lazy middle class itselfcontains a space, so all three quantifiers can lay claim to the same space
character. When a column definition holds a run of whitespace that isn't
followed by
UNIQUE, the engine tries every way of splitting that run acrossthe three quantifiers, which is cubic in the length of the run.
SQLite preserves the original whitespace of a
CREATE TABLEstatement insqlite_master, so any account that can create a table can leave a long gapin a column definition and make later reflection of that schema hang. This is
a denial-of-service on schema reflection.
SQLAlchemy Version in Use
2.1.0b3 (also present on 2.0)
DBAPI (i.e. the database driver)
pysqlite (stdlib sqlite3)
Database Vendor and Major Version
SQLite
Python Version
3.11
Operating system
Linux / macOS
To Reproduce
Error
No exception;
get_unique_constraintsjust spends cubic time backtrackingbefore returning. Timing the tail pattern directly on the run of whitespace:
roughly 8x per doubling, i.e. O(n^3). At 1000 spaces the full reflection call
takes about 11 seconds.
Additional context
Fix is to tokenise the inter-keyword whitespace with disjoint classes so a
space only ever belongs to a separator and never to a token. PR #13398.