fix(dialect): convert JSON test values to NULL and valid JSON - #6113
Open
manan28025 wants to merge 1 commit into
Open
manan28025 wants to merge 1 commit into
manan28025 wants to merge 1 commit into
Conversation
transform_values wrapped every JSON value in PARSE_JSON('{value}'), so a
column left out of a unit test's input rows became PARSE_JSON('None') and
DuckDB failed with "Malformed JSON". Dicts were rendered with Python's repr,
and quotes in the value broke the SQL string. None now becomes NULL, dicts
and lists are serialized with json.dumps, and the value is passed as an
escaped string literal.
Signed-off-by: neatninja <manan81140@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #5387.
transform_values()wrapped every JSON value asPARSE_JSON('{value}'). When a JSON column is left out of a unit test's input rows, its value isNone, so the fixture becamePARSE_JSON('None')and DuckDB failed withMalformed JSON at byte 0 of input: unexpected character. Input: "None". The same formatting also broke two other cases: a dict given for a JSON column was rendered with Python's repr ({'a': 1}, not valid JSON), and a single quote in the value ended the SQL string early.Now
NonebecomesNULL, dicts, lists and other Python scalars are serialized withjson.dumps, and the value is passed toPARSE_JSONas an escaped string literal. JSON strings without quotes produce the same SQL as before.Test Plan
test_select_from_values_for_batch_range_json_values(NULL, a dict, and a string with a single quote). It fails on main.test_json_column_missing_from_input_rows, a unit test that leaves a JSON column out of its input rows. It fails on main with the error from the issue and passes now.Checklist
make styleand fixed any issuesmake fast-test) (ran the tests/core fast and slow suites, see above)git commit -s) per the DCO