Skip to content

CSTValidationError Must have at least one space before 'for' keyword. #854

Closed

Description

@tushar-deepsource

Code

[i if True else ""for i in "abc"]

Parser output

>>> import libcst
>>> libcst.parse_expression('[i if True else ""for i in "abc"]')
Traceback (most recent call last):
  File "/Users/tusharsadhwani/code/marvin-python/venv3/lib/python3.10/site-packages/libcst/_parser/base_parser.py", line 151, in _add_token
    plan = stack[-1].dfa.transitions[transition]
KeyError: ReservedString(])

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/tusharsadhwani/code/marvin-python/venv3/lib/python3.10/site-packages/libcst/_parser/base_parser.py", line 156, in _add_token
    self._pop()
  File "/Users/tusharsadhwani/code/marvin-python/venv3/lib/python3.10/site-packages/libcst/_parser/base_parser.py", line 220, in _pop
    new_node = self.convert_nonterminal(tos.dfa.from_rule, tos.nodes)
  File "/Users/tusharsadhwani/code/marvin-python/venv3/lib/python3.10/site-packages/libcst/_parser/python_parser.py", line 44, in convert_nonterminal
    return self.nonterminal_conversions[nonterminal](self.config, children)
  File "/Users/tusharsadhwani/code/marvin-python/venv3/lib/python3.10/site-packages/libcst/_parser/conversions/expression.py", line 1152, in convert_testlist_comp_list
    return _convert_testlist_comp(
  File "/Users/tusharsadhwani/code/marvin-python/venv3/lib/python3.10/site-packages/libcst/_parser/conversions/expression.py", line 1183, in _convert_testlist_comp
    comprehension_type(elt=elt.value, for_in=for_in, lpar=(), rpar=()),
  File "<string>", line 9, in __init__
  File "/Users/tusharsadhwani/code/marvin-python/venv3/lib/python3.10/site-packages/libcst/_nodes/base.py", line 118, in __post_init__
    self._validate()
  File "/Users/tusharsadhwani/code/marvin-python/venv3/lib/python3.10/site-packages/libcst/_nodes/expression.py", line 3511, in _validate
    raise CSTValidationError(
libcst._nodes.base.CSTValidationError: Must have at least one space before 'for' keyword.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/tusharsadhwani/code/marvin-python/venv3/lib/python3.10/site-packages/libcst/_parser/entrypoints.py", line 160, in parse_expression
    result = _parse(
  File "/Users/tusharsadhwani/code/marvin-python/venv3/lib/python3.10/site-packages/libcst/_parser/entrypoints.py", line 56, in _parse
    return _pure_python_parse(
  File "/Users/tusharsadhwani/code/marvin-python/venv3/lib/python3.10/site-packages/libcst/_parser/entrypoints.py", line 89, in _pure_python_parse
    result = parser.parse()
  File "/Users/tusharsadhwani/code/marvin-python/venv3/lib/python3.10/site-packages/libcst/_parser/base_parser.py", line 110, in parse
    self._add_token(token)
  File "/Users/tusharsadhwani/code/marvin-python/venv3/lib/python3.10/site-packages/libcst/_parser/base_parser.py", line 170, in _add_token
    raise ParserSyntaxError(
libcst._exceptions.ParserSyntaxError: Syntax Error @ 1:33.
Internal error: Must have at least one space before 'for' keyword.

[i if True else ""for i in "abc"]
                                ^

Activity

  1. added
    bugSomething isn't working
    parsingConverting source code into CST nodes
    on May 25, 2023
  2. zsol commented on May 25, 2023

    @zsol
    Contributor

    Note that the parse error doesn't happen anymore with the native parser, but the validation error is still there. That should be reasonably easy to fix

  3. added a commit that references this issue on May 28, 2023
    aa51671
  4. mx781 commented on Dec 28, 2023

    @mx781

    @zsol Looks like there might still be some uncovered cases - the following fails on the latest PyPI version:

    cst.parse_module("[(b if 1 else a)for x in []]")
  5. added a commit that references this issue on Jan 31, 2024
    2acc293
  6. mx781 commented on Feb 21, 2024

    @mx781

    @zsol seems like it's something specific to ternary statements that's not getting parsed correctly as the following also fail with a similar error:

    cst.parse_module('assert(1 if True else 0)')
    cst.parse_module('1 if True else(1 if True else 0)')
  7. zsol commented on Feb 25, 2024

    @zsol
    Contributor

    We'll probably need to relax the validation rules even further than #935, if it's still broken in recent releases

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingparsingConverting source code into CST nodes

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions