Documentation

In expressions like longstringchar and longbyteschar, I will suggest changes to make it clearer and more precise.

Before:

longstringchar ::= <any source character except "\">
longbyteschar ::= <any ASCII character except "\">

After:

longstringchar ::= <any source character except "\" or triple quote>
longbyteschar ::= <any ASCII character except "\" or triple quote>

Because the tokenizer can recognize three consecutive quotes (mapped by three longstringchar or longbyteschar) which ends up becoming one triple quote (for example). This will break longstring or longbytes recognition as show below:

longstring ::= "'''" longstringitem* "'''" | '"""' longstringitem* '"""'
longbytes ::= "'''" longbytesitem* "'''" | '"""' longbytesitem* '"""'

longstringitem ::= longstringchar | stringescapeseq
longbytesitem ::= longbyteschar | bytesescapeseq

Otherwise we will have triple quotes inside the longstringitem or longbytesitem. Can this? Absolutely not.

Thank you for your attention.

José Oliveira