Repository navigation
IDLE needs syntax highlighting for f-strings #73473
Description
| BPO | 29287 |
|---|---|
| Nosy | @rhettinger, @terryjreedy |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/terryjreedy'
closed_at = None
created_at = <Date 2017-01-16.22:20:08.028>
labels = ['expert-IDLE', 'type-feature', '3.7']
title = 'IDLE needs syntax highlighting for f-strings'
updated_at = <Date 2017-01-31.18:50:14.225>
user = 'https://github.com/rhettinger'bugs.python.org fields:
activity = <Date 2017-01-31.18:50:14.225>
actor = 'terry.reedy'
assignee = 'terry.reedy'
closed = False
closed_date = None
closer = None
components = ['IDLE']
creation = <Date 2017-01-16.22:20:08.028>
creator = 'rhettinger'
dependencies = []
files = []
hgrepos = []
issue_num = 29287
keywords = []
message_count = 2.0
messages = ['285585', '286547']
nosy_count = 3.0
nosy_names = ['rhettinger', 'terry.reedy', 'peter.otten']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'test needed'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue29287'
versions = ['Python 3.7']Metadata
Metadata
Assignees
Projects
- Status
Activity
rhettinger commentedon Jan 16, 2017
Follow the lead from Vim, MagicPython, and PyCharm. Needs separate colorization to make the expression distinct from the rest of the string. Needs close-brace matching. Would be desirable to have autocompletion as well.
https://twitter.com/raymondh/status/819085176764366848
terryjreedy commentedon Jan 31, 2017
This is 3 related but somewhat distinct proposals.
Special handling (normal syntax colorizing) of f-expressions (the grammatical term used at https://docs.python.org/3/reference/lexical_analysis.html#formatted-string-literals).
Brace matching within the strings. (For normal strings, this would be nice when the string is to be used as a format string, but there is no way to know how the string will be used.)
Normal identifier autocompletion (as opposed to within-string file name autocompletion).
I have two types of doubts about #1
Should it be done?
a. I have not yet used f-strings, so I not know if I would want uncolorized holes within them.
b. Do beginners use f-strings? Would they find this more useful than confusing?
c. Is this the sort of 'advanced feature' that Guido has said should *not* be copied from other editors?
Can it be done sensibly within the limits of IDLE's colorizer.
IDLE's colorizer defines a giant regex that joins regexes that match keywords, builtins, comments, and strings (and newlines for synchronization). Each of the latter is a named capturing group that joins alternatives for that group. Keywords and built-in names are recognized when complete. Partial comments and strings are recognized as soon as '#' or an open quote is typed. There is a human-verified test of colorizing that could, I believe, be turned into a unit test of the re matching. This would be needed for approach b. below.
The compiled re is used in a ColorDelegator instance that is part of a chain of delegators tied to a text widget. The class code is not documented and I do not understand it well enough to modify it without adding tests. But it was not designed for easy testing.
Sidenote: There are DEBUG prints in multiple methods (but not in recolorize_main). Some messages can come from multiple methods. I should add message to the r...main method and prefix all messages with an indicator of the source so the control flow is easier to follow
I see two possible approaches to separately colorizing f-expressions within an f-string.
a. Follow the example of 'def' and 'class'. They are recognized as a special case (of builtin) and when they occur, a separate 'if' clause and re is used to colorize the following name.
The problem with doing this with f-strings is that we want to recursively apply the re...main function to a short substring, and the function is not designed for that. We also want to do this separately for each embedded f-expression. It might work to write a reduced version of recolorize_main as recolorize_fexp.
This approach would allow for {} matching once a closing quote is typed, but not identifier autocompletion.
b. Do the special-casing by writing special regexes to recognize a null f-string (no embedded f-expression), and beginning, middle, and ending string parts of an f-string. But I don't know if it is possible to write an re that will *only* match null f-strings.
That aside, the f-expression would then be treated normally, and autocomplete should just work. {} matching would be harder. Without adding new state variables, I imagine that the end quote of the invalid f"a{b" would be seen as the beginning of a new string.
Duoquadragesimal commentedon May 8, 2022
I'm pretty much a beginner and use f-strings, I do think that adding colourisation for them would help to distinguish them in my programs. like
it's a bit dense, but I like doing things in as few lines as possible (maybe not good practice? idk). In this example, in IDLE everything in the curly braces in the f-string shows as green, which makes it a bit hard to like, see what's what when I'm running. I don't know anything about how hard it would be to implement. But it would be a useful feature to me. There is though, I think, the issue of like. How it looks a bit like the string has ended (looking at other editors like vscode). Maybe to get around this, f-strings would change text colour but the background colour would stay as that of a string? Then if you set a different string background you can see that it's all part of a string, but you can also see functions being executed in the braces which I would like.
terryjreedy commentedon Apr 28, 2023
PEP-701 Syntactic formalization of f-strings, which will allow nested f-strings, will be in either 3.12 or later. Internally, there will no longer be an 'f-string'. Instead, the c tokenizer will spit out multiple f-strings pieces, with various labels. Module tokenizer is supposed to be updated to match. If it is, f-strings identified by IDLE could be fed into tokenizer.