Repository navigation
CSV reader should support QUOTE_NOTNULL and QUOTE_STRINGS #113732
Description
Feature or enhancement
New quoting rules QUOTE_NOTNULL and QUOTE_STRINGS were introduced in #67230. But they only affect CSV writer, not CSV reader. I think that they should affect CSV reader in the same way as QUOTE_NONNUMERIC does.
- QUOTE_NOTNULL -- unquoted empty strings are returned as None.
- QUOTE_STRINGS -- unquoted empty strings are returned as None and unquoted numeric strings are returned as float.
It is perhaps too late to change this in 3.12, so it can be considered as a new feature in 3.13.
Activity
serhiy-storchaka commentedon Jan 5, 2024
On other hand, it is the documented behavior. So this can be considered as a bugfix.
pythongh-113732: Fix support of QUOTE_NOTNULL and QUOTE_STRINGS in cs…
pythongh-113732: Fix support of QUOTE_NOTNULL and QUOTE_STRINGS in cs…
merwok commentedon Jan 5, 2024
serhiy-storchaka commentedon Jan 5, 2024
I think that it is less disruptive than alternatives. For now, not much people use 3.12, and even less use new quoting options. But the mass adaptation of 3.12 will start before releasing 3.13, and at that time more code can be broken by 3.13 changes.
It was better to do this before 3.12.0, but it was not done, and now we have no good options. The longer we delay the solution, the worse options we have. cc @Yhg1s
merwok commentedon Jan 5, 2024
Ah, I missed that the options were introduced in 3.12! Option 1 to fix in 3.12.2 seems great then. Great that you tagged the release manager to confirm.
[-]CVS reader should support QUOTE_NOTNULL and QUOTE_STRINGS[/-][+]CSV reader should support QUOTE_NOTNULL and QUOTE_STRINGS[/+]Yhg1s commentedon Jan 16, 2024
We should consider observable behaviour, rather than documentation, to be leading. Changing the behaviour in 3.12.2 is too disruptive. Patch releases also should not add new warnings. Given that it was documented to behave this way, I don't think it's necessary to add a new warning for this. That it doesn't work as documented is a bug we can't fix in 3.12, but we can in 3.13.
gh-113732: Fix support of QUOTE_NOTNULL and QUOTE_STRINGS in csv.read…
merwok commentedon Jan 30, 2024
Would it be useful to add a note in the 3.12 docs?
pythongh-113732: Fix support of QUOTE_NOTNULL and QUOTE_STRINGS in cs…
Prometheus3375 commentedon Mar 11, 2024
Adding a note to the documentation will be useful indeed. Was just writing some .csv involved code and because of the documentation of new quoting constants increased python version. Then I realized that these constants does not work for reading as documentation suggests. Hopefully, I found this issue.
Actually, instead of the note of bugged behavior maybe it will be better to simply remove reader instruction in 3.12 documentation and then add in 3.13 as an addition.
Also, one of the
Noneis not properly formatted.merwok commentedon Mar 12, 2024
I don’t think so: it is common to read a newer version of the docs than the Python version you’re using. It’s best to be clear about the behaviour and the change here.
Does someone want to open a pull request for this?
If not, we should open a doc ticket to make sure this doesn’t get forgotten.