Skip to content

gh-108590: Improve sqlite3 docs on encoding issues and how to handle those #108699

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
erlend-aasland merged 18 commits into from
Oct 25, 2023
Merged

gh-108590: Improve sqlite3 docs on encoding issues and how to handle those #108699

Changes from 1 commit
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
139ad73
gh-108590: Add sqlite3 text factory howto
erlend-aasland Aug 30, 2023
88a7599
Apply suggestions from code review
erlend-aasland Aug 30, 2023
f9aac63
Add note to iterdump(); explain why we use CAST
erlend-aasland Aug 31, 2023
60aebce
Merge Alex's review
erlend-aasland Aug 31, 2023
1fead55
Pull in main
erlend-aasland Aug 31, 2023
fdc240f
Mention that execute() and friends only accept UTF-8 encoded strings
erlend-aasland Aug 31, 2023
62073e5
Try to emphasize that strings with surrogate escapes cannot be passed…
erlend-aasland Aug 31, 2023
35c6e9d
Add seealso for the Unicode HOWTO
erlend-aasland Aug 31, 2023
d136048
Pull in main
erlend-aasland Sep 1, 2023
3b9f620
Try to address reviews:
erlend-aasland Sep 4, 2023
6487fa6
Pull in main
erlend-aasland Sep 4, 2023
57f1a57
Fix note
erlend-aasland Sep 4, 2023
c4cbe84
Address review: don't need to show how to create the database
erlend-aasland Sep 4, 2023
9b78bf2
Remove execute*() changes
erlend-aasland Sep 4, 2023
a91fa7c
Update Doc/library/sqlite3.rst
erlend-aasland Sep 7, 2023
3411791
Update Doc/library/sqlite3.rst
erlend-aasland Sep 7, 2023
dc4c820
Apply suggestions from CAM and Ezio
erlend-aasland Oct 12, 2023
7e0e615
Keep it simple
erlend-aasland Oct 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Try to emphasize that strings with surrogate escapes cannot be passed… 
… to sqlite3 API
  • Loading branch information
@erlend-aasland
erlend-aasland committed Aug 31, 2023
commit 62073e5364c49d6acb85fe6e7dd0d95b22173afd
7 changes: 5 additions & 2 deletions Doc/library/sqlite3.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2640,8 +2640,11 @@ The loop above will print the offending line using Unicode surrogate escapes:
INSERT INTO "test" VALUES('a\udc9f');
COMMIT;

Note that in order to write the invalid Unicode sequence to a file,
you will also have to use ``errors="surrogateescape"`` as an argument to :func:`open`:
Note that strings containing surrogate escapes must be treated with care.
You cannot pass them back to SQLite, for example using :meth:`~Cursor.execute`,
since the :mod:`!sqlite3` module APIs only accept UTF-8 encoded strings.
In order to write strings containing surrogate escapes to a file,
you will have to use ``errors="surrogateescape"`` as an argument to :func:`open`:

.. testcode::

Expand Down