Skip to content

bpo-27645: Supporting native backup facility of SQLite #377

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
lelit wants to merge 21 commits into from
Closed

bpo-27645: Supporting native backup facility of SQLite #377

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
d93391d
Expose the SQLite Online Backup API as Connection.backup()
lelit Jul 28, 2016
7b1277a
Preliminary documentation for sqlite3.Connection.backup()
lelit Jul 28, 2016
1258d47
Reduce code nesting depth by using a finally label, as other function…
lelit Mar 1, 2017
d8ea457
Wrap SQLite calls between BEGIN_ALLOW_THREADS and END_ALLOW_THREADS
lelit Mar 1, 2017
35fa0ad
Add an entry in Misc/NEWS related to issue #27645
lelit Mar 1, 2017
c1af5a0
Remove extraneous entry introduced in d4439d6c5b2a67682f3db74d395c603…
lelit Mar 1, 2017
c07d982
Attribute the patch to myself
lelit Mar 1, 2017
cdc7eda
Make pages and progress keyword-only args in sqlite3.Connection.backup()
lelit Mar 1, 2017
6d3eb69
Remove reference to SQLite docs in Connection.backup() documentation
lelit Mar 1, 2017
6fab17f
Use f-string in the sqlite3.Connection.backup() example
lelit Mar 1, 2017
6449f3d
Explicitly mark sqlite3.Connection.backup() as added in v3.7
lelit Mar 1, 2017
f84d375
Reduce chances of cluttering future “git blame”
lelit Mar 1, 2017
e748942
Parametrize the name of the database involved in the backup
lelit Mar 3, 2017
426dad4
Test propagation of exception raised in backup's progress callback
lelit Mar 3, 2017
895726b
Use a better name for variable
lelit Mar 3, 2017
77dfa39
Do not delay next iteration if the result was OK
lelit Mar 3, 2017
71726ba
Omit the backup method when underlying SQLite library is older than 3…
lelit Mar 3, 2017
c4fbc20
Assert that the non-mandatory arguments are keyword-only
lelit Mar 3, 2017
ff1609b
Pass also the current status of the ongoing backup to the progress ca…
lelit Mar 4, 2017
d688f20
Slightly different way handling backup step's error state
lelit Mar 4, 2017
4ed6c31
When an error occurs while the backup is going on, remove the target …
lelit Mar 4, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Make pages and progress keyword-only args in sqlite3.Connection.backup()
  • Loading branch information
@lelit
lelit committed Mar 1, 2017
commit cdc7edad9d004c000f734ccdc865025988c58a9f
2 changes: 1 addition & 1 deletion Doc/library/sqlite3.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -521,7 +521,7 @@ Connection Objects
f.write('%s\n' % line)


.. method:: backup(filename[, pages, progress])
.. method:: backup(filename, *, pages=0, progress=None)

This method exposes the `API`__ that allows to make a backup of a SQLite

@berkerpeksag berkerpeksag Mar 1, 2017

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to add a link to SQLite docs.

@lelit lelit Mar 1, 2017

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok.

database into the mandatory argument *filename*, even while it's being accessed
Expand Down
10 changes: 5 additions & 5 deletions Lib/sqlite3/test/backup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ def progress(remaining, total):
journal.append(remaining)

with NamedTemporaryFile(suffix='.sqlite') as bckfn:
self.cx.backup(bckfn.name, 1, progress)
self.cx.backup(bckfn.name, pages=1, progress=progress)
self.testBackup(bckfn.name)

self.assertEqual(len(journal), 2)
Expand All @@ -44,7 +44,7 @@ def progress(remaining, total):
journal.append(remaining)

with NamedTemporaryFile(suffix='.sqlite') as bckfn:
self.cx.backup(bckfn.name, 0, progress)
self.cx.backup(bckfn.name, progress=progress)
self.testBackup(bckfn.name)

self.assertEqual(len(journal), 1)
Expand All @@ -57,7 +57,7 @@ def progress(remaining, total):
journal.append(remaining)

with NamedTemporaryFile(suffix='.sqlite') as bckfn:
self.cx.backup(bckfn.name, -1, progress)
self.cx.backup(bckfn.name, pages=-1, progress=progress)
self.testBackup(bckfn.name)

self.assertEqual(len(journal), 1)
Expand All @@ -66,7 +66,7 @@ def progress(remaining, total):
def CheckNonCallableProgress(self):
with NamedTemporaryFile(suffix='.sqlite') as bckfn:
with self.assertRaises(TypeError) as cm:
self.cx.backup(bckfn.name, 1, 'bar')
self.cx.backup(bckfn.name, pages=1, progress='bar')
self.assertEqual(str(cm.exception), 'progress argument must be a callable')

def CheckModifyingProgress(self):
Expand All @@ -79,7 +79,7 @@ def progress(remaining, total):
journal.append(remaining)

with NamedTemporaryFile(suffix='.sqlite') as bckfn:
self.cx.backup(bckfn.name, 1, progress)
self.cx.backup(bckfn.name, pages=1, progress=progress)
self.testBackup(bckfn.name)

cx = sqlite.connect(bckfn.name)
Expand Down
11 changes: 6 additions & 5 deletions Modules/_sqlite/connection.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1478,7 +1478,7 @@ pysqlite_connection_iterdump(pysqlite_Connection* self, PyObject* args)
}

static PyObject *
pysqlite_connection_backup(pysqlite_Connection* self, PyObject* args)
pysqlite_connection_backup(pysqlite_Connection* self, PyObject* args, PyObject* kwds)

@berkerpeksag berkerpeksag Mar 3, 2017

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I read SQLite release notes correctly, sqlite3_backup_init has been added in 3.6.11 so we need to expose this only if SQLite version is 3.6.11 or newer.

@lelit lelit Mar 3, 2017

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, is it better to omit the whole function or rather raise an NotImplementedError?

@berkerpeksag berkerpeksag Mar 3, 2017 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's omit the whole function. This is what we do for other version specific features in the stdlib. Please also update the documentation to note that this feature is available from SQLite 3.6.11.

@lelit lelit Mar 3, 2017

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, done. Thank you!

{
char* filename;
int pages = -1;
Expand All @@ -1487,9 +1487,10 @@ pysqlite_connection_backup(pysqlite_Connection* self, PyObject* args)
int rc;
sqlite3 *bckconn;
sqlite3_backup *bckhandle;
static char *keywords[] = {"filename", "pages", "progress", NULL};

if (!PyArg_ParseTuple(args, "s|iO:backup(filename, pages, progress)",
&filename, &pages, &progress)) {
if (!PyArg_ParseTupleAndKeywords(args, kwds, "s|$iO:backup", keywords,
&filename, &pages, &progress)) {
goto finally;
}

Expand Down Expand Up @@ -1734,8 +1735,8 @@ static PyMethodDef connection_methods[] = {
PyDoc_STR("Abort any pending database operation. Non-standard.")},
{"iterdump", (PyCFunction)pysqlite_connection_iterdump, METH_NOARGS,
PyDoc_STR("Returns iterator to the dump of the database in an SQL text format. Non-standard.")},
{"backup", (PyCFunction)pysqlite_connection_backup, METH_VARARGS,
PyDoc_STR("Execute a backup of the database. Non-standard.")},
{"backup", (PyCFunction)pysqlite_connection_backup, METH_VARARGS | METH_KEYWORDS,
PyDoc_STR("Makes a backup of the database. Non-standard.")},
{"__enter__", (PyCFunction)pysqlite_connection_enter, METH_NOARGS,
PyDoc_STR("For context manager. Non-standard.")},
{"__exit__", (PyCFunction)pysqlite_connection_exit, METH_VARARGS,
Expand Down