Skip to content
Prev Previous commit
Next Next commit
sqlite: set threading mode to multithreaded
  • Loading branch information
geeksilva97 committed Dec 1, 2025
commit 5bec78ea32dec159a1e77d524c454e7461ec6a3a
1 change: 1 addition & 0 deletions deps/sqlite/sqlite.gyp
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
'SQLITE_ENABLE_RBU',
'SQLITE_ENABLE_RTREE',
'SQLITE_ENABLE_SESSION',
'SQLITE_THREADSAFE=2',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a performance penalty for purely single-thread use cases?

If so, would it make sense to allow setting the threading mode at runtime?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know for sure about any performance penalti. But as discusses in the issue, due to the node.js nature, this will work fine for sync. Better-sqlite uses it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood. Thanks! This can be closed.

If anything I think this will speed up single-threaded use cases, because apparently by default SQLite uses internal serialization. Did not test it though.

@mceachen mceachen Feb 12, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I was going to post this on the main thread, but this is the context that matters, so I'll add it here) (edit: I rewrote for brevity):

Synchronous node:sqlite operations can run while your new async operations are executing. This will cause concurrent access to the same SQLite connection from multiple threads. This violates SQLITE_THREADSAFE=2 requirements and can cause db corruption undefined behavior (I could have sworn I read about this on the how-to-corrupt page but I don't see it now.)

SQLITE_THREADSAFE=2 requires no connection be used by multiple threads simultaneously (docs). Currently, nothing prevents sync methods (main thread) from running while async operations are on the thread pool.

Suggested fix: throw from sync methods when has_running_task_ is true:

diff --git a/src/node_sqlite.cc b/src/node_sqlite.cc
--- a/src/node_sqlite.cc
+++ b/src/node_sqlite.cc
@@ -1257,6 +1257,8 @@ void Database::Prepare(const FunctionCallbackInfo<Value>& args) {
   ASSIGN_OR_RETURN_UNWRAP(&db, args.This());
   Environment* env = Environment::GetCurrent(args);
   THROW_AND_RETURN_ON_BAD_STATE(env, !db->IsOpen(), "database is not open");
+  THROW_AND_RETURN_ON_BAD_STATE(env, db->has_running_task_,
+    "Cannot call sync methods while async operation is running");

Same check needed in Exec() (sync path), Function(), Aggregate(), SetAuthorizer(), and LoadExtension(). Tested locally — works as expected.

(SQLITE_THREADSAFE=1 would also prevent undefined behavior, but the main thread would silently block on SQLite's internal mutex until the async op completes, defeating the purpose of async.)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the in-depth examples. I'd like to mention another option (which I thought has been discussed in the issue, but maybe we were talking past each other):
Have one sqlite connection for synchronous access and maintain a pool of (seperate) sqlite3 db connections for asynchronous access. For each asynchronous access one connection from the pool would be exclusively used. The main difficulty with this approach is obviously db.prepare(). One would have to track which db connection owns the prepared statement instance and only dispatch operations from that prepared statement object to its owning db connection. Another drawback of that design is the inherent potential of SQLITE_BUSY failures for concurrent write access.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, if you don’t want to cross the streams, you need multiple database instances opened. Rather than transparently managing this magically begins the scenes in a pool, though, I think it may be less surprising to actually push the complexity of managing per -database handles to the user—they can figure out what makes sense in their app, and when one query wedges another, it’s their fault.

],
'include_dirs': ['.'],
'sources': [
Expand Down
1 change: 1 addition & 0 deletions deps/sqlite/unofficial.gni
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ template("sqlite_gn_build") {
"SQLITE_ENABLE_RBU",
"SQLITE_ENABLE_RTREE",
"SQLITE_ENABLE_SESSION",
"SQLITE_THREADSAFE=2",
]
}

Expand Down
22 changes: 21 additions & 1 deletion src/node_sqlite.cc
Original file line number Diff line number Diff line change
Expand Up @@ -474,6 +474,7 @@ class SQLiteAsyncTask : public ThreadPoolWork {
after_(result_, resolver);
Finalize();
}
db_->ProcessNextAsyncTask();
}

void Finalize() { db_->RemoveAsyncTask(this); }
Expand Down Expand Up @@ -745,6 +746,25 @@ void Database::RemoveAsyncTask(ThreadPoolWork* async_task) {
async_tasks_.erase(async_task);
}

void Database::ScheduleAsyncTask(ThreadPoolWork* work) {
if (has_running_task_) {
task_queue_.push(work);
} else {
has_running_task_ = true;
work->ScheduleWork();
}
}

void Database::ProcessNextAsyncTask() {
if (task_queue_.empty()) {
has_running_task_ = false;
} else {
ThreadPoolWork* next_work = task_queue_.front();
task_queue_.pop();
next_work->ScheduleWork();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned before, setting the thread safety build flag to 1/serializable is pretty much guaranteed to be at least as efficient as building our own scheduling mechanism, so I'd remove this code and instead do that

}
}

void Database::DeleteSessions() {
// all attached sessions need to be deleted before the database is closed
// https://www.sqlite.org/session/sqlite3session_create.html
Expand Down Expand Up @@ -1234,8 +1254,8 @@ Local<Promise::Resolver> MakeSQLiteAsyncWork(
}

auto* work = new SQLiteAsyncTask<T>(env, db, resolver, task, after);
work->ScheduleWork();
db->AddAsyncTask(work);
db->ScheduleAsyncTask(work);
return resolver;
}

Expand Down
5 changes: 5 additions & 0 deletions src/node_sqlite.h
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@

#include <list>
#include <map>
#include <queue>
#include <unordered_set>

namespace node {
Expand Down Expand Up @@ -194,6 +195,8 @@ class Database : public BaseObject {
void AddBackup(BackupJob* backup);
void AddAsyncTask(ThreadPoolWork* async_task);
void RemoveAsyncTask(ThreadPoolWork* async_task);
void ScheduleAsyncTask(ThreadPoolWork* async_task);
void ProcessNextAsyncTask();
void FinalizeBackups();
void UntrackStatement(Statement* statement);
bool IsOpen();
Expand Down Expand Up @@ -230,6 +233,8 @@ class Database : public BaseObject {
bool ignore_next_sqlite_error_;

std::set<ThreadPoolWork*> async_tasks_;
std::queue<ThreadPoolWork*> task_queue_;
bool has_running_task_ = false;
std::set<BackupJob*> backups_;
std::set<sqlite3_session*> sessions_;
std::unordered_set<Statement*> statements_;
Expand Down
Loading