Repository navigation
fix(memory): open AgentDB with our better-sqlite3 so Node 24 does not abort - #3955
Conversation
… abort On Node 24.19+, `memory search`, `memory_store` and the AgentDB MCP tools abort intermittently with `Assertion failed: (env) != nullptr` in node::RemoveEnvironmentCleanupHook, from Statement::~Statement() in agentdb/node_modules/better-sqlite3 (exit 134 / "Connection closed"). agentdb 3.0.0-alpha.x pins better-sqlite3 ^11.8.1. 11.x has no Node 24 prebuild, so npm compiles it against the local Node headers, and 24.19+ headers carry a partial ObjectWrap cleanup-hook backport (nodejs/node#65446) that aborts when a Statement is collected outside an Environment. Our own better-sqlite3 12.x ships a Node 24 prebuild built before that header change. The `better-sqlite3` override in the published package.json files does not reach agentdb: npm ignores overrides in a dependency, and `npm i -g ruflo` still installs the nested 11.10.0. ControllerRegistry and AgentDBBackend now replace the AgentDB instance's initializeDatabase() so its handle is opened with this package's better-sqlite3; AgentDB's own loader stays as the fallback and forceWasm is untouched.
Follow-up to the AgentDB driver change. shared-sqlite.ts (#3693/#3883) resolved better-sqlite3 from agentdb's location so CLI handles share AgentDB's native copy. With AgentDB now opened through @claude-flow/memory's better-sqlite3, that put graph-edge-writer, the memory-initializer repair paths, backup restore, distillation, tuning and the attestation log back on agentdb's nested 11.x: `hooks post-task` still loaded both addons (and aborted on Node 24), and mixing copies on one file can delete the -wal/-shm sidecars under AgentDB's live handle. - @claude-flow/memory exports getHostSqliteDriver(), the one source of the constructor; useHostSqliteDriver() opens AgentDB with it and reports whether it replaced the loader. - shared-sqlite resolves better-sqlite3 from @claude-flow/memory when the installed memory carries agentdb-native-driver.js, else from agentdb as before, so either memory version keeps one copy per process. - @claude-flow/memory 3.0.3, CLI pin 3.0.3 (leaf-publish gate). Tests: identity between AgentDB's handle constructor and loadBetterSqlite3(); #3693 owner model for the memory holder and the pre-3.0.3 fallback; both AgentDB construction sites; routing against a real nested copy; a drift guard on agentdb's initializeDatabase contract.
…flow/memory The root CI job resolves the published @claude-flow/memory 3.0.2, which does not route AgentDB through its own driver; there the owner must stay agentdb.
d413b8d to
613ea1e
Compare
Rebased onto main, where ControllerRegistry and AgentDBBackend build AgentDB inside withAgentdbLockRecovery(); useHostSqliteDriver() now runs inside that closure right after each `new AgentDB`, so the instance a stale-lock retry creates also opens its handle with getHostSqliteDriver(). Tests drive the retry path for both sites. @claude-flow/memory 3.0.3 was published by 3.56.3 without the driver, so this ships as 3.0.4 and the CLI pins 3.0.4. The #3693 memory-owner test now reports a skip when no native addon is available instead of passing.
613ea1e to
c536e77
Compare
vitest applies queued doMock/doUnmock entries for one path in resolve order, so an afterEach doUnmock could win over the next test's doMock and the stale-lock retry test saw the real agentdb (CI: 0 fake instances).
|
Reviewed and merged. Verified: on Node 24.21.0, better-sqlite3 11.10.0 (the copy npm nests under agentdb) aborts with the |
Problem
On Node 24 (seen on 24.21.0), memory commands and the AgentDB MCP tools abort part of the time. This affects
ruflo memory search --smart,memory_search_unified,memory_storeandagentdb_pattern-search. Claude Code sees "Connection closed", and the CLI exits with 134:On Node 22 the same install and data never abort.
Root cause
node::ObjectWrap" without its registry follow-up (24.19.0:node::ObjectWrapcleanup hooks backported without the cleanup hook registry, aborts on every 24.x runtime nodejs/node#65446; the backport PR [v24.x backport] src: keep global list of addon-provided cleanup hooks nodejs/node#65943 was closed without merging). The fault is in the header-onlyObjectWrap. Any addon compiled against 24.19+ headers callsRemoveEnvironmentCleanupHook()from~ObjectWrap(). When an allocation-driven GC collects a better-sqlite3Statementwhile no Environment is current, the process aborts. better-sqlite3 has a matching report: Intermittent ObjectWrap cleanup abort during Statement GC on Node 24 WiseLibs/better-sqlite3#1515.better-sqlite3: ^11.8.1, so npm nests 11.10.0 underagentdb/. 11.10.0 has nonode-v137(Node 24) prebuild, so on Node 24 it is compiled from source against the local headers. The resulting binary importsRemoveEnvironmentCleanupHook.@claude-flow/memoryand@claude-flow/clidepend on, ships anode-v137prebuild. That prebuild was built in June 2026, before 24.19, and does not importRemoveEnvironmentCleanupHook:initializeDatabase()opens its handle through the nested 11.x copy.better-sqlite3override inpackage.json/ruflo/package.jsonnever reaches users. npm ignoresoverridesin a dependency's package.json, and that includes a global install. I checked this with an isolatednpm install -g --prefix <scratch> ruflo@3.56.2. The published package carriesoverrides.better-sqlite3 = 12.9.0, but the install still contains:Fix
@claude-flow/memorygets a newuseHostSqliteDriver(agentdb).ControllerRegistry.initAgentDB()andAgentDBBackend.initialize()call it right afternew AgentDB(...).The call sits inside the
withAgentdbLockRecovery(...)closure, right after eachnew AgentDB(...), so the fresh instance that a stale-lock retry creates is rerouted too.It replaces that instance's
initializeDatabase()so the handle is opened with the better-sqlite3 that@claude-flow/memoryresolves (12.x prebuild). It applies the same setup AgentDB's native branch does:journal_mode = WALandusingWasm = false.If our driver cannot open the file, AgentDB's own loader runs as before.
forceWasminstances are left alone. No public API or schema changes.One better-sqlite3 copy per process (#3693/#3883 invariant)
cli/src/memory/shared-sqlite.tsgives every CLI-owned handle the same native copy as AgentDB's. Those handles cover graph-edge-writer, the memory-initializer repair paths, backup restore, distillation, tuning and the attestation log.Before this change it resolved better-sqlite3 from agentdb's location. With AgentDB now opened through memory's copy, that would have left those handles on the nested 11.x copy:
hooks post-taskwould have kept loading both addons, and it aborted 10/10 on Node 24 unpatched.-wal/-shmsidecars under AgentDB's live handle.What changes:
One source of truth.
@claude-flow/memoryexportsgetHostSqliteDriver(), the single source of the constructor.useHostSqliteDriver()opens AgentDB with it and returns whether it replaced the loader.Owner selection.
shared-sqlitepicks the owner withresolveSqliteOwnerEntry():@claude-flow/memory, when the installed memory shipsagentdb-native-driver.js(3.0.4+);agentdb, as before.Either memory version therefore keeps one copy per process.
Verification (real runs, Node 24.21.0 and Node 22.23.3, linux-x64)
The fix was checked against a scratch copy of a real
ruflo@3.56.1install. The rebuilt@claude-flow/memory/distfiles (agentdb-native-driver.js,controller-registry.js,agentdb-backend.js,index.js) and the rebuilt CLIdist/src/memory/shared-sqlite.jswere dropped into the copy. Apart from the fix, the builtcontroller-registry.jsandagentdb-backend.jsare byte-identical to the published@claude-flow/memory@3.0.2, except for a trailing newline. The data was a copied.swarm/with AgentDB-schema tables that reproduces the abort. The nested 11.10.0 copy stayed in place for all runs.memory search -q "agentic-qe ruflo plugin" --smart -n patterns(fixture A)memory store+memory search --path .swarm/agentdb-memory.db, 5 roundsmcp start;memory_search_unified,memory_store,agentdb_pattern-search,memory_search_unified)hooks post-task -i t1 --success truememory storehooks post-taskRe-run after the rebase onto main (3.56.3). I used a fresh scratch install of
ruflo@3.56.3, which ships the published@claude-flow/memory@3.0.3without the driver. Apart from the fix, the published memory 3.0.3 dist is identical to this branch's build ofcontroller-registry.js,agentdb-backend.js,index.jsandagentdb-lock-guard.js. I then patched in this branch's built memory dist and CLIshared-sqlite.js. Node 24.21.0,.swarm/fixture A:hooks post-task -i t1 --success truememory search -q "agentic-qe ruflo plugin" --smart -n patternsWith this branch,
hooks post-taskloads onlynode_modules/better-sqlite3/build/Release/better_sqlite3.node.Other checks:
Search output. With the fix, the output is identical to a non-crashing unpatched run, apart from the cwd path.
Loaded binaries. I listed every loaded
better_sqlite3.nodewithprocess.report.getReport().sharedObjectsfrom a--requirepreload.node_modules/agentdb/node_modules/better-sqlite3/...andnode_modules/better-sqlite3/....node_modules/better-sqlite3/build/Release/better_sqlite3.nodeis loaded, on both Node 24 and Node 22.hooks post-taskbefore the fix loadsagentdb/node_modules/better-sqlite3and aborts. After it, only the top-level copy is loaded, on Node 24 and Node 22.Negative control. I swapped in a better-sqlite3 12.11.1 binary built from source against 24.21.0 headers. The fixed tree then aborts again in 9/10 runs. So the cause is the build headers, not the 11.x code. This is also the residual risk below.
Unit tests (memory).
npx vitest runinv3/@claude-flow/memoryafter the rebase: 607 tests in 34 files; 606 pass. The one failure wasbenchmark.test.ts"resolve 10k paths" (209 ms against a 200 ms limit) under a full parallel run. That file is untouched, and it passes 3/3 on its own.tsc --noEmitis clean.src/agentdb-native-driver.test.tshas 13 tests:forceWasm, and thatgetHostSqliteDriver()is this package's better-sqlite3;node_modules/agentdb(the control opens with the nested copy; with the helper, the handle comes fromgetHostSqliteDriver());ControllerRegistry.initAgentDB,AgentDBBackend.initialize) against a fake agentdb, on the first attempt and on the stale-lock retry path (a dead-writer lock plus a first instance that fails withAGENTDB_SQLJS_CONCURRENT_MODIFICATION);AgentDB.initialize()→initializeDatabase(dbPath)contract.Mutation checks: deleting either call site fails its test. Leaving the retry instance unrerouted fails that site's retry test. Turning the helper into a no-op fails 4 tests.
CLI tests. In
v3/@claude-flow/cli, after a workspacepnpm -r build, tsc is clean.__tests__/shared-sqlite-host-driver.test.ts: the owner is the installed@claude-flow/memory, andloadBetterSqlite3()is bothgetHostSqliteDriver()and the constructor of AgentDB's handle.graph-writer-shared-sqlite-3693.test.tsnow also models the memory-owned holder with its own native copy, plus the fallback to agentdb when memory has no driver.shared-sqlite.ts, 2 of these new tests fail.Release note
@claude-flow/memoryis not bundled into the train. 3.56.3 already published@claude-flow/memory@3.0.3without this driver. So this PR bumps memory to 3.0.4 and pins the CLI to3.0.4independencies,optionalDependenciesand thev3/pnpm-lock.yamlspecifier.node scripts/audit-leaf-package-publish.mjs --base origin/mainreports:Publish order matters:
@claude-flow/memory@3.0.4must be on npm before, or in the same release as, the CLI that carries this pin.The CLI lists memory in both
dependenciesandoptionalDependencies, and the optional entry wins. So an unpublished pin would most likely install the CLI without@claude-flow/memory, rather than fail the install, and the fix would silently not reach users.The signed
verification/*/manifest.md.jsonfiles are regenerated by the release witness step, not here.Not covered / residual risk
--build-from-source,npm rebuild. better-sqlite3's 12.x line ended at 12.11.1 (June 2026), so^12.9.0keeps getting the pre-regression prebuild where one exists. Bumping to 13.x needs this checked again. The real fix is 24.19.0:node::ObjectWrapcleanup hooks backported without the cleanup hook registry, aborts on every 24.x runtime nodejs/node#65446.^11.8.1. Raising it to^12.8.0in ruvnet/agentdb would remove the nested copy for every consumer. Companion draft PR: fix: require better-sqlite3 ^12.8.0 so Node 24 does not abort on statement GC agentdb#29.agentic-flow/reasoningbankandagentic-flow/routerloaded nobetter_sqlite3.node. Any path that does load it is still exposed.ControllerRegistry/AgentDBBackendare routed. AgentDB's internaldb-fallback.jsloader, which is used on the WASM fallback path, is unchanged.scripts/audit-better-sqlite3-override.mjs(agentdb pins better-sqlite3@^11.8.1, which has no Node 24/26 prebuilt and fails to compile — silent fallback to sql.js WASM loses data #2219) asserts an override that, as shown above, does not apply to installed consumers. It is left unchanged here.🤖 Generated by Ruflo & AQE