Skip to content

fix(memory): open AgentDB with our better-sqlite3 so Node 24 does not abort - #3955

Merged
ruvnet merged 5 commits into
mainfrom
fix/node24-better-sqlite3-abort
Oct 10, 2026
Merged

ruvnet merged 5 commits into
mainfrom
fix/node24-better-sqlite3-abort

Conversation

@proffesor-for-testing

@proffesor-for-testing proffesor-for-testing commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

On Node 24 (seen on 24.21.0), memory commands and the AgentDB MCP tools abort part of the time. This affects ruflo memory search --smart, memory_search_unified, memory_store and agentdb_pattern-search. Claude Code sees "Connection closed", and the CLI exits with 134:

#  node[...]: void node::RemoveEnvironmentCleanupHook(v8::Isolate*, CleanupHook, void*) at ../src/api/hooks.cc:142
#  Assertion failed: (env) != nullptr
 3: node::RemoveEnvironmentCleanupHook(...) [node]
 4: Statement::~Statement() [.../node_modules/agentdb/node_modules/better-sqlite3/build/Release/better_sqlite3.node]

On Node 22 the same install and data never abort.

Root cause

  • Node regression. Node 24.19.0 backported "add cleanup hooks to node::ObjectWrap" without its registry follow-up (24.19.0: node::ObjectWrap cleanup hooks backported without the cleanup hook registry, aborts on every 24.x runtime nodejs/node#65446; the backport PR [v24.x backport] src: keep global list of addon-provided cleanup hooks nodejs/node#65943 was closed without merging). The fault is in the header-only ObjectWrap. Any addon compiled against 24.19+ headers calls RemoveEnvironmentCleanupHook() from ~ObjectWrap(). When an allocation-driven GC collects a better-sqlite3 Statement while no Environment is current, the process aborts. better-sqlite3 has a matching report: Intermittent ObjectWrap cleanup abort during Statement GC on Node 24 WiseLibs/better-sqlite3#1515.
  • Why ruflo gets a source-built copy. agentdb 3.0.0-alpha.20 declares better-sqlite3: ^11.8.1, so npm nests 11.10.0 under agentdb/. 11.10.0 has no node-v137 (Node 24) prebuild, so on Node 24 it is compiled from source against the local headers. The resulting binary imports RemoveEnvironmentCleanupHook.
  • Why our own copy is safe. The top-level better-sqlite3 12.11.1, which @claude-flow/memory and @claude-flow/cli depend on, ships a node-v137 prebuild. That prebuild was built in June 2026, before 24.19, and does not import RemoveEnvironmentCleanupHook:
    $ nm -D --undefined-only node_modules/better-sqlite3/build/Release/better_sqlite3.node | grep CleanupHook
                     U _ZN4node25AddEnvironmentCleanupHook...
    $ nm -D --undefined-only node_modules/agentdb/node_modules/better-sqlite3/build/Release/better_sqlite3.node | grep CleanupHook
                     U _ZN4node25AddEnvironmentCleanupHook...
                     U _ZN4node28RemoveEnvironmentCleanupHook...
    
    A CLI run loads both binaries into one process. AgentDB's initializeDatabase() opens its handle through the nested 11.x copy.
  • Why the existing override does not help. The better-sqlite3 override in package.json / ruflo/package.json never reaches users. npm ignores overrides in a dependency's package.json, and that includes a global install. I checked this with an isolated npm install -g --prefix <scratch> ruflo@3.56.2. The published package carries overrides.better-sqlite3 = 12.9.0, but the install still contains:
    12.11.1 ruflo/node_modules/better-sqlite3
    11.10.0 ruflo/node_modules/agentic-flow/node_modules/better-sqlite3
    11.10.0 ruflo/node_modules/agentdb/node_modules/better-sqlite3
    
    The workspace hides this because the pnpm override is honoured there and agentdb resolves 12.9.0. That is why CI does not see it.

Fix

@claude-flow/memory gets a new useHostSqliteDriver(agentdb). ControllerRegistry.initAgentDB() and AgentDBBackend.initialize() call it right after new AgentDB(...).

The call sits inside the withAgentdbLockRecovery(...) closure, right after each new AgentDB(...), so the fresh instance that a stale-lock retry creates is rerouted too.

It replaces that instance's initializeDatabase() so the handle is opened with the better-sqlite3 that @claude-flow/memory resolves (12.x prebuild). It applies the same setup AgentDB's native branch does: journal_mode = WAL and usingWasm = false.

If our driver cannot open the file, AgentDB's own loader runs as before. forceWasm instances are left alone. No public API or schema changes.

One better-sqlite3 copy per process (#3693/#3883 invariant)

cli/src/memory/shared-sqlite.ts gives every CLI-owned handle the same native copy as AgentDB's. Those handles cover graph-edge-writer, the memory-initializer repair paths, backup restore, distillation, tuning and the attestation log.

Before this change it resolved better-sqlite3 from agentdb's location. With AgentDB now opened through memory's copy, that would have left those handles on the nested 11.x copy:

  • hooks post-task would have kept loading both addons, and it aborted 10/10 on Node 24 unpatched.
  • Mixing copies on one file can delete the -wal/-shm sidecars under AgentDB's live handle.

What changes:

  • One source of truth. @claude-flow/memory exports getHostSqliteDriver(), the single source of the constructor. useHostSqliteDriver() opens AgentDB with it and returns whether it replaced the loader.

  • Owner selection. shared-sqlite picks the owner with resolveSqliteOwnerEntry():

    • @claude-flow/memory, when the installed memory ships agentdb-native-driver.js (3.0.4+);
    • otherwise agentdb, as before.

    Either memory version therefore keeps one copy per process.

Verification (real runs, Node 24.21.0 and Node 22.23.3, linux-x64)

The fix was checked against a scratch copy of a real ruflo@3.56.1 install. The rebuilt @claude-flow/memory/dist files (agentdb-native-driver.js, controller-registry.js, agentdb-backend.js, index.js) and the rebuilt CLI dist/src/memory/shared-sqlite.js were dropped into the copy. Apart from the fix, the built controller-registry.js and agentdb-backend.js are byte-identical to the published @claude-flow/memory@3.0.2, except for a trailing newline. The data was a copied .swarm/ with AgentDB-schema tables that reproduces the abort. The nested 11.10.0 copy stayed in place for all runs.

Run Unpatched 3.56.1 With this fix
Node 24 memory search -q "agentic-qe ruflo plugin" --smart -n patterns (fixture A) 5/10 aborted (134) 20/20 exit 0
Node 24, same search (fixture B) 4/5 aborted (earlier run) 10/10 exit 0
Node 24 memory store + memory search --path .swarm/agentdb-memory.db, 5 rounds – 10/10 exit 0, the stored entry is found
Node 24 MCP stdio (mcp start; memory_search_unified, memory_store, agentdb_pattern-search, memory_search_unified) 5/8 sessions SIGABRT 8/8 sessions, every call ok
Node 22 same search (patched Node 22 install) 0/6 aborted 10/10 exit 0
Node 24 hooks post-task -i t1 --success true 10/10 aborted 20/20 exit 0
Node 24 memory store – 10/10 exit 0
Node 24 MCP stdio, re-run after the shared-sqlite change – 8/8 sessions, every call ok
Node 22 hooks post-task – 10/10 exit 0
Node 24 search, re-run after the shared-sqlite change (fixture A / B) – 20/20 and 10/10 exit 0

Re-run after the rebase onto main (3.56.3). I used a fresh scratch install of ruflo@3.56.3, which ships the published @claude-flow/memory@3.0.3 without the driver. Apart from the fix, the published memory 3.0.3 dist is identical to this branch's build of controller-registry.js, agentdb-backend.js, index.js and agentdb-lock-guard.js. I then patched in this branch's built memory dist and CLI shared-sqlite.js. Node 24.21.0, .swarm/ fixture A:

Run 3.56.3 as published With this branch
hooks post-task -i t1 --success true 10/10 aborted 20/20 exit 0
memory search -q "agentic-qe ruflo plugin" --smart -n patterns 9/10 aborted 20/20 exit 0

With this branch, hooks post-task loads only node_modules/better-sqlite3/build/Release/better_sqlite3.node.

Other checks:

  • Search output. With the fix, the output is identical to a non-crashing unpatched run, apart from the cwd path.

  • Loaded binaries. I listed every loaded better_sqlite3.node with process.report.getReport().sharedObjects from a --require preload.

    • Before the fix, two binaries are loaded: node_modules/agentdb/node_modules/better-sqlite3/... and node_modules/better-sqlite3/....
    • After the fix, only node_modules/better-sqlite3/build/Release/better_sqlite3.node is loaded, on both Node 24 and Node 22.
    • hooks post-task before the fix loads agentdb/node_modules/better-sqlite3 and aborts. After it, only the top-level copy is loaded, on Node 24 and Node 22.
  • Negative control. I swapped in a better-sqlite3 12.11.1 binary built from source against 24.21.0 headers. The fixed tree then aborts again in 9/10 runs. So the cause is the build headers, not the 11.x code. This is also the residual risk below.

  • Unit tests (memory). npx vitest run in v3/@claude-flow/memory after the rebase: 607 tests in 34 files; 606 pass. The one failure was benchmark.test.ts "resolve 10k paths" (209 ms against a 200 ms limit) under a full parallel run. That file is untouched, and it passes 3/3 on its own. tsc --noEmit is clean. src/agentdb-native-driver.test.ts has 13 tests:

    • routing to our copy, the fallback, forceWasm, and that getHostSqliteDriver() is this package's better-sqlite3;
    • a real nested copy of the addon under a temp node_modules/agentdb (the control opens with the nested copy; with the helper, the handle comes from getHostSqliteDriver());
    • both construction sites (ControllerRegistry.initAgentDB, AgentDBBackend.initialize) against a fake agentdb, on the first attempt and on the stale-lock retry path (a dead-writer lock plus a first instance that fails with AGENTDB_SQLJS_CONCURRENT_MODIFICATION);
    • a drift guard on agentdb's AgentDB.initialize() → initializeDatabase(dbPath) contract.

    Mutation checks: deleting either call site fails its test. Leaving the retry instance unrerouted fails that site's retry test. Turning the helper into a no-op fails 4 tests.

  • CLI tests. In v3/@claude-flow/cli, after a workspace pnpm -r build, tsc is clean.

Release note

@claude-flow/memory is not bundled into the train. 3.56.3 already published @claude-flow/memory@3.0.3 without this driver. So this PR bumps memory to 3.0.4 and pins the CLI to 3.0.4 in dependencies, optionalDependencies and the v3/pnpm-lock.yaml specifier.

node scripts/audit-leaf-package-publish.mjs --base origin/main reports:

@claude-flow/memory  cli pins 3.0.4  source 3.0.4 ✓
@claude-flow/memory: source changed AND version bumped 3.0.3 -> 3.0.4 ✓

Publish order matters: @claude-flow/memory@3.0.4 must be on npm before, or in the same release as, the CLI that carries this pin.

The CLI lists memory in both dependencies and optionalDependencies, and the optional entry wins. So an unpublished pin would most likely install the CLI without @claude-flow/memory, rather than fail the install, and the fix would silently not reach users.

The signed verification/*/manifest.md.json files are regenerated by the release witness step, not here.

Not covered / residual risk

When users get the fix: this PR leaves the CLI version at 3.56.3. The fix reaches users only with the next CLI release (3.56.4+), published together with or after @claude-flow/memory@3.0.4.

🤖 Generated by Ruflo & AQE

… abort

On Node 24.19+, `memory search`, `memory_store` and the AgentDB MCP tools
abort intermittently with `Assertion failed: (env) != nullptr` in
node::RemoveEnvironmentCleanupHook, from Statement::~Statement() in
agentdb/node_modules/better-sqlite3 (exit 134 / "Connection closed").

agentdb 3.0.0-alpha.x pins better-sqlite3 ^11.8.1. 11.x has no Node 24
prebuild, so npm compiles it against the local Node headers, and 24.19+
headers carry a partial ObjectWrap cleanup-hook backport
(nodejs/node#65446) that aborts when a Statement is collected outside an
Environment. Our own better-sqlite3 12.x ships a Node 24 prebuild built
before that header change. The `better-sqlite3` override in the published
package.json files does not reach agentdb: npm ignores overrides in a
dependency, and `npm i -g ruflo` still installs the nested 11.10.0.

ControllerRegistry and AgentDBBackend now replace the AgentDB instance's
initializeDatabase() so its handle is opened with this package's
better-sqlite3; AgentDB's own loader stays as the fallback and forceWasm
is untouched.
Follow-up to the AgentDB driver change. shared-sqlite.ts (#3693/#3883)
resolved better-sqlite3 from agentdb's location so CLI handles share
AgentDB's native copy. With AgentDB now opened through
@claude-flow/memory's better-sqlite3, that put graph-edge-writer, the
memory-initializer repair paths, backup restore, distillation, tuning and
the attestation log back on agentdb's nested 11.x: `hooks post-task` still
loaded both addons (and aborted on Node 24), and mixing copies on one file
can delete the -wal/-shm sidecars under AgentDB's live handle.

- @claude-flow/memory exports getHostSqliteDriver(), the one source of the
  constructor; useHostSqliteDriver() opens AgentDB with it and reports
  whether it replaced the loader.
- shared-sqlite resolves better-sqlite3 from @claude-flow/memory when the
  installed memory carries agentdb-native-driver.js, else from agentdb as
  before, so either memory version keeps one copy per process.
- @claude-flow/memory 3.0.3, CLI pin 3.0.3 (leaf-publish gate).

Tests: identity between AgentDB's handle constructor and
loadBetterSqlite3(); #3693 owner model for the memory holder and the
pre-3.0.3 fallback; both AgentDB construction sites; routing against a
real nested copy; a drift guard on agentdb's initializeDatabase contract.
…flow/memory

The root CI job resolves the published @claude-flow/memory 3.0.2, which does
not route AgentDB through its own driver; there the owner must stay agentdb.
@proffesor-for-testing
proffesor-for-testing force-pushed the fix/node24-better-sqlite3-abort branch from d413b8d to 613ea1e Compare October 10, 2026 07:49
Rebased onto main, where ControllerRegistry and AgentDBBackend build AgentDB
inside withAgentdbLockRecovery(); useHostSqliteDriver() now runs inside that
closure right after each `new AgentDB`, so the instance a stale-lock retry
creates also opens its handle with getHostSqliteDriver(). Tests drive the
retry path for both sites.

@claude-flow/memory 3.0.3 was published by 3.56.3 without the driver, so
this ships as 3.0.4 and the CLI pins 3.0.4. The #3693 memory-owner test
now reports a skip when no native addon is available instead of passing.
@proffesor-for-testing
proffesor-for-testing force-pushed the fix/node24-better-sqlite3-abort branch from 613ea1e to c536e77 Compare October 10, 2026 07:49
vitest applies queued doMock/doUnmock entries for one path in resolve order,
so an afterEach doUnmock could win over the next test's doMock and the
stale-lock retry test saw the real agentdb (CI: 0 fake instances).
@proffesor-for-testing
proffesor-for-testing marked this pull request as ready for review October 10, 2026 08:22
@ruvnet
ruvnet merged commit c75a1ed into main Oct 10, 2026
122 checks passed
@ruvnet

ruvnet commented Oct 10, 2026

Copy link
Copy Markdown
Owner

Reviewed and merged. Verified: on Node 24.21.0, better-sqlite3 11.10.0 (the copy npm nests under agentdb) aborts with the RemoveEnvironmentCleanupHook assertion (exit 134) and imports that symbol, whereas 12.11.1 does not and survives the same loop. Memory suite 607/607, cli full suite unchanged from main's known failure set, tsc clean, pnpm-lock edit minimal. Non-blocking note: if db.pragma('journal_mode = WAL') throws in useHostSqliteDriver, the freshly opened handle is not closed and AgentDB's own fallback is skipped; worth a small follow-up (close the handle and fall back to agentdbLoader). Release note: this bumps @claude-flow/memory to 3.0.4 and the CLI pins it exactly, so memory 3.0.4 has to be published before the CLI. It ships in the next release; nothing is published by this merge. Thank you @proffesor-for-testing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants