Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
52d92c1
fix(codex): stop runtime helpers from leaking past their idle timeout
possibilities Aug 11, 2026
f5bf873
fix(codex): address CodeRabbit round 1 on the helper-leak fix
possibilities Aug 11, 2026
1590cd1
test(codex): pin the sweep-retry test to the four-attempt budget, ord…
possibilities Aug 11, 2026
6bb2049
fix(codex): reap app helpers stranded by the detach grace
possibilities Aug 11, 2026
c18a5df
fix(codex): degrade an unreadable connection count to zero
possibilities Aug 11, 2026
c03af14
fix(codex): correct the connection-count comment and cover the reap o…
possibilities Aug 11, 2026
e57da41
fix(codex): accept only a positive socket count as evidence of a cons…
possibilities Aug 11, 2026
16a335f
test: add owned-PID probes for helper-lifecycle fixtures (#668)
ndycode Aug 13, 2026
bf0a749
fix(codex): reap only helpers that were never handed to a consumer
ndycode Aug 13, 2026
040e0c1
fix(runtime): stop unbind from orphaning helper owner files (#666)
ndycode Aug 13, 2026
7c10722
refactor(runtime): one identity-checked helper selector for both read…
ndycode Aug 13, 2026
109db04
docs: correct the runtime app helper status path in the README
ndycode Aug 13, 2026
51c5ca4
refactor(runtime): narrow the helper PID instead of casting it
ndycode Aug 13, 2026
c42bf54
fix(runtime): address the review on the helper-lifecycle follow-ups
ndycode Aug 13, 2026
e087c43
test: stress the helper lifecycle at the scale the leak report described
ndycode Aug 13, 2026
8f0b578
test: fix the review round on the stress suite
ndycode Aug 13, 2026
6b9f327
test: make the spawn-failure coverage exercise the helpers, not node
ndycode Aug 13, 2026
02e0c9b
test: cover the failed-spawn branch in withLivePid
ndycode Aug 13, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix(codex): reap only helpers that were never handed to a consumer
The detached reap added in #665 could kill a live `codex app` session.

`codex app` relies on the detach grace rather than an explicit
`detachOnExit`, so its launcher exits inside the grace window and the
helper's owner is dead from the first tick. From then on the only thing
standing between the desktop app and a dead proxy was
`countOpenConnections() === 0` — and the proxy never sets
`server.keepAliveTimeout`, so Node closes idle client sockets after its
5s default. A user who stops typing for the length of the detached
window has zero sockets and no new requests, so the helper exits
`owner-gone`, and the next message gets ECONNREFUSED against a dead
localhost port with nothing left to restart it. Pre-#665 that session
survived for the full 12h idle timeout.

Gate the reap on the helper having *never* served a request. Every
leaked helper in the #663 report had `totalRequests: 0`, so the leak is
entirely a never-served phenomenon and the narrower gate closes it in
full; a helper that served anything was genuinely handed off and falls
back to the idle timeout and the 24h lifetime ceiling, which is where it
sat before the detached window existed.

Two more lifecycle fixes in the same tick:

- The owner verdict is now three-valued. "No owner PID was recorded" and
  "the owner is confirmed dead" are different facts, and collapsing them
  into one `false` started the detached clock on the first tick for any
  helper launched without an owner PID — invoked directly, which is the
  documented reproduction in #663, or spawned by a pre-upgrade launcher —
  and reaped it silently 15 minutes later. `unknown` fires neither
  branch, which is what the pre-#664 `ownerPid && isAlive(ownerPid)`
  guard did.

- The status heartbeat now accounts for the detached window.
  `publishToken` zeroes `idleExpiresAt`, so the published deadline only
  catches up on a heartbeat; pinned to the idle window alone, `rotation
  status` kept advertising a 12h deadline for a helper seconds from
  exiting, and under a short DETACHED_IDLE_MS override it never caught up
  at all.

Also in this commit, both from the same review pass:

- The metadata sweep runs after the helper spawn instead of before it.
  It is synchronous and unbounded — readdir, a readFileSync per live
  candidate, rmSync with a blocking backoff, bounded `ps` probes — and
  the state it cleans up is exactly the state that makes it slow, so it
  sat in front of `codex app` and TUI startup. Nothing about spawning
  depends on it. The launch timeout is armed after it either way.

- Sweep deletions are guarded by an mtime re-check. Classifying a file as
  stale and deleting it are two moments, and a PID freed between them can
  be handed to a helper starting right now, which republishes that exact
  path before the delete lands.

- The published wrapper's fault injectors need an explicit
  CODEX_MULTI_AUTH_TEST_FAULT_INJECTION=1 opt-in and a strict digits-only
  parse. `Number.parseInt` reads "2abc" as 2 and "1e3" as 1, so a value
  that was never meant to be a count could arm an injector in a user's
  install and silently defeat the first N metadata deletions of every
  sweep (#668).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015b4Lew3oHNEYmTZtg7zEWz
  • Loading branch information
ndycode and claude committed Aug 13, 2026
commit bf0a749f2a6e8574438f323373ab594114042ce5
2 changes: 1 addition & 1 deletion docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ These are safe for most operators and frequently used in day-to-day workflows.
| `CODEX_MULTI_AUTH_FORCE_ACCOUNT=<index\|email\|id>` | Force one account for a single forwarded `codex-multi-auth-codex` run (equivalent to the `--account` flag, which wins when both are set). Ephemeral and fail-hard; requires the runtime rotation proxy. See [Force an account for one invocation](reference/commands.md#force-an-account-for-one-invocation) |
| `CODEX_MULTI_AUTH_APP_ROTATION_IDLE_MS=<ms>` | Override idle shutdown for the wrapper-launched Codex app helper |
| `CODEX_MULTI_AUTH_APP_ROTATION_MAX_LIFETIME_MS=<ms>` | Absolute ceiling on a runtime helper's life regardless of activity (default 24h; `0` disables) |
| `CODEX_MULTI_AUTH_APP_ROTATION_DETACHED_IDLE_MS=<ms>` | Idle window that applies once a helper's launcher is gone and nothing is connected (default 15m; `0` restores the full idle timeout) |
| `CODEX_MULTI_AUTH_APP_ROTATION_DETACHED_IDLE_MS=<ms>` | Idle window that applies once a helper's launcher is gone, nothing is connected, and the helper has never served a request (default 15m; `0` restores the full idle timeout) |
| `CODEX_MULTI_AUTH_APP_BIND=0/1` | Alias-style opt-out for first-run packaged Codex app bind (see also `CODEX_MULTI_AUTH_APP_BIND_INSTALL`) |
| `CODEX_MULTI_AUTH_APP_BIND_INSTALL=0/1` | Opt out/in of packaged Codex app bind self-heal on first durable CLI run or rotation enable |
| `CODEX_MULTI_AUTH_APP_LAUNCHER_INSTALL=0/1` | Opt out/in of supported user-level launcher routing on first durable CLI run or rotation enable |
Expand Down
2 changes: 1 addition & 1 deletion docs/development/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -205,7 +205,7 @@ Helper self-reaping is identity-checked and bounded. A detached helper decides "
| Recheck cadence | At most once a minute; a `ps` spawn per tick would cost more than it saves. A *failed* re-read keeps the previous verdict rather than declaring a live owner dead — under the process-table pressure this exists for, `fork` itself can fail. |
| Degraded check | Where no start time is known at all, the check degrades to bare liveness. |
| Idle timeout | `CODEX_MULTI_AUTH_APP_ROTATION_IDLE_MS`, default 12h, refreshed by traffic and by a live owner. |
| Detached window | `CODEX_MULTI_AUTH_APP_ROTATION_DETACHED_IDLE_MS`, default 15m, `0` disables. Applies from the moment the owner is confirmed dead. The detach grace hands helpers off optimistically — any launcher exiting cleanly within it leaves its helper running — so every short forwarded command stranded a helper that then held the full idle timeout with no owner, no traffic, and nothing connected. |
| Detached window | `CODEX_MULTI_AUTH_APP_ROTATION_DETACHED_IDLE_MS`, default 15m, `0` disables. Applies from the moment the owner is *confirmed* dead — a helper with no recorded owner PID is not a helper whose owner is dead, and stays on the idle timeout. The detach grace hands helpers off optimistically — any launcher exiting cleanly within it leaves its helper running — so every short forwarded command stranded a helper that then held the full idle timeout with no owner, no traffic, and nothing connected. The window only reaps a helper that has **never served a request**: every leaked helper in #663 had `totalRequests: 0`, while a live `codex app` session that is merely idle between turns holds no socket either (the proxy leaves `keepAliveTimeout` at Node's 5s default), so reaping on the socket check alone would kill a working proxy under the desktop app. A helper that served anything falls back to the idle timeout and the lifetime ceiling. |
| Connection gating | The detached window fires only while the proxy reports zero open client connections, so a consumer who really did take the handoff — `codex app` giving the desktop app its proxy — is never reaped out from under. Traffic after the owner's death pushes the deadline out by another window, so a consumer that reconnects per request survives on its own evidence. An unreadable connection count fails open into reaping: treating "unknown" as "attached" would restore the leak for any shape that stopped answering. |
| Lifetime ceiling | `CODEX_MULTI_AUTH_APP_ROTATION_MAX_LIFETIME_MS`, default 24h, `0` disables. Unconditional on activity — the backstop that turns any future accounting bug into a bounded leak instead of an unbounded one. |
| Telemetry | Per process: each helper publishes `runtime-rotation-app-helper.<pid>.json` (the un-suffixed legacy path is still read for pre-upgrade helpers), publishes only on change plus a heartbeat rather than every tick, removes its owner file on exit, and each launcher sweeps metadata files whose helper PID is dead before spawning the next one — terminal status stamps survive until that sweep, long enough to be read without accumulating forever. |
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated
Expand Down
2 changes: 1 addition & 1 deletion docs/development/CONFIG_FIELDS.md
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,7 @@ Cross-process refresh lease knobs: `CODEX_AUTH_REFRESH_LEASE`, `CODEX_AUTH_REFRE
| `CODEX_MULTI_AUTH_RUNTIME_ROTATION_PROXY` | Toggle localhost Responses proxy for forwarded Codex sessions (`1`/`true` to enable, `0`/`false` to disable) |
| `CODEX_MULTI_AUTH_APP_ROTATION_IDLE_MS` | Override idle timeout for the wrapper-launched Codex app runtime helper |
| `CODEX_MULTI_AUTH_APP_ROTATION_MAX_LIFETIME_MS` | Absolute ceiling on a runtime helper's life regardless of activity (default 24h; `0` disables). The backstop that bounds the leak if activity accounting is ever wrong again |
| `CODEX_MULTI_AUTH_APP_ROTATION_DETACHED_IDLE_MS` | Idle window applied from the moment a helper's launcher is confirmed dead, and only while no client connection is open (default 15m; `0` keeps the full idle timeout). Bounds helpers stranded by the detach grace |
| `CODEX_MULTI_AUTH_APP_ROTATION_DETACHED_IDLE_MS` | Idle window applied from the moment a helper's launcher is confirmed dead, and only while no client connection is open and the helper has never served a request (default 15m; `0` keeps the full idle timeout). Bounds helpers stranded by the detach grace; a helper that served traffic, or one with no recorded owner PID, stays on the idle timeout |
| `CODEX_MULTI_AUTH_APP_ROTATION_OWNER_PID` | Internal owner PID used by the wrapper-launched app helper |
| `CODEX_MULTI_AUTH_APP_ROTATION_OWNER_START_TIME_MS` | Internal owner process start time (epoch ms) the helper uses to tell its launcher from a later process that recycled the PID |
| `CODEX_MULTI_AUTH_REAL_CODEX_HOME` | Internal original Codex home pointer used by runtime rotation helpers |
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/settings.md
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,7 @@ Common operator overrides (aligned with [../configuration.md](../configuration.m
- `CODEX_MULTI_AUTH_FORCE_ACCOUNT` — force one account for a single forwarded `codex-multi-auth-codex` run (selector: index/email/id); `--account` wins when both are set
- `CODEX_MULTI_AUTH_APP_ROTATION_IDLE_MS`
- `CODEX_MULTI_AUTH_APP_ROTATION_MAX_LIFETIME_MS` — absolute ceiling on a helper's life regardless of activity (default 24h; `0` disables)
- `CODEX_MULTI_AUTH_APP_ROTATION_DETACHED_IDLE_MS` — idle window once the helper's launcher is gone and nothing is connected (default 15m; `0` keeps the full idle timeout)
- `CODEX_MULTI_AUTH_APP_ROTATION_DETACHED_IDLE_MS` — idle window once the helper's launcher is confirmed gone, nothing is connected, and the helper has never served a request (default 15m; `0` keeps the full idle timeout)
- `CODEX_MULTI_AUTH_APP_BIND_INSTALL`
- `CODEX_MULTI_AUTH_APP_LAUNCHER_INSTALL`
- `CODEX_TUI_V2`
Expand Down
Loading