fix(core): release per-run process listeners and task history results - #36974
Merged
Merged
Conversation
…#36866) ## Current Behavior Every `runDiscreteTasks` call registers SIGINT/SIGTERM/SIGHUP handlers (plus the forked runner's message/exit handlers) that are never removed. Each closes over its orchestrator, so a long-lived process like an Nx Cloud agent retains every orchestrator it ever created: the task hasher (with its native project-graph copy), the lifecycles, and every task result with full terminal output. `TaskHistoryLifeCycle.pendingResults` also keeps every `TaskResult` for the life of the process. A profiled customer agent grew from 237MB to 5.5GB over 50 tasks. ## Expected Behavior Process listeners are removed once a run completes, task history stops retaining terminal output, and agent memory stays flat across assignment batches. ## Backport notes Three places where 22.7.x differs from the branch this came from: - `setupSignalHandlers` still registers three separate `once` handlers with different bodies here, rather than the single `handleSignal` helper. Each is hoisted to a named const so `dispose()` can remove it; the bodies are unchanged. A `once` listener that never fires stays on the emitter, so the leak is the same. - `ForkedProcessTaskRunner.cleanup()` is synchronous on this branch, so the `await` in `dispose()` is a formality. The comment says the children are signalled first rather than reaped first. - `packages/nx` runs on jest here, not vitest, so the three specs use `jest.*`. `task-orchestrator.spec.ts` does not exist on 22.7.x and is added with only the tests this change introduces. (cherry picked from commit 0ae9069) Co-authored-by: FrozenPandaz <jasonjean1993@gmail.com> Co-authored-by: nx-cloud[bot] <71083854+nx-cloud[bot]@users.noreply.github.com>
Contributor
|
View your CI Pipeline Execution ↗ for commit bd035b2
☁️ Nx Cloud last updated this comment at |
FrozenPandaz
approved these changes
Sep 9, 2026
Contributor
|
This pull request has already been merged/closed. If you experience issues related to these changes, please open a new issue referencing this pull request. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of #36866 to 22.7.x. The leak goes back to at least 22.5, so agents on this line still grow unbounded.
The port differs from the original in three places, since 22.7.x still has separate signal handlers, a synchronous
ForkedProcessTaskRunner.cleanup(), and jest rather than vitest. The commit message covers each.Current Behavior
Every
runDiscreteTaskscall registers SIGINT/SIGTERM/SIGHUP handlers that are never removed. Each closes over its orchestrator, so an Nx Cloud agent retains every orchestrator it ever created, along with its task hasher and every task result.TaskHistoryLifeCycle.pendingResultskeeps everyTaskResultfor the life of the process too.Expected Behavior
Listeners are released once a run completes, and agent memory stays flat across assignment batches.
Related Issue(s)
Fixes #