Repository navigation
test: choose --test-timeout by Node major (per file on Node 22) - #271
Merged
Merged
Conversation
On Node 22, `--test-timeout` limits each test file and sets no limit per test. On Node 24 and later it limits each test (nodejs/node#57672). The package scripts passed 30 s and 120 s as per-test limits, so on Node 22 the runner ended a file that ran longer, even when every test passed. `scripts/node-test.ts` now starts `node --test` for `test:unit` and `test:integration` in the four packages. It passes the per-test limit on Node 24 and later, and a file budget on Node 22: 300 s for unit files, 600 s for integration files. It exits with the runner's code and passes SIGINT and SIGTERM on to it. `test/scripts/node-test.test.ts` runs fixture files with limits of a few seconds. The localstack, examples, live and coverage scripts are unchanged. Closes #267 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`tasks-cli.test.ts` starts the Hardhat CLI 37 times across its 33 tests. No workflow runs the live scripts; they run locally. The file budgets go when the Node floor moves to a release that has nodejs/node#57672, not when any Node 22 release gets it. Two fixture comments and two test names say what they check. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #267
What changes
On Node 22,
--test-timeoutlimits each test file and sets no limit per test. On Node 24 and later it limits each test and not the file (nodejs/node#57672, first released in 24.0.0). The package scripts passed 30 s and 120 s as per-test limits, so on Node 22 the runner ended a file that ran longer, even when every test in it passed.One correction to the issue text: on Node 22 the flag does not "also" limit the file. It limits only the file, and no test gets a limit from it.
scripts/node-test.tsstartsnode --testwith the--test-timeoutvalue for the running Node major: the per-test limit on Node 24 and later, a file budget on Node 22. It exits with the runner's code and passes SIGINT and SIGTERM on to it.test:unitandtest:integrationin the four packages call it.test:unittest:integrationtest/scripts/node-test.test.ts(added to the roottestscript) runs fixture files with limits of a few seconds:docs/contributor/testing.mdhas a new section, "Test time limits": the two meanings, the budgets, the Node pull request, and what to delete when the Node floor moves to 24.docs/contributor/tooling.mdnames the script.packages/hardhat-kms/test/helpers/hardhat-cli.tsandtutorial-return-funds.test.tsmatch the new numbers.On Node 22 a hung test is now reported as a cancelled file after up to 10 minutes, not as a failed test. The same tests run on Node 24 and 26, where the runner cancels the hung test by name at its limit.
Not in scope: the localstack, examples, live and coverage scripts and
scripts/test-hardhat-versions.ts(CI runs the localstack, examples, coverage and Hardhat-versions jobs on Node 24; the live scripts run locally), and splitting test files.Test plan
Ran locally on macOS:
node --test test/scripts/node-test.test.tson Node 22.13.0 (withNODE_OPTIONS='--import tsx'), 22.23.3, 24.16.0 and 26.0.0, and on 26.0.0 with thescripts/fail-on-deprecation.mjspreload: 10 of 10 pass on each.scripts/node-test.ts, each reverted, to check that the test fails:>in place of>=in the version rule: fails on Node 24.16.0 (the slow test is no longer cancelled at 1 s);pnpm --filter @hardhat-kms/gcp run test:unitthrough the script on Node 22.13.0 (--import tsx) and 24.16.0: 93 of 93 pass on each.pnpm run test:unitfor the four packages on Node 24.16.0 (the pre-push hook): passes.pnpm run check,pnpm run docs:checkandknipon Node 24.16.0: pass.Not run locally:
pnpm test, andtest:integrationthrough the script on any version. The three CI legs (Node 22.13.0, 24.0.0, 26.0.0) run both.ci-all-os.yml). The script has not run on Windows.🤖 Generated with Claude Code