We should kick off tests in a random order. This will flush out any interdependencies per test. However the order should be reproducible to debug such interdependencies. Ruby's minitest has a --seed option which can be set on the CLI to control the order. A random seed is used if no value is provided and it's reported in the output.
Starting tests in a random order within a test file is the easiest place to start. Presumably .serial would run in source order.
We could consider running test files in a random order as well, though they're already somewhat random because each test file runs in its own process, and because each test file runs in its own process it'd be hard to run them in a deterministic order. So probably best not to even try.
Thoughts?
IssueHunt Summary
Backers (Total: $80.00)
Submitted pull Requests
Tips
We should kick off tests in a random order. This will flush out any interdependencies per test. However the order should be reproducible to debug such interdependencies. Ruby's minitest has a
--seedoption which can be set on the CLI to control the order. A random seed is used if no value is provided and it's reported in the output.Starting tests in a random order within a test file is the easiest place to start. Presumably
.serialwould run in source order.We could consider running test files in a random order as well, though they're already somewhat random because each test file runs in its own process, and because each test file runs in its own process it'd be hard to run them in a deterministic order. So probably best not to even try.
Thoughts?
IssueHunt Summary
Backers (Total: $80.00)
Submitted pull Requests
Become a backer now!
Or submit a pull request to get the deposits!
Tips