Repository navigation
[buildbot] Disable the cpu resources on slow buildbots #74602
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2017-06-10.09:06:56.020>
created_at = <Date 2017-05-20.22:06:09.443>
labels = ['3.7', 'tests']
title = '[buildbot] Disable the cpu resources on slow buildbots'
updated_at = <Date 2017-06-10.09:06:56.019>
user = 'https://github.com/vstinner'bugs.python.org fields:
activity = <Date 2017-06-10.09:06:56.019>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2017-06-10.09:06:56.020>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2017-05-20.22:06:09.443>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 30417
keywords = []
message_count = 13.0
messages = ['294052', '295086', '295089', '295094', '295219', '295224', '295225', '295226', '295594', '295599', '295600', '295601', '295625']
nosy_count = 6.0
nosy_names = ['brett.cannon', 'belopolsky', 'pitrou', 'vstinner', 'zach.ware', 'serhiy.storchaka']
pr_nums = ['1928', '1951', '1961', '1962', '1964', '2057', '2058', '2059']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue30417'
versions = ['Python 3.7']
Activity
vstinner commentedon May 20, 2017
Follow-up of the issue bpo-30314. Copy of Serhiy Storchaka's msg294048:
test_tools is so slow because it proceeds *all* Python files. This controlled by the "cpu" resource (only 10 random files are chosen if it is disabled).
Maybe disable the "cpu" resource on slow buildbots?
Good point, I like the idea.
It seems like regrtest accepts -u all,-cpu option.
serhiy-storchaka commentedon Jun 3, 2017
And disable the "tzdata" resource for skipping generated tests for all timezones in test_datetime. Or make them requiring not only "tzdata", but "cpu".
serhiy-storchaka commentedon Jun 3, 2017
This made tests faster by a third!
brettcannon commentedon Jun 3, 2017
One thing to note is if we want to speed up things like the coverage run on Travis we may want to make this optionally more deterministic rather than fully random for the 10 selected files, otherwise coverage shifts and we can't rely on any coverage metrics per-PR to know if code is increasing or decreasing coverage. Maybe having a
-u deterministicresource to take out the randomness for coverage runs but leave the randomness for buildbots? This might also require tweaking our Travis tests as they currently use the buildbot make rule (I think).P.S. I bet there are also some multiprocessing tests that go a bit overboard that we could consider scaling back, e.g. the coverage run skips a bunch of tests because they seem to process the entire stdlib.
zware commentedon Jun 5, 2017
New changeset c53b13b by Zachary Ware in branch 'master':
bpo-30417: Disable 'cpu' and 'tzdata' resources on Travis (GH-1928)
c53b13b
zware commentedon Jun 6, 2017
New changeset d125738 by Zachary Ware in branch '3.6':
[3.6] bpo-30417: Disable 'cpu' and 'tzdata' resources on Travis (GH-1928)
d125738
zware commentedon Jun 6, 2017
New changeset 58e5dd1 by Zachary Ware in branch '3.5':
[3.5] bpo-30417: Disable 'cpu' resource on Travis (GH-1928)
58e5dd1
zware commentedon Jun 6, 2017
New changeset ddf8f14 by Zachary Ware in branch '2.7':
[2.7] bpo-30417: Disable 'cpu' resource on Travis (GH-1928)
ddf8f14
zware commentedon Jun 10, 2017
New changeset 42e3acd by Zachary Ware in branch 'master':
bpo-30417: Disable
cpuresource on AppVeyor (GH-1951)42e3acd
zware commentedon Jun 10, 2017
New changeset e380c19 by Zachary Ware in branch '3.6':
[3.6] bpo-30417: Disable
cpuresource on AppVeyor (GH-1951)e380c19
zware commentedon Jun 10, 2017
New changeset 91581d4 by Zachary Ware in branch '3.5':
[3.5] bpo-30417: Disable
cpuresource on AppVeyor (GH-1951)91581d4
zware commentedon Jun 10, 2017
New changeset 3c7b67b by Zachary Ware in branch '2.7':
[2.7] bpo-30417: Disable
cpuresource on AppVeyor (GH-1951)3c7b67b
vstinner commentedon Jun 10, 2017
Thanks for taking care of our CI, Zach ;-) Buildbots, Travis CI job and AppVeyor are now fixed if I understood correctly, so I close the issue.
Serhiy Storchaka: "And disable the "tzdata" resource for skipping generated tests for all timezones in test_datetime. Or make them requiring not only "tzdata", but "cpu"."
Hum, I have no real opinion on this issue. I think that the first main issue was test_tools, and disabling cpu on our CI has fixed the issue. I'm not sure that we should go deeper. *If* you consider thta that test_datetime is too slow, yeah, maybe add also a requirement on the "cpu" resource. But I don't recall to have seen "test_datetime" in the final "top 10 slowest tests" (even if I didn't look recently).
Brett Cannon: "One thing to note is if we want to speed up things like the coverage run on Travis we may want to make this optionally more deterministic rather than fully random for the 10 selected files, (...)"
Using a fixed list of filenames would benefit to everyone, not only to the CI. But someone has to select these files :-)
Brett: "Maybe having a
-u deterministicresource to take out the randomness for coverage runs but leave the randomness for buildbots?"We may start by using a fixed random seed for our coverage tests? And make sure that each test file starts with the same seed? IMHO coverage is a different issue, so I suggest to open a new issue if you consider that it's worth it to enhance the coverage CI job ;-)