test_math.test_sumprod_stress() takes 5 minutes on debug build #110327
Description
On the " PPC64LE Fedora Stable Clang 3.x" with Python build by "/configure CC=clang LD=clang --with-pydebug", test_math takes 7 min 39 sec:
make buildbottest 'TESTOPTS=-j2 --junit-xml test-results.xml -j10 --fail-rerun ${BUILDBOT_TESTOPTS}' TESTPYTHONOPTS= TESTTIMEOUT=1200
== CPython 3.13.0a0 (heads/main:269005e784, Oct 3 2023, 23:52:56) [Clang 16.0.6 (Fedora 16.0.6-3.fc38)]
== Linux-5.8.17-200.fc32.ppc64le-ppc64le-with-glibc2.37 little-endian
== Python build: debug
== cwd: /home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang/build/build/test_python_worker_2081694æ
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
== resources (10): audio, cpu, curses, decimal, gui, largefile, network, subprocess, urlfetch, walltime
(...)
10 slowest tests:
- test_math: 7 min 39 sec
- test.test_multiprocessing_spawn.test_processes: 2 min 49 sec
- test_pathlib: 2 min 36 sec
- test_unparse: 2 min 30 sec
- test_tokenize: 2 min 29 sec
- test.test_multiprocessing_forkserver.test_processes: 2 min 22 sec
- test_capi: 1 min 52 sec
- test_cppext: 1 min 42 sec
- test_unicodedata: 1 min 39 sec
- test_tarfile: 1 min 33 sec
Can we split test_math into a test package made of multiple tests to run them in parallel?
Should we remove/skip some slow tests? Buildbot uses --slow-ci which enables -u all resources.
Activity
vstinner commentedon Oct 4, 2023
Timings as JUnit XML of
./python -m test test_math -v --junit-xml=results-all.xml -u all: results-all.xml.txt (5 min 28 sec).vstinner commentedon Oct 5, 2023
test_math.test_sumprod_stress() alone takes 5 min 13 sec. The test was added by issue GH-100485: commit 47b9f83.
vstinner commentedon Oct 5, 2023
This test uses 9 pools of lengths: [10, 9, 9, 11, 10, 9, 8, 5, 8]. It uses 4 nested loops for a total of 5,969,664 iterations.
IMO that's excessive and it should be possible to minimize the test to a few cases. I don't think that there is a need to test 5,969,664 cases.
vstinner commentedon Oct 5, 2023
@rhettinger @serhiy-storchaka: Can you please have a look at test_math.test_sumprod_stress() and see if we can test a few cases, rather than testing 5,969,664 cases?
On my laptop with Python built with
gcc -O0,./python -m test test_math -u alltakes 10 min 50 sec :-(vstinner commentedon Oct 5, 2023
gcc -O3(./configure),./python -m test test_math -u all -v -m test_sumprod_stresstakes 25 seconds.gcc -Og(./configure --with-pydebug),./python -m test test_math -u all -v -m test_sumprod_stresstakes 1 min 49 sec.serhiy-storchaka commentedon Oct 5, 2023
Sorry, I don't know exactly what it is testing and what can be cut out without losing the value of the test.
skirpichev commentedon Oct 6, 2023
@vstinner,
test_sumprod_stress()should cover some lines, mentioned in #110000 (e.g. overflows for big integers), but I think at least in this function we could test only "exact" floats/ints, while mixed inputs could go to thetestSumProd().vstinner commentedon Dec 20, 2023
Well, apparently, other people don't care much, so let's leave it like that for now. If someone wants to propose a change, please open a new issue.
[-]Running test_math on PPC64LE Fedora Stable Clang 3.x takes 7 min 39 sec[/-][+]test_math.test_sumprod_stress() takes 5 minutes on debug build[/+]