Skip to content

Add timing and junit xml output to testsuite - #9436

Merged
sklam merged 12 commits into
numba:mainfrom
sklam:misc/tests_timing_and_xml
Mar 4, 2024
Merged

Add timing and junit xml output to testsuite#9436
sklam merged 12 commits into
numba:mainfrom
sklam:misc/tests_timing_and_xml

Conversation

@sklam

@sklam sklam commented Feb 13, 2024 •
edited
Loading

Copy link
Copy Markdown
Member

Changes testsuite to:

  • Add custom logic to write junit xml output using --junit.
  • Record test runtime and print them with --show-timing.
  • Record PID and active tests per PID. Report these info at crash.

@sklam sklam added the skip_release_notes Skip towncrier requirement label Feb 14, 2024
@sklam
sklam marked this pull request as ready for review February 14, 2024 19:43
@gmarkall

gmarkall commented Feb 20, 2024

Copy link
Copy Markdown
Member

I tried clicking through to a test report (e.g. https://dev.azure.com/numba/numba/_TestManagement/Runs?runId=1926474&_a=runCharts) and I got a 401 (permission denied) error - are you able to access that? Do some permissions need changing somewhere?

@sklam

sklam commented Feb 22, 2024

Copy link
Copy Markdown
Member Author

I tried clicking through to a test report (e.g. https://dev.azure.com/numba/numba/_TestManagement/Runs?runId=1926474&_a=runCharts) and I got a 401 (permission denied) error - are you able to access that? Do some permissions need changing somewhere?

I finally figured it out. I need to enable "Test Plans" feature in the project settings. The link now works!
You can also find the xml files in the attachments.

@sklam sklam mentioned this pull request Feb 22, 2024
Closed
1 task done
@gmarkall

gmarkall commented Feb 23, 2024

Copy link
Copy Markdown
Member

I tried clicking through to a test report (e.g. https://dev.azure.com/numba/numba/_TestManagement/Runs?runId=1926474&_a=runCharts) and I got a 401 (permission denied) error - are you able to access that? Do some permissions need changing somewhere?

I finally figured it out. I need to enable "Test Plans" feature in the project settings. The link now works! You can also find the xml files in the attachments.

I still get a 401 for that link :-(

@sklam

sklam commented Feb 23, 2024

Copy link
Copy Markdown
Member Author

Can you see the attachment the "Tests" tab:

Screenshot 2024-02-23 at 10 14 01 AM

Need to click on a test run (e.g. "JUnit_TestResults_NNNN") in the table then click "attachments" on the right panel

@gmarkall

gmarkall commented Feb 27, 2024

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

azure-pipelines Bot commented Feb 27, 2024

Copy link
Copy Markdown
Contributor
Azure Pipelines successfully started running 1 pipeline(s).

@esc
esc self-requested a review February 28, 2024 16:57
esc
esc requested changes Feb 28, 2024
View reviewed changes

@esc esc left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found a minor typo. Also, tried to log in to Azure to view the test results, as it appears as though I need to login to view them

This is what my screen looks like:

Screenshot 2024-02-28 at 23 08 02

Comment thread numba/tests/test_runtests.py Outdated
@esc esc added 4 - Waiting on author Waiting for author to respond to review 4 - Waiting on reviewer Waiting for reviewer to respond to author and removed 3 - Ready for Review labels Feb 28, 2024
@esc

esc commented Feb 28, 2024

Copy link
Copy Markdown
Member

Can you see the attachment the "Tests" tab:

Can you share more details about this screen and how to get there?

@esc

esc commented Feb 29, 2024

Copy link
Copy Markdown
Member

OK, so there are some UI contortions that one has to go through, in order to retrieve the file, but it does seem to be possible w/o logging in.

Screenshot 2024-02-29 at 15 13 23

esc
esc previously approved these changes Feb 29, 2024
View reviewed changes

@esc esc left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spellchecked and reviewed the code, looked at the generated RTD and worked out how to download and view the XML on azure.

LGTM

@esc esc removed the 4 - Waiting on author Waiting for author to respond to review label Feb 29, 2024
@esc esc added 5 - Ready to merge Review and testing done, is ready to merge and removed 4 - Waiting on reviewer Waiting for reviewer to respond to author labels Feb 29, 2024
@sklam @esc
Update numba/tests/test_runtests.py
2d5b346 
Co-authored-by: Emergency Self-Construct <esc@users.noreply.github.com>
@sklam
sklam merged commit 42616dc into numba:main Mar 4, 2024
@sklam sklam added this to the 0.60.0-rc1 milestone Mar 4, 2024
sklam added a commit to sklam/numba that referenced this pull request Sep 19, 2024
@sklam
Revert "Merge pull request numba#9436 from sklam/misc/tests_timing_an…
b1595a9 
…d_xml"

This reverts commit 42616dc, reversing
changes made to a7fc4b5.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@esc esc esc approved these changes

Assignees

No one assigned

Labels

5 - Ready to merge Review and testing done, is ready to merge skip_release_notes Skip towncrier requirement

Projects

None yet

Milestone

0.60.0-rc1

Development

Successfully merging this pull request may close these issues.

3 participants

@sklam @gmarkall @esc