CI diagnostic for leak-sanitizer annotations (broken on purpose) - #2426
HaraldNordgren wants to merge 4 commits into
Conversation
|
There is an issue in commit d24faf4:
|
d24faf4 to
7ae8217
Compare
|
There is an issue in commit 7ae8217:
|
04467a1 to
6525519
Compare
10636fc to
cdfd166
Compare
|
There is an issue in commit cdfd166:
|
|
Diagnostic done, confirmed what was needed. |
A leak is only discovered once, at the end of a whole script, well
after every test has already reported ok, and it gets no annotation at
all, so a leak-sanitizer job's only visible failure is:
Process completed with exit code 1.
Give a leak its own annotation. Point it at the test script, the exact
line isn't known, only which script the leak turned up in, and put the
sanitizer report in a log group next to it, so it stays visible.
Once a script has one leak, it keeps running: the sanitizer log
directory is never cleared between tests, so every later test in the
same script sees the same leftover log entries and also reports "not
ok", burying the one real failure in copies of itself. Stop a
leak-sanitizer script at its first failure with --immediate instead.
A failing test already gets its own annotation once its script
finishes, but --immediate exits as soon as that test fails, before
reaching the code that writes it. Write the annotation first, so
turning on --immediate here does not silently drop it.
Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
When a test fails, GitHub shows an annotation naming it, for example:
failed: t1060.17 partial clone of corrupted repository
but the location GitHub attaches to that annotation is the CI
workflow file itself, not the test script, so there is nothing
pointing at where the test actually lives.
Find the line a test is defined on by searching its script for the
test's own description as a fixed string, using the first match, and
attach that file and line to the annotation instead. Fall back to
line 1 when the description is not found verbatim, which happens when
a test builds its description at runtime instead of writing it out
literally.
Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
…osis Drop test_must_fail from "%(raw) with --shell must fail" so the test itself fails, to see on real CI whether the escaped description and the unescaped ::group:: title (which contains the same %(raw) text via the test body) actually render correctly or get corrupted.
cdfd166 to
91a807c
Compare
|
There is an issue in commit 4996bd8:
|
|
There are issues in commit 91a807c:
|
91a807c to
94ae0ba
Compare
|
There is an issue in commit 4996bd8:
|
|
There are issues in commit 94ae0ba:
|
Diagnostic PR for #2419, it adds two leaks to confirm that the annotations work correctly.