Skip to content

CI diagnostic for leak-sanitizer annotations (broken on purpose) - #2426

Draft
HaraldNordgren wants to merge 4 commits into
git:masterfrom
HaraldNordgren:ci-leak-annotation-diagnostic
Draft

HaraldNordgren wants to merge 4 commits into
git:masterfrom
HaraldNordgren:ci-leak-annotation-diagnostic

Conversation

@HaraldNordgren

@HaraldNordgren HaraldNordgren commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Diagnostic PR for #2419, it adds two leaks to confirm that the annotations work correctly.

@gitgitgadget-git

Copy link
Copy Markdown

There is an issue in commit d24faf4:
DO NOT MERGE: deliberately introduce two leaks for CI diagnosis

  • Commit not signed off

@HaraldNordgren
HaraldNordgren force-pushed the ci-leak-annotation-diagnostic branch from d24faf4 to 7ae8217 Compare September 27, 2026 22:11
@gitgitgadget-git

Copy link
Copy Markdown

There is an issue in commit 7ae8217:
DO NOT MERGE: deliberately introduce two leaks for CI diagnosis

  • Commit not signed off

@HaraldNordgren
HaraldNordgren force-pushed the ci-leak-annotation-diagnostic branch 4 times, most recently from 04467a1 to 6525519 Compare September 28, 2026 16:12
@HaraldNordgren
HaraldNordgren marked this pull request as draft September 28, 2026 18:55
@HaraldNordgren
HaraldNordgren force-pushed the ci-leak-annotation-diagnostic branch 3 times, most recently from 10636fc to cdfd166 Compare October 1, 2026 21:28
@gitgitgadget-git

Copy link
Copy Markdown

There is an issue in commit cdfd166:
DO NOT MERGE: deliberately fail a %(raw)-containing test for CI diagnosis

  • Commit not signed off

@HaraldNordgren

Copy link
Copy Markdown
Contributor Author

Diagnostic done, confirmed what was needed.

A leak is only discovered once, at the end of a whole script, well
after every test has already reported ok, and it gets no annotation at
all, so a leak-sanitizer job's only visible failure is:

    Process completed with exit code 1.

Give a leak its own annotation. Point it at the test script, the exact
line isn't known, only which script the leak turned up in, and put the
sanitizer report in a log group next to it, so it stays visible.

Once a script has one leak, it keeps running: the sanitizer log
directory is never cleared between tests, so every later test in the
same script sees the same leftover log entries and also reports "not
ok", burying the one real failure in copies of itself. Stop a
leak-sanitizer script at its first failure with --immediate instead.

A failing test already gets its own annotation once its script
finishes, but --immediate exits as soon as that test fails, before
reaching the code that writes it. Write the annotation first, so
turning on --immediate here does not silently drop it.

Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
When a test fails, GitHub shows an annotation naming it, for example:

    failed: t1060.17 partial clone of corrupted repository

but the location GitHub attaches to that annotation is the CI
workflow file itself, not the test script, so there is nothing
pointing at where the test actually lives.

Find the line a test is defined on by searching its script for the
test's own description as a fixed string, using the first match, and
attach that file and line to the annotation instead. Fall back to
line 1 when the description is not found verbatim, which happens when
a test builds its description at runtime instead of writing it out
literally.

Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
@HaraldNordgren HaraldNordgren reopened this Oct 2, 2026
…osis

Drop test_must_fail from "%(raw) with --shell must fail" so the test
itself fails, to see on real CI whether the escaped description and
the unescaped ::group:: title (which contains the same %(raw) text via
the test body) actually render correctly or get corrupted.
@HaraldNordgren
HaraldNordgren force-pushed the ci-leak-annotation-diagnostic branch from cdfd166 to 91a807c Compare October 2, 2026 07:39
@gitgitgadget-git

Copy link
Copy Markdown

There is an issue in commit 4996bd8:
DO NOT MERGE: deliberately fail a %(raw)-containing test for CI diagnosis

  • Commit not signed off

@gitgitgadget-git

Copy link
Copy Markdown

There are issues in commit 91a807c:
DO NOT MERGE: deliberately fail a test for baseline annotation comparison

  • Commit checks stopped - the message is too short
  • Commit not signed off

@HaraldNordgren
HaraldNordgren force-pushed the ci-leak-annotation-diagnostic branch from 91a807c to 94ae0ba Compare October 2, 2026 07:40
@gitgitgadget-git

Copy link
Copy Markdown

There is an issue in commit 4996bd8:
DO NOT MERGE: deliberately fail a %(raw)-containing test for CI diagnosis

  • Commit not signed off

@gitgitgadget-git

Copy link
Copy Markdown

There are issues in commit 94ae0ba:
DO NOT MERGE: deliberately fail a test for baseline annotation comparison

  • Commit checks stopped - the message is too short
  • Commit not signed off

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant