Skip to content

test: raise coverage and stop persisting checkout credentials - #595

Merged
shenxianpeng merged 20 commits into
mainfrom
chore/repo-health-check
Oct 2, 2026
Merged

shenxianpeng merged 20 commits into
mainfrom
chore/repo-health-check

Conversation

@shenxianpeng

@shenxianpeng shenxianpeng commented Oct 1, 2026 •

Copy link
Copy Markdown
Member

Summary

This pull request makes three kinds of change:

  • SHA pins: none needed. Every action on main is already pinned to a full commit SHA, and each SHA matches the tag in its comment. The two org reusable workflows reference @main, a branch, and are left as they are.
  • persist-credentials: false on 4 actions/checkout steps that never use the token after checkout: the build and install jobs in main.yml, CodSpeed, and the publish job. commit-check.yml and scorecard.yml already set it.
  • Coverage (coverage.py on the commit_check package, as nox -s coverage measures it):
    • Lines: 99% (22 of 2257 missed) → 100%.
    • With branches: 98% → 99.9%.
    • Tests: 28 new test functions (31 cases) across 8 test files. They pass against the source on main.
    • # pragma: no cover on two lines in engine.py that cannot run: the abstract BaseValidator.validate body, and a second body check in BodyValidator that the first check always pre-empts.
    • config_test.py now restores commit_check.config after the tomli fallback tests. Before this, four config tests failed on Python 3.10 and made real HTTP requests.

The earlier commits on this branch are undone by commits on top rather than rewritten, because AGENTS.md asks for additive commits and no force pushes. The diff contains only the changes above.

How it was verified

  • Tests: the full suite passes on Python 3.14 (1148 tests) and on 3.10 (1147, one 3.11-only test skipped), with network access blocked.
  • pre-commit: pre-commit run --all-files passes.
  • actionlint: on the changed workflows it reports only the two findings main already has.

BodyValidator checked for text after the subject twice. The message is
stripped before it is split, so its first and last lines both carry
text: whenever there is more than one line there are at least two
non-empty ones, and the first check has already passed. The second
check could never run.

BaseValidator.validate is abstract; its docstring is the whole body,
so the trailing `pass` is dropped as well.
The suite stood at 99% of lines (22 missed) and 98% with branches.
These tests reach every remaining line and all but one branch:

- engine: the base subject rule, a subject with no leading word, an
  author with no identity anywhere, an empty message under the
  sign-off rule, co-authors read from a --rev commit, a merge-base
  target that resolves to no ref, push lines that name no ref pair, an
  unresolvable tag push, ignore_authors for an author not on the list,
  a single accepted disclosure trailer, a co-author fix withheld by the
  pattern, and an unknown check in the detailed run
- api: validate_author with a name only and with no arguments,
  validate_push without a [push] table in the defaults
- main: pre-commit push data with no remote tip, stdin left unread
  without a check that takes it, a --fix that never satisfies its rule
- config, rule builder and util helpers: a scalar section in the
  unknown-setting scan, a found config that vanishes before it is
  read, files = false, a blank GITHUB_REF_NAME, submodule entries in
  ls-tree output, a git diagnostic in the identity lookup

The one branch left is the tag rule builder's guard for a catalog entry
other than "tag", which the catalog does not contain.
Three patterns shipped with commit-check backtrack super-linearly on a
long line, so a long enough commit message holds up the hook or the CI
job that checks it:

- CC012's sign-off pattern `Signed-off-by: .+ <.+@.+>` is cubic in a
  line that never closes the address: 2.5 KB took 2.6 s, and 15 KB was
  still running after 20 s. Each part now stops at the first delimiter
  that can end it, and only the first "Signed-off-by: " on a line is
  read.
- find_trailers (the disclose policy) read a value with a lazy
  `[^\n]*?` before trailing blanks: quadratic in a run of blanks.
- The header pattern behind every CC001 fix suggestion had the same
  lazy shape, so any subject CC001 rejects went through it.

The replacements match exactly what the old patterns matched. Tests
compare each one with its predecessor over 20,000 generated inputs,
and run inputs that took the old patterns over 15 s, which now finish
in well under 0.1 s.
A setting in [commit], [branch] or [push] whose value has the wrong
type either crashed or was quietly read as something else:

- `commit = 5`, or `[[commit]]` (a list of tables), and a non-string
  message or author pattern stopped the run with a bare
  AttributeError and exit code 1, the code a rejected commit gets.
- `ignore_authors = 5` raised a TypeError from inside the engine,
  which reads that list itself.
- A string where a list belongs was read a character at a time:
  `allow_commit_types = "feat,fix"` allowed the types f, e, a, t, i, x,
  and `ignore_authors = "bot"` skipped any author whose name is part
  of "bot".
- `allow_force_push = "false"` counted as true and let force pushes
  through; `subject_max_length = "72"` switched the rule off.

The rule builder now checks each value against the type of its default
before it builds anything, and refuses a mismatch as a config error
(exit code 2) that names the setting, as it already does for a regex
that does not compile. [files] and [tag] keep their lenient handling,
and so do the AI settings, whose readers already check their values.
The README's list of exit-code-2 causes says so.
The build, install, benchmark and publish jobs never push or fetch with
the token after checkout, so actions/checkout no longer leaves it in
.git/config (zizmor: artipacked). commit-check.yml and scorecard.yml
already did this.

Dependabot now waits seven days after a release before proposing it,
for both GitHub Actions and pip (zizmor: dependabot-cooldown).
GitHub notes on these runs that ubuntu-latest moves to Ubuntu 26 from
October 19, 2026. The CodSpeed and Commit Check jobs were the only ones
still on ubuntu-latest; they now name ubuntu-24.04 like every other job,
so the image (and the Python builds setup-python finds for it) changes
when the repository decides, not on GitHub's schedule.

actionlint:
- publish-package.yml: a release trigger takes no branches filter;
  GitHub ignored it, so dropping it changes nothing.
- codspeed.yml: quote ".[test]" so the shell cannot read it as a glob
  (shellcheck SC2102).
pre-commit hooks:
- ruff-pre-commit v0.15.20 -> v0.15.22, the last 0.15 release. 0.16
  turns on 413 lint rules by default instead of 59 (123 findings here)
  and formats Markdown code blocks, so adopting it is left as a
  decision rather than folded into this update.
- mirrors-mypy v2.1.0 -> v2.3.1. mypy now reads through a getattr with
  a literal name, so _LazyOpener.handlers says plainly that typeshed
  does not declare OpenerDirector.handlers.
- codespell v2.4.2 -> v2.4.3
- commit-check v2.17.0 -> v2.18.2, the latest published release

GitHub Actions: codecov/codecov-action v7.0.0 -> v7.1.1, pinned to its
commit as before; it only adds an optional `cleanup` input.

uv.lock: `uv lock --upgrade`, including pytest 9.1.1, pytest-codspeed
5.0.3, pytest-mock 3.16.0 and coverage 7.16.2, the versions CI already
installs. The suite passes against the lock on Python 3.10 and 3.14.
On Python 3.10 four config tests failed, and reached the network on the
way: the inherit_from and URL loading tests patch
"commit_check.config._opener.open", and none of those patches took.

Two tomli fallback tests import commit_check.config afresh and then put
the original back in sys.modules, but not on the commit_check package,
whose `config` attribute the fresh import had rebound. Python 3.10's
patch() resolves a dotted target through package attributes, so every
later patch landed on the copy while the code under test used the
original. Python 3.11 resolves it through sys.modules, which is why the
suite only fails on 3.10, which CI does not run the tests on.

Both tests now restore the attribute too. The suite passes on 3.10 and
3.14 with DNS lookups blocked.
v2.18.2 is the latest release on PyPI and its tag exists, so the
snippet installs it instead of v2.18.1 (AGENTS.md).
@coderabbitai

coderabbitai Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 3087b169-8c2d-4176-83e6-d31c4f366769

📥 Commits

Reviewing files that changed from the base of the PR and between f3985fb and 1ea2df4.

📒 Files selected for processing (12)
  • .github/workflows/codspeed.yml
  • .github/workflows/main.yml
  • .github/workflows/publish-package.yml
  • commit_check/engine.py
  • tests/api_test.py
  • tests/config_merger_test.py
  • tests/config_test.py
  • tests/engine_fixes_test.py
  • tests/engine_test.py
  • tests/main_test.py
  • tests/rule_builder_test.py
  • tests/util_test.py
 _____________________________________________________________________________________________________________________________________
< Prototype to learn. Prototyping is a learning experience. Its value lies not in the code you produce, but in the lessons you learn. >
 -------------------------------------------------------------------------------------------------------------------------------------
  \
   \   \
        \ /\
        ( )
      .( o ).
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the chore label Oct 1, 2026
Comment thread .github/workflows/codspeed.yml Fixed
Comment thread .github/workflows/codspeed.yml Fixed
@codecov

codecov Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (f3985fb) to head (1ea2df4).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff             @@
##             main      #595      +/-   ##
===========================================
+ Coverage   99.02%   100.00%   +0.97%     
===========================================
  Files          14        14              
  Lines        2257      2252       -5     
===========================================
+ Hits         2235      2252      +17     
+ Misses         22         0      -22     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@codspeed

codspeed Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 601 untouched benchmarks
⏩ 126 skipped benchmarks1


Comparing chore/repo-health-check (1ea2df4) with main (f3985fb)

Open in CodSpeed

Footnotes

  1. 126 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

The two type-check tests built the RuleBuilder inside the block, so a
ConfigError from the constructor would have passed for one from
build_all_rules (SonarCloud python:S5778). The builder is now made
first, and only the call under test sits in the block.
The type check runs every time rules are built, which the Python API
does on every call. It called get_default_config() each time and
checked each value through a generator, adding about 6.4 us to a
build that takes about 15 us; CodSpeed reported the API benchmarks
15-18% slower.

The default types are now worked out once at import, and a value of
exactly its default's type passes with one comparison (a list, with
one pass over its items). Anything else still goes through the full
check, so what is accepted and refused does not change. The check now
costs about 2 us on a full config and 0.5 us on a partial one.
The linear CC012 pattern scanned every line a character at a time
through an alternation, and backtracked over the scan when a line had
no sign-off: about 5 us per message, against 0.25 us for the cubic
pattern it replaced.

It now skips whole lines, and takes each part up to its first
delimiter with a lookahead and a backreference: Python's lookaheads
are atomic, so nothing is scanned twice. About 1.3 us per message,
still linear (2.6 ms for 300 KB of adversarial input, 61 ms for a
million lines), and still accepting exactly what
`Signed-off-by: .+ <.+@.+>` accepts: checked against it on 1.2
million generated messages, and by the existing tests.
Quoting `.[test]` changed the line's text, so SonarCloud no longer
matched it to the two findings on it (githubactions:S8541 and S8544)
that are already resolved as won't-fix on main. It raised them again
as new code, and they failed the PR's quality gate on security rating.

The line is back as it is on main. actionlint's shellcheck note on it
(SC2102, info level) returns, and is harmless: no file in the checkout
matches `.[test]` as a glob.
A value of exactly its default's type now takes a shortcut, so a
subclass, such as an IntEnum from a Python caller, goes the long way
round. This test shows the long way still accepts it, and covers the
line that does.
Comment thread .github/workflows/codspeed.yml Outdated
Comment thread .github/workflows/commit-check.yml Outdated
Co-authored-by: Xianpeng Shen <xianpeng.shen@gmail.com>
Dependabot already applies a 3-day cooldown to version updates by default, which is enough here.
The maintainer asked for this pull request to carry only SHA pins,
persist-credentials: false and tests that raise coverage. The source
changes go back to main, to be proposed separately:

- "fix: keep three built-in patterns linear on long input" and its
  follow-up "refactor: speed up the linear sign-off pattern ..."
- "fix: refuse config values of the wrong type", its follow-up
  "refactor: work out the config setting types once ...", and the
  README sentence it added
- "refactor: drop unreachable code from the engine"

The tests that only pass with those changes go with them: the pattern
equivalence and linear-time tests, and the setting-type tests in
rule_builder_test.py, api_test.py and main_test.py. The coverage tests,
which pass against the source on main, stay.
Dependency and documentation updates are out of scope for this pull
request, so these go back to main:

- the pre-commit hook revisions (ruff, mypy, codespell, commit-check),
  with the typing change in config.py that the newer mypy needed
- codecov/codecov-action v7.1.1, back to v7.0.0
- the uv.lock refresh
- the README's pre-commit pin, back to v2.18.1
Dropping it was an actionlint clean-up with no effect on when the
workflow runs, and other workflow edits are out of scope for this pull
request. persist-credentials: false on the checkout stays.
With the dead code back in place, two spots in engine.py cannot run:

- BaseValidator.validate is abstract, and no subclass calls it through
  super(), so its `pass` never runs.
- BodyValidator's second body check: the message is stripped before
  it is split, so two or more lines always hold two non-empty ones,
  and the first check has already passed.

Both are marked `# pragma: no cover`, which changes nothing at run
time. Every other line is covered by tests.
@shenxianpeng shenxianpeng changed the title chore: repository health check (tests, security, deps, CI) test: raise coverage to 100% and stop persisting checkout credentials Oct 2, 2026
@sonarqubecloud

sonarqubecloud Bot commented Oct 2, 2026

Copy link
Copy Markdown

@shenxianpeng shenxianpeng changed the title test: raise coverage to 100% and stop persisting checkout credentials test: raise coverage and stop persisting checkout credentials Oct 2, 2026
@shenxianpeng
shenxianpeng marked this pull request as ready for review October 2, 2026 19:53
@shenxianpeng
shenxianpeng requested a review from a team as a code owner October 2, 2026 19:53
@shenxianpeng
shenxianpeng merged commit 8d7864a into main Oct 2, 2026
32 of 33 checks passed
@shenxianpeng
shenxianpeng deleted the chore/repo-health-check branch October 2, 2026 19:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants