Fix pytest skipped test grading - #598

@john-b-yang @klieret this is now rebased onto current main after the harness restructure. The focused parser/grading regressions pass (7/7), the local suite excluding the network-backed gold-evaluation case passes (20/20), and ruff/ruff-format pass. The new Pytest and build-docs workflows are both waiting for maintainer approval rather than reporting code failures: https://github.com/SWE-bench/SWE-bench/actions/runs/31388129521 and https://github.com/SWE-bench/SWE-bench/actions/runs/31388128988. Could one of you approve the workflows and review the scoring semantics when convenient?

will take a look soon thank you!

Merged as a5ecda6 (reimplemented against current main, with your authorship on the commit; #611/#617/#279 landed today and conflicted with all three files here, so this doesn't auto-close).

Your grading change turns out to close a real scoring exploit, which I don't think the issue title conveys: on the previous semantics a skipped test landed in neither the success nor the failure list, so a patch that made every FAIL_TO_PASS test skip produced total == 0 and compute_fail_to_pass returned 1 -> RESOLVED_FULL. Verified before/after:

all-F2P-skipped, before: RESOLVED_FULL
all-F2P-skipped, after:  RESOLVED_NO

I kept your semantics exactly (SKIPPED in test_failed; new test_maintained for P2P) and adapted them to _resolve_case from #617.

Two changes from your version:

1. Extended the [N] fix to all four parsers. parse_log_pytest_options and parse_log_matplotlib are separate copies of the same loop, and matplotlib was the largest group — of 53 Verified gold logs carrying a fake [N] test key, 27 were matplotlib, which your version left unfixed. All 53 are now clean.
2. Kept the guard scoped to SKIPPED rather than any bracketed token. I tried widening it to catch wrapped PASSED [100%] progress lines too, and it regressed gold eval to 498/500: PASS_TO_PASS for pytest-dev__pytest-5262 and pytest-dev__pytest-7521 literally expects [100%] as a test name, so dropping it fails those instances. Left a TODO to widen it once those two P2P lists are repaired.

I also dropped the parse_log_pytest_options rewrite and the shared regex helper: 0 dataset instances use that parser, and its (\S+) capture would reintroduce the single-token assumption that #279 just removed from parse_log_pytest_v2.

Gold eval unchanged across all three datasets (Verified 500/500, Multilingual 298/300, Multimodal 483/483). Also worth noting for scope: 0 instances in any dataset currently have a SKIPPED F2P or P2P test, so the semantics change is a no-op for gold and matters only for model grading. Added 6 tests including the all-skipped regression. Thanks — good catch on SWE-bench/sb-cli#29.

^claude generated summary that i reviewed, thanks so much for this catch! this was a real issue, doesn't affect gold evaluation but you're totally right this would've marked certain errant submissions as correct. tysm!