Fix consumed character tracking in Jaro-Winkler matching - #15483
dhairyajangir wants to merge 2 commits into
Conversation
|
Please add one or more tests (without removing or modifying any existing tests) that fail with the current algorithm but pass with the proposed algorithm. Please add a timeit or similar benchmark that measures the performance difference on a large tree (like 2k items). |
Preserve all eight existing examples and append three regressions that fail before the fix. Add an optional 2,000-character timing comparison without changing the production implementation.
|
Implemented in Added three regression assertions; all eight original examples remain unchanged. The three additions fail against the original algorithm, and all eleven examples pass with the fix. I interpreted the request's “2k-item tree” wording as approximately 2,000-character strings for Jaro-Winkler. The same file now supports: python3.15t strings/jaro_winkler.py --benchmarkThe identical benchmark block on the original and proposed algorithms gives the following, on CPython 3.15.0rc2 free-threaded with GIL disabled, pinned to one CPU, best of five single-call repeats:
This exposes a substantial performance cost in the current fix on these long inputs. The Python index/set scan is slower than the original C-level string operations; these workload-specific ratios are not a performance-improvement claim. For the trailing-space case, the original result is incorrect at The follow-up leaves production code unchanged. Independent review verified test preservation, the failing/passing cases, and benchmark fairness. All applicable all-files pre-commit hooks pass. |
Describe your change
Track consumed target indices in
jaro_winklerinstead of replacing matched characters with spaces. A literal space can currently match those replacement characters:jaro_winkler("aa ", "aa")returns1.1333333333333333andjaro_winkler("a a", "aa")returns1.15, exceeding the documented range.The matching loop now consumes the actual unused position inside the existing window. This also avoids whole-string
.index()selecting a repeated character before that window. Window size, score formulas, and all existing doctests are unchanged.Validation under CPython 3.15.0rc2 free-threaded:
a,b, and space stay within[0, 1]; the original code has 3,034 out-of-range results.pre-commit run --all-files --show-diff-on-failure: passed.The original external oracle results above remain applicable: this follow-up does not change the production function. In response to the maintainer's request, the same file now includes appended regression doctests and an optional benchmark. The “2k-item tree” wording is interpreted as approximately 2,000-character strings for this string algorithm.
Run:
The exact same benchmark block was run with the original algorithm from
84b73d08and the proposed implementation, using CPython 3.15.0rc2 free-threaded, GIL disabled, CPU 0 affinity, and the best of five single-call repetitions. Input construction and the displayed correctness score are outside the timer.These measurements expose a material performance cost in the current correctness fix on these long inputs. The proposed index/set scan runs in Python, whereas the original uses C-level string operations; the original trailing-space result is incorrect and exceeds 1. These are workload-specific local measurements, not a performance improvement or a universal ratio. The performance tradeoff is reported for maintainer review; no optimization is silently included in this tests-and-benchmark follow-up.
This original change was prepared with AI assistance and independently reviewed. The follow-up independently preserves all original examples and production statements, proves the new regressions, and compares identical benchmark code.
Checklist