fix(eval): stop a comma from swallowing the GSM8K answer - #3484

Summary

GSM8K scores a correct answer as wrong whenever the model's last sentence puts a comma after the final number. _extract_numeric_answer returns an empty string instead of the number, check_answer short-circuits on the empty prediction, and the question is counted wrong (the external-API path reports it as parse_error).

>>> from omlx.eval.gsm8k import _extract_numeric_answer
>>> _extract_numeric_answer("The total is 72 clips, altogether.")
''

Root cause

omlx/eval/gsm8k.py:55. When the response has no #### N marker, extraction falls back to the last number in the text:

numbers = re.findall(r"-?[\d,]+(?:\.\d+)?", text)
if numbers:
    return numbers[-1].replace(",", "")

[\d,]+ needs only one character from the class, so a bare , is a match. On "The total is 72 clips, altogether." findall returns ['72', ',']; the last element is the comma, and .replace(",", "") turns it into "". The number is right there in the text, and the function reports that it found nothing.

The #### branch at line 51 uses the same class and has the same hole (#### , captures a comma).

Fix

Anchor the pattern on a digit at both ends and share it between the two branches:

_NUMBER_PATTERN = r"-?\d(?:[\d,]*\d)?(?:\.\d+)?"

Thousands separators still match, because they sit between digits: 1,234 and 1,234.50 are unchanged. 72, now matches just 72. A lone comma matches nothing, so it can no longer become the "last number".

Verification

tests/test_eval.py::TestGSM8K gains three cases. Red on main at aa8db73, green with the change:

$ python -m pytest tests/test_eval.py -k TestGSM8K -q      # on main + the new tests
FAILED tests/test_eval.py::TestGSM8K::test_extract_numeric_answer_trailing_comma_after_last_number
  assert '' == '72'
FAILED tests/test_eval.py::TestGSM8K::test_extract_numeric_answer_keeps_thousands_separator
  assert '' == '1234.50'
2 failed, 9 passed

$ python -m pytest tests/test_eval.py -q                   # with the change
84 passed in 5.72s

Scoring keys are untouched: I ran both the old and the new extractor over all 1,319 gold answers in omlx/eval/data/gsm8k_test.jsonl and every one of them extracts the same value (they all carry #### N, so they never reached the fallback).

Adjacent suites: pytest tests/test_eval.py tests/test_mbpp_extract_code.py tests/test_accuracy_benchmark.py tests/test_accuracy_upload.py tests/test_admin_external_accuracy_diagnostics.py gives 155 passed.

Full suite with the CI filter, pytest tests/ -m "not slow and not integration" -n 2 on an M-series Mac with Python 3.11.15: 10807 passed, 176 skipped, 0 failed in 5m30s.

ruff check and black --check report the same findings on this file before and after the change (one pre-existing UP045 in get_category, and black wants to reformat an untouched dict literal in load_dataset), so nothing new was introduced.