[preprocessing] fill_gaps treats multi-quarter frequencies ('2QE', '2QS') as plain quarterly and inserts spurious rows #283

What happened + What you expected to happen

With pandas input, fill_gaps(df, freq="2QE") (or "2QS") on a half-yearly series behaves as if the frequency were "QE": for a 6-period series with one missing period it returns 11 rows with 6 NaNs instead of 6 rows with 1 NaN. No warning is raised. Expected: the same timestamps as pd.date_range(start, end, freq="2QE"). Semi-annual reporting is common in finance, so this silently corrupts such data.

Cause: in utilsforecast/preprocessing.py (id_time_grid), the quarter branch sets n = 3; freq = "M", which discards the offset's multiple (offset.n = 2), so the step becomes 3 months instead of 6.

Suggested fix: n *= 3 instead of n = 3. With that change, 2QE and 2QS match pd.date_range, and the other frequencies I checked (D, 2D, W, W-TUE, 2W, ME, 2ME, MS, 3MS, QE, QS, QE-FEB, QS-FEB, YE, 2YE, YS, YS-JUL, h, 3h, 15min, B) are unchanged. Happy to open a PR adding "2QE"/"2QS" to the frequency parametrization in tests/test_preprocessing.py.

AI disclosure: I found this with the help of an AI assistant (Claude), which also ran the reproduction below and drafted this issue; I reviewed it before posting.

Output of the script below:

utilsforecast 0.2.16 pandas 2.3.3

freq=2QE: input rows=5, got rows=11, expected rows=6
  got ds     : ['2020-03-31', '2020-06-30', '2020-09-30', '2020-12-31', '2021-03-31', '2021-06-30', '2021-09-30', '2021-12-31', '2022-03-31', '2022-06-30', '2022-09-30']
  expected ds: ['2020-03-31', '2020-09-30', '2021-03-31', '2021-09-30', '2022-03-31', '2022-09-30']
  NaN rows inserted: 6 (expected 1)
  equal: False

freq=2QS: input rows=5, got rows=11, expected rows=6
  got ds     : ['2020-01-01', '2020-04-01', '2020-07-01', '2020-10-01', '2021-01-01', '2021-04-01', '2021-07-01', '2021-10-01', '2022-01-01', '2022-04-01', '2022-07-01']
  expected ds: ['2020-01-01', '2020-07-01', '2021-01-01', '2021-07-01', '2022-01-01', '2022-07-01']
  NaN rows inserted: 6 (expected 1)
  equal: False

Versions / Dependencies

utilsforecast 0.2.16 (PyPI; the code is unchanged on main 07aed5f), pandas 2.3.3, Python 3.11, Linux.

Reproducible example

# utilsforecast.preprocessing.fill_gaps (pandas): multi-quarter freqs ('2QE', '2QS', '2Q-DEC' ...)
# are treated as plain quarterly because the quarter branch sets n = 3 instead of n *= 3.
import warnings; warnings.simplefilter("ignore")
import numpy as np, pandas as pd
import utilsforecast
from utilsforecast.preprocessing import fill_gaps, id_time_grid
print("utilsforecast", utilsforecast.__version__, "pandas", pd.__version__)

for freq in ["2QE", "2QS"]:
    # half-yearly (semi-annual reporting) series, with one period missing
    ds = pd.date_range("2020-01-01", periods=6, freq=freq)
    df = pd.DataFrame({"unique_id": "firm", "ds": ds, "y": np.arange(6.0)}).drop(index=2)
    res = fill_gaps(df, freq=freq)
    expected = pd.date_range(ds[0], ds[-1], freq=freq)   # independent: pandas' own calendar
    print(f"\nfreq={freq}: input rows={len(df)}, got rows={len(res)}, expected rows={len(expected)}")
    print("  got ds     :", [str(d.date()) for d in res.ds])
    print("  expected ds:", [str(d.date()) for d in expected])
    print("  NaN rows inserted:", int(res.y.isna().sum()), "(expected 1)")
    print("  equal:", list(res.ds) == list(expected))