FoundationForecast is a Python library: one FoundationForecast API over many time-series foundation model wrappers. Human-oriented setup lives in docs/contributing.md.
Use Conventional Commits: type(scope): subject.
- One type per commit (
feat,fix,test,docs,refactor,chore, …). - Prefer separate commits by type (implementation, then tests, then docs)—not one mixed commit per change.
- Before every commit, run pre-commit so hooks match CI lint (
uv run pre-commit run --all-files, oruv run pre-commit runon staged files). Fix hook failures (or include auto-fixed files) before committing.
Releases are SemVer tags on main. There is no rolling CHANGELOG.md; each version gets its own file under docs/changelogs/ (see v0.1.11.md for structure: Features, Fixes, Documentation, Tooling, compare link).
-
Add
docs/changelogs/vMAJOR.MINOR.PATCH.mdand link it at the top ofdocs/changelogs/index.md. -
Register the file in
mkdocs.ymlunder Changelogs (newest first). -
Bump
pyproject.toml[project].version(single source of truth; no__version__in the package). -
Run
uv lock. If dependencies or the editablefoundationforecastpin changed for benchmarks, refresh locks underexperiments/gift-evaland/orexperiments/fev-benchwithuv lockthere too. -
Update user-visible docs for the release when behavior or the model surface changes:
README.md(support matrix, highlights timeline),docs/model-hub.md, API pages underdocs/api/, and examples/notebooks as needed. -
Open a PR titled
chore(release): vX.Y.Z, merge tomain. -
Publish from
main(creates the tag and GitHub Release). Set the GitHub release name with--title(same as the tag, e.g.vX.Y.Z):gh release create vX.Y.Z --title "vX.Y.Z" --discussion-category "General" -F docs/changelogs/vX.Y.Z.md
Pushing the release tag runs:
.github/workflows/release.yaml— ensures the GitHub Release exists, then PyPI publish (uv build/uv publishwith trusted publishing) and post-publish import smoke tests..github/workflows/notify-release.yaml— Discord notification with the same changelog body.
Behavior changes need tests that fail without the change. Treat CI layout as the contract for what must stay green.
uv sync --group dev --group docs
uv run pytest # excludes `docs` and `benchmark` markers (see pyproject.toml)
uv run pre-commit run --all-filesDocs and docstring examples:
uv run pytest -m docs -o addopts= -n 0
uv run --group docs zensical buildWhat CI enforces (.github/workflows/ci.yaml)
| Job | Purpose |
|---|---|
| test | uv run pytest on Python 3.10–3.13; coverage gate (≥80%) on 3.12. Needs HF_TOKEN / TABPFN_TOKEN for model tests. |
| test-docs | Executable markdown in docs/ (except changelog files), README.md, and Python docstrings via mktestdocs. |
| build-docs | zensical build (strict) |
| lint | Full pre-commit (ruff, mypy on foundationforecast/ only—experiments/ excluded) |
| test-gift-eval | Modal GPU run for configs/ci_subset.yaml, then make verify-ci (S3 sync + strict per-job MASE/CRPS vs Hugging Face references). |
| test-fev-bench | Modal CI for known-dynamic tasks, S3 sync, then tests/test_replication.py vs fev-bench leaderboard CSVs. |
Performance regressions: .github/workflows/codspeed.yml (pytest -m benchmark under tests/benchmarks/).
Cross-model contract (tests/models/test_models.py)
Public behavior on model wrappers is exercised for every entry in tests/models/conftest.py models via @pytest.mark.parametrize("model", models).
- New cross-cutting feature (e.g. known-future exog,
level, quantiles, date handling): add or extend parametrized tests intest_models.pyso each model runs the scenario. Use capability checks andpytest.skipwhen a checkpoint legitimately does not support the feature (seesupports_native_futr_exog()in the exog tests)—do not omit models from the parametrization. - New public model: append representative instances to
conftest.pymodelsso existingtest_models.pycoverage applies automatically; add model-specific tests undertests/models/test_<name>.pywhen needed. Followdocs/contributing.mdfor exports, model hub, and API docs.
Also add focused tests where the API is not model-specific: tests/core/, tests/test_foundation_forecast.py. Use @pytest.mark.models when tests download weights (CI caches Hugging Face and TabPFN checkpoints).
Replication (leaderboard MASE/CRPS or fev-bench test_error): extend experiments/gift-eval and/or experiments/fev-bench when a wrapper must match an official submission—not a substitute for test_models.py.
foundationforecast/— library (core/,models/).tests/— unit, integration, docs, and CodSpeed benchmarks.docs/— MkDocs site; changelogs indocs/changelogs/v*.md.experiments/gift-eval/,experiments/fev-bench/— Modal + S3 replication harnesses (editable install of repo root).