Skip to content
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
c371fc4
feat: update site selection and report workflows
dreuzy Jun 6, 2026
b11b6bc
docs: refresh configuration reference
dreuzy Jun 6, 2026
5bc9aa4
examples: refresh site selection workflow configs
dreuzy Jun 6, 2026
17ea2b7
refactor: rename DEM network candidate options
dreuzy Jun 6, 2026
72d58c9
refactor: finalize DEM network naming
dreuzy Jun 6, 2026
bff78b2
refactor(site-selection): align DEM workflow terminology
dreuzy Jun 6, 2026
0737522
docs: refresh config reference diagrams
dreuzy Jun 6, 2026
0f5dc11
test: add validity frame to pytest path
dreuzy Jun 6, 2026
a8d5b1b
Clean up site selection legacy naming
dreuzy Jun 6, 2026
f395311
Document site selection HTML reporting
dreuzy Jun 6, 2026
13c64fd
Complete site selection workflow example
dreuzy Jun 7, 2026
1985c95
docs: expand site selection workflow documentation
dreuzy Jun 7, 2026
f0f586c
docs: align site selection guide example
dreuzy Jun 7, 2026
3a71830
docs: finalize HTML report workflow
dreuzy Jun 7, 2026
e151daa
fix: shorten Windows regression paths
dreuzy Jun 7, 2026
c653431
examples: align site selection testbed filters
dreuzy Jun 7, 2026
3f5e1be
docs: record HTML report validation
dreuzy Jun 7, 2026
f6ce451
chore: ignore generated example data blobs
dreuzy Jun 7, 2026
33e2829
feat: warn on site-selection DEM boundary basins
dreuzy Jun 7, 2026
d04548b
docs: add Finistere map context layer
dreuzy Jun 7, 2026
ddcc912
examples: add regenerated Bretagne testbed configs
dreuzy Jun 7, 2026
3465375
fix: expose validity frame package exports
dreuzy Jun 7, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
docs: finalize HTML report workflow
  • Loading branch information
dreuzy committed Jun 7, 2026
commit 3a71830a5d2c9008fa477464b73b4d08508f497c
89 changes: 76 additions & 13 deletions docs/_dev_notes/html_block_reports_audit.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,10 +49,75 @@ Le chantier n'est pas totalement ferme:
- tous les producteurs ne proposent pas encore la selection de niveau bloc par
bloc;
- le rapport testbed volumineux reste hors migration;
- `network_transient/sections.py` reste present pour compatibilite de tests et
de helpers, meme si le rendu final passe par les blocs;
- il faut encore faire une revue visuelle humaine et maintenir la courte
documentation "comment creer un rapport HTML par blocs" dans RTD.
- la revue de livraison ciblee du rapport bassin Nancon et du rapport
site-selection est faite, mais les gros cas regionaux restent des revues
produit separees;
- la courte documentation "comment creer un rapport HTML par blocs" dans RTD
doit rester maintenue avec les evolutions du socle.

## Livraison ciblee 2026-06-07

Cette reprise clot le lot "HTML en fin de simulation" pour le profil
`catchment_gauged` et le nettoyage legacy associe.

Etat du code actif:

- le pipeline de simulation termine par `DisplayStep()` puis `HtmlReportStep()`;
- `HtmlReportStep` construit uniquement le profil `catchment_gauged`;
- `site_selection` reste gere par son workflow dedie, en reutilisant
l'intention `[report.html]`;
- `generic_simulation` reste reserve dans le schema mais sans builder livre;
- les anciens wrappers de compatibilite
`hydromodpy/display/catchment_report/semantic_artifacts.py` et
`hydromodpy/reporting/site_selection/intent.py` ont ete supprimes;
- les imports actifs pointent directement vers
`hydromodpy.display.report_semantics` ou vers
`cfg.report_html_build_at_end`.

Validation automatisee relancee:

```powershell
python -m pytest tests/unit/display/test_report_blocks_html.py tests/unit/display/test_catchment_report_artifact_contract.py tests/unit/display/test_catchment_report_pipeline.py tests/unit/display/test_catchment_report_postflight.py tests/unit/display/test_catchment_report_settings.py tests/unit/display/test_catchment_report_docs_contract.py tests/unit/display/test_catchment_report_examples_contract.py tests/unit/display/test_report_config.py tests/unit/display/test_report_artifacts.py tests/unit/test_pipeline_html_report_step.py tests/unit/test_pipeline_display_step.py tests/unit/cli/test_report_catchment.py -q
```

Resultat: `80 passed`.

```powershell
python -m pytest tests/unit/site_selection/test_manifest_report.py tests/unit/site_selection/test_workflow_plan_run_workflow.py tests/unit/site_selection/test_workflow_plan_planning.py tests/unit/site_selection/test_example_configs.py tests/unit/site_selection/test_legacy_contract.py tests/unit/site_selection/test_synthetic_spatial_review.py -q
```

Resultat: `33 passed`.

Revue HTML locale:

```powershell
python -m hydromodpy report catchment examples/projects/16_nancon_natural_calibration/catchment_report_transient_nwt_html.toml --report-only
```

Sorties a utiliser pour la revue finale:

```text
examples/projects/16_nancon_natural_calibration/outputs/nancon_transient_nwt_html_report/web/index.html
examples/projects/16_nancon_natural_calibration/outputs/nancon_transient_nwt_html_report/web_review/by_block/index.html
examples/projects/16_nancon_natural_calibration/outputs/nancon_transient_nwt_html_report/web_review/compact/index.html
examples/projects/16_nancon_natural_calibration/outputs/nancon_transient_nwt_html_report/web_review/standard/index.html
examples/projects/16_nancon_natural_calibration/outputs/nancon_transient_nwt_html_report/web_review/audit/index.html
examples/projects/16_nancon_natural_calibration/outputs/nancon_transient_nwt_html_report/block_report_postflight.json
```

Le postflight de cette sortie indique:

```text
expected_count = 22
present_count = 22
missing_count = 0
```

Point d'attention: `outputs/nancon_real_figures` appartient a l'ancien exemple
de rapport manuel `catchment_report.toml`, cible `simulation_name =
"transient_nwt"` et ne doit pas servir de validation finale pour le chantier
`[report.html]` de fin de simulation. Il peut signaler des figures manquantes
car il ne cible pas le run `transient_nwt_html_report`.

## Validation plug-and-play au 2026-05-24

Expand Down Expand Up @@ -1520,15 +1585,13 @@ python examples/projects/16_nancon_natural_calibration/build_nancon_real_figures

### Reste a faire

- Decider si `network_transient/sections.py` doit rester comme compatibilite de
tests/helpers ou etre progressivement deprecie.
- `network_transient/sections.py` n'existe plus dans le code actif; il n'y a
plus de decision de compatibilite a prendre sur ce fichier.
- Reporter la migration de `generate_testbed_web_report.py`: le fichier est
trop volumineux pour etre migre dans le meme lot.
- Eventuellement ajouter un guide court dans la documentation developpeur:
"comment creer un rapport HTML par blocs".
- Faire une revue visuelle humaine des pages generees dans `%TEMP%`, notamment:
- lisibilite du sommaire;
- densite des metriques;
- pertinence des libelles du rapport calibration naturel/B0;
- taille acceptable de la carte site-selection embarquee.
- Le guide developpeur court existe maintenant:
`docs/source/architecture/how-to/add-a-block-html-report.rst`.
- Revue visuelle ciblee faite pour les pages de livraison Nancon et
site-selection. Les prochaines revues visuelles doivent porter sur des cas
produit precis, pas sur le refactoring HTML lui-meme.
- Appliquer ensuite le meme pattern au rapport testbed, dans un lot separe.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,9 @@ workflow validates that the explicit profile is either omitted or equal to
``site_selection``, then stores the request on the workflow-local
``SiteSelectionConfig.report_html_build_at_end`` flag.

Older site-selection-specific switches such as ``write_report_html`` are not
part of the current public contract. Example TOMLs should use ``[report.html]``
so report intent is expressed the same way as other HydroModPy workflows.
Older site-selection-specific report switches are not part of the current
public contract. Example TOMLs should use ``[report.html]`` so report intent is
expressed the same way as other HydroModPy workflows.

Completed Run Flow
------------------
Expand Down
83 changes: 83 additions & 0 deletions docs/source/user_guide/catchment-report.rst
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,40 @@ TOML, manifest, or regenerated file fingerprints rather than committing the full
HTML and figure tree. The full report can still be inspected locally after a
run through the printed ``html_report`` path.

What the report shows
---------------------

The Nancon example illustrates the kind of evidence a catchment report brings
together: spatial context, reference hydrography, observed or simulated series,
and simulation diagnostics. These figures are generated report artifacts, copied
here only as documentation illustrations.

.. list-table::
:widths: 50 50

* - .. figure:: /_static/user_guide/catchment_report/nancon_map_dem_context.png
:alt: Nancon DEM context map.
:width: 100%

Catchment and DEM context.

- .. figure:: /_static/user_guide/catchment_report/nancon_hydrographic_network_comparison.png
:alt: Nancon hydrographic network comparison.
:width: 100%

Reference and generated hydrographic networks.
* - .. figure:: /_static/user_guide/catchment_report/nancon_timeseries_discharge.png
:alt: Nancon discharge time series.
:width: 100%

Discharge time series used for report interpretation.

- .. figure:: /_static/user_guide/catchment_report/nancon_simulated_active_network_reference_overlay.png
:alt: Nancon simulated active network overlay.
:width: 100%

Simulated active network compared with the reference network.

TOML contract
-------------

Expand Down Expand Up @@ -236,6 +270,28 @@ duplicating the lower-level pipeline switches. ``build_at_end = true`` implies
profile = "catchment_gauged"
config_path = "../16_nancon_natural_calibration/catchment_report_transient_nwt_html.toml"

Current profile support is intentionally narrow:

.. list-table::
:header-rows: 1
:widths: 24 24 52

* - Profile
- Status
- Builder
* - ``catchment_gauged``
- Supported for simulation runs
- The simulation pipeline renders display artifacts, then
``HtmlReportStep`` builds the catchment report from ``config_path``.
* - ``site_selection``
- Supported by the site-selection workflow
- The site-selection workflow reads the same ``[report.html]`` intent but
uses its own manifest-driven renderer.
* - ``generic_simulation``
- Reserved
- The profile is accepted by the configuration schema for future use, but
no end-of-run HTML builder is shipped for it yet.

Using ``enabled = true`` without ``build_at_end`` means that the run should
prepare the report artifacts but the final HTML can be built later from the
manifest.
Expand Down Expand Up @@ -325,6 +381,33 @@ The Nancon example uses the same generic report contract as any other basin:
path = "../../data/hydrometry/hydrometry_custom_NANCON_19820201_20220125_D.csv"
station_id = "NANCON"

This lower-level example is useful when rebuilding the report manually from
the historical ``transient_nwt`` artifacts. It is not the recommended
validation target for the end-of-run ``[report.html]`` path.

For the optional HTML report built at the end of a simulation, use the Nancon
overlay that points to the display-artifact run:

.. code-block:: toml

base_config = "catchment_report.toml"

[report]
output_dir = "outputs/nancon_transient_nwt_html_report"

[layout]
simulation_name = "transient_nwt_html_report"
transient_config_name = "run_transient_nwt_html_report.toml"

The corresponding report-only rebuild is:

.. code-block:: bash

hmp report catchment examples/projects/16_nancon_natural_calibration/catchment_report_transient_nwt_html.toml --report-only

Its expected final check is ``missing_count = 0`` in
``outputs/nancon_transient_nwt_html_report/block_report_postflight.json``.

Adding a New Basin
------------------

Expand Down
2 changes: 1 addition & 1 deletion hydromodpy/display/catchment_report/artifacts.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@
GEOLOGY_DATA_ROOT,
REPO_ROOT,
)
from hydromodpy.display.catchment_report.semantic_artifacts import semantic_artifact_id
from hydromodpy.display.report_artifacts import ReportArtifactIndex
from hydromodpy.display.report_semantics import semantic_artifact_id


@dataclass(frozen=True)
Expand Down
2 changes: 1 addition & 1 deletion hydromodpy/display/catchment_report/context.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,12 @@

from hydromodpy.display.catchment_report.inputs import CatchmentReportInputs
from hydromodpy.display.catchment_report.resources import REPO_ROOT
from hydromodpy.display.catchment_report.semantic_artifacts import semantic_artifact_id
from hydromodpy.display.report_artifacts import (
ReportArtifact,
ReportArtifactManifest,
ReportArtifactRequirement,
)
from hydromodpy.display.report_semantics import semantic_artifact_id

REPORT_ARTIFACT_MANIFEST_NAME = "report_artifact_manifest.json"

Expand Down
8 changes: 4 additions & 4 deletions hydromodpy/display/catchment_report/contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,15 @@
)
from hydromodpy.display.catchment_report.block_specs import ReportBlockSpec
from hydromodpy.display.catchment_report.resources import REPO_ROOT
from hydromodpy.display.catchment_report.semantic_artifacts import (
SEMANTIC_ARTIFACT_ID_BY_FIGURE_ID,
semantic_artifact_id,
)
from hydromodpy.display.report_artifacts import (
ReportArtifact,
ReportArtifactManifest,
ReportArtifactRequirement,
)
from hydromodpy.display.report_semantics import (
SEMANTIC_ARTIFACT_ID_BY_FIGURE_ID,
semantic_artifact_id,
)

CATCHMENT_GAUGED_PROFILE = "catchment_gauged"
REPORT_ARTIFACT_MANIFEST_NAME = "report_artifact_manifest.json"
Expand Down
13 changes: 0 additions & 13 deletions hydromodpy/display/catchment_report/semantic_artifacts.py

This file was deleted.

1 change: 0 additions & 1 deletion hydromodpy/reporting/site_selection/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@ manifest.
- `blocks.py`: report-block construction and detail-level variants.
- `figures.py`: static map rendering from manifest-declared spatial artifacts.
- `plan.py`: dry-run and planning report helpers.
- `intent.py`: small shared report-intent definitions.

## Invariants

Expand Down
14 changes: 0 additions & 14 deletions hydromodpy/reporting/site_selection/intent.py

This file was deleted.

2 changes: 1 addition & 1 deletion hydromodpy/reporting/streamlit_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -350,7 +350,7 @@ def _fmt_toml(val: Any) -> str:
if isinstance(val, str):
return f'"{val}"'
if isinstance(val, Path):
return f'"{val}"'
return f'"{val.as_posix()}"'
if isinstance(val, (list, tuple)):
inner = ", ".join(_fmt_toml(i) for i in val)
return f"[{inner}]"
Expand Down
3 changes: 3 additions & 0 deletions hydromodpy/spatial/site_selection/pipelines/build.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ def build_site_selection_from_point_records(
config: SiteSelectionConfig,
point_records: Iterable[Any],
dem_init_path: str | Path | None = None,
map_dem_path: str | Path | None = None,
output_root: str | Path | None = None,
crs_project: str | None = None,
backend: object | None = None,
Expand Down Expand Up @@ -172,6 +173,8 @@ def build_site_selection_from_point_records(
flow_manifest["dem_path"] = str(dem_path)
flow_manifest["dem_source"] = config.dem.source
flow_manifest["intermediate_rasters_kept"] = config.output.keep_intermediate_rasters
if map_dem_path is not None:
flow_manifest["map_dem_path"] = str(Path(map_dem_path).expanduser().resolve())
if reference_bundle is not None:
flow_manifest["reference_network"] = reference_bundle.to_manifest_record()
output_paths.update(
Expand Down
24 changes: 19 additions & 5 deletions hydromodpy/workflow/site_selection.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,6 @@
from hydromodpy.data.variables.dem.config import DemConfig as DataDemConfig
from hydromodpy.data.variables.hydrometry.config import HydrometryConfig
from hydromodpy.reporting.site_selection.html import render_site_selection_html_report
from hydromodpy.reporting.site_selection.intent import (
site_selection_report_html_requested,
)
from hydromodpy.reporting.site_selection.plan import (
render_site_selection_plan_html_report,
)
Expand Down Expand Up @@ -1207,6 +1204,22 @@ def build_observed_site_selection_from_toml(
candidate_outlets=candidate_outlets,
)
_emit_progress(progress_callback, f"{cfg.selection_id}: calculation DEM ready: {dem_path}")
map_dem_path = None
if cfg.dem.review_map_dem_background not in {"none", "delineation_dem"}:
_emit_progress(progress_callback, f"{cfg.selection_id}: resolving review-map DEM")
map_dem_record = _maybe_resolve_map_dem_for_review(
config=cfg,
catchments=[],
config_path=path,
dem_loader=dem_loader,
)
map_dem_value = map_dem_record.get("map_dem_path")
if map_dem_value:
map_dem_path = Path(str(map_dem_value)).expanduser().resolve()
_emit_progress(
progress_callback,
f"{cfg.selection_id}: review-map DEM ready: {map_dem_path}",
)
_emit_progress(
progress_callback,
f"{cfg.selection_id}: building flow products, catchments and report artifacts",
Expand All @@ -1215,6 +1228,7 @@ def build_observed_site_selection_from_toml(
config=cfg,
point_records=records,
dem_init_path=dem_path,
map_dem_path=map_dem_path,
backend=backend,
flow_products_builder=flow_products_builder,
delineation_builder=delineation_builder,
Expand Down Expand Up @@ -1366,7 +1380,7 @@ def run_site_selection_workflow(config_path: str | Path) -> dict[str, Any]:
plan = plan_site_selection(path)
manifest_path = plan.write_manifest() if cfg.input.write_plan_manifest else None
report_path = None
if site_selection_report_html_requested(cfg):
if cfg.report_html_build_at_end:
if manifest_path is None:
manifest_path = plan.write_manifest()
report_path = render_site_selection_plan_html_report(manifest_path)
Expand Down Expand Up @@ -1724,7 +1738,7 @@ def _planned_outputs(cfg: SiteSelectionConfig) -> list[str]:
if cfg.output.write_regional_lab_csv:
outputs.append("regional_lab_csv")
outputs.append("report_artifact_manifest")
if site_selection_report_html_requested(cfg):
if cfg.report_html_build_at_end:
outputs.append("report_html")
return outputs

Expand Down
1 change: 1 addition & 0 deletions tests/unit/architecture/layer_matrix.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ tolerances:
- {src: data, tgt: results, reason: "cross-DB ATTACH bridge: data.DataEntry.used_by reads results.cross_db (commit 1bd5f31ef)"}
- {src: results, tgt: data, reason: "cross-DB ATTACH bridge: results.cross_db reads DataCatalogDuckDB via ATTACH read-only (commit 1bd5f31ef)"}
- {src: spatial, tgt: data, reason: "site-selection BD Topage outlet snapping delegates the optional hydrography fetch to data managers through a narrow helper"}
- {src: spatial, tgt: display, reason: "site-selection outputs write a generic report-artifact manifest consumed by display HTML report builders"}
- {src: calibration, tgt: display, reason: "network-transient calibration diagnostics reuse the shared static HTML report-block renderer"}

# Files exempt from the layer rule. Bootstrap shims that wire forward refs,
Expand Down
Loading