Corrected alignment of the OnBoard page to accommodate various screen sizes - #6

Corrected alignment of the OnBoard page to accommodate various screen sizes

* UN-3450 [FEAT] Add golden-path smoke test for callback worker

Adds workers/tests/test_callback_sanity.py mirroring the existing
test_executor_sanity.py pattern. Covers:

- Worker enums and registry (WorkerType.CALLBACK, QueueName.CALLBACK +
  CALLBACK_API, TaskName.PROCESS_BATCH_CALLBACK, health port 8083,
  WorkerRegistry queue config + task routing).
- Celery task wiring (process_batch_callback, process_batch_callback_api,
  healthcheck — registration, retry config, max_retries, autoretry_for).
- Full dispatch -> task -> return round-trip via Celery eager mode,
  using the simple healthcheck task (process_batch_callback itself
  needs a configured InternalAPIClient + downstream HTTP, which is
  heavy mocking territory unsuitable for a smoke test — covered later
  in #1.2 characterisation suite).

Coverage gains on callback module:
- callback/__init__.py:  0% -> 100%
- callback/worker.py:    0% -> 52%
- callback/tasks.py:     0% -> 10%
- Module total:          0% -> 12%

14 tests, run in ~17s. Regression net for spine PRs #6 (CallbackStatus
enum migration) and #13 (chord -> Barrier lift in callback chain).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* UN-3450 [FEAT] Address Greptile review on callback smoke test

Two P2 findings from Greptile, both fixed:

1. test_healthcheck_result_is_json_serializable: drop the `default=str`
   fallback in json.dumps. With it, any non-JSON-serializable value
   (UUID, datetime, custom object) gets silently coerced to a string —
   exactly the failure mode the test claims to catch. Without it, the
   assertion faithfully tests "round-trips cleanly via JSON" the way
   Celery's serializer would.

2. Add registration check for `process_batch_callback_django_compat`
   (the backward-compat task name `workflow_manager.workflow_v2.
   file_execution_tasks.process_batch_callback`). Both this task and
   `process_batch_callback` delegate to `_process_batch_callback_core`,
   so refactors that touch the core (e.g. the upcoming CallbackStatus
   enum migration) affect this path too. Three-line addition.

15 tests now (was 14), runtime 11s.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…test-infra fix)

Seven of Vishnu's findings against ``524ae9184`` addressed. Three
flagged IMPORTANT (silent-failure + missing test coverage), four
SUGGESTION (drift hazard, comment/behaviour mismatch, hollow canary,
duplicate-test cleanup). The other three (hand-built fixture,
SDK1 ``dict[str, Any]`` boundary, ``as_header`` TypedDict refactor)
deferred — see PR thread acknowledgments.

* **#11a (SUGGESTION, drift)** — ``workers/queue_backend/dispatch.py``
  still hand-built the fairness header instead of calling
  ``fairness.as_header()``. Wire-format encoding now has a single
  source so the two sites can't drift. ``FAIRNESS_HEADER_NAME``
  import dropped (no longer used here).

* **#9 (SUGGESTION, comment/behaviour mismatch)** —
  ``ExecutionDispatcher._build_send_kwargs`` was forwarding ``{}``
  as-is despite the docstring calling it "likely indicates a
  producer-side build bug". Changed ``if headers is not None`` ->
  ``if headers``: falsy is dropped so the on-wire shape matches the
  no-headers baseline and a miswired producer surfaces immediately.
  Docstring rewritten to describe the new contract.

* **#3 (SUGGESTION, wording)** — ``canary_helpers`` docstring
  overstated adoption. Softened to "intended single home" and
  named the two characterisation walkers still inlining the logic
  as a known follow-up.

* **#5 + #6 (IMPORTANT cleanup + SUGGESTION fixture)** — six
  structurally-identical ``*_forwards_headers`` /
  ``*_omits_headers_when_none`` tests collapsed to three
  parametrized methods over the three dispatch entry points.
  Fixture now uses ``FairnessKey(...).as_header()`` rather than
  hand-built dicts, so the wire shape exercised matches what real
  producers emit (including ``pipeline_priority``). Net: ~60 LOC
  removed, per-method failure granularity preserved via parametrize
  IDs. Also added empty-dict drop assertions covering #9.

* **#7 (SUGGESTION, missing combined test)** — new
  ``test_dispatch_with_callback_combines_headers_and_callbacks``
  passes ``on_success``, ``on_error``, ``task_id``, and ``headers``
  together and asserts all four land on the same ``send_task``
  call. A key-merge regression in ``_build_send_kwargs`` would
  have slipped through the single-kwarg forwarding tests.

* **#8 (SUGGESTION, hollow canary)** — the
  ``execute_extraction`` dispatch canary only ever asserted the
  empty (passing) case against the live tree. Added a positive-
  detection unit test feeding ``ast.parse`` of a known-bad snippet
  and a blind-spot lock test (constant ref, f-string,
  ``apply_async`` all evade the detector — documenting the scope
  so a future widening intentionally trips the asserts).

* **#4 (IMPORTANT, untested helper)** — ``_fairness_headers`` in
  ``structure_tool_task`` was untested; a regression flipping
  ``NON_API`` -> ``API`` or dropping ``headers=`` at any of the
  three call sites would have stayed green. Added focused unit
  tests in new ``test_structure_tool_task.py`` (wire shape,
  org_id propagation, ``NON_API`` not ``API``) and extended
  ``test_sanity_phase5.TestStructureToolSingleDispatch`` to assert
  ``dispatch.call_args.kwargs["headers"]`` carries the expected
  shape.

* **#2 (IMPORTANT, vacuous-pass)** — ``iter_production_trees``
  warned-and-continued on ``SyntaxError`` but neither canary
  module promoted the warning to error. A botched merge in a
  production file would have dropped silently from the audit set
  and every canary would have passed vacuously over a smaller
  tree. Added ``pytestmark = pytest.mark.filterwarnings(
  "error::UserWarning")`` on both ``test_executor_dispatch`` and
  ``test_fairness_key``, plus a new ``test_canary_helpers.py``
  that unit-tests both the warn-on-broken behaviour and the
  promote-to-error contract the canary modules rely on.

**Bundled test-infra fix (unrelated but unblocks CI):** the
``test_callback_sanity.TestEagerHealthcheckRoundTrip`` tests
selected the healthcheck task via ``endswith(".healthcheck")``
against ``eager_app.tasks``, which is a shared celery global
registry containing ``callback.worker.healthcheck``,
``executor.worker.healthcheck``,
``file_processing.worker.healthcheck`` etc. The bare ``next(...)``
returned whichever was inserted first — non-deterministic across
pytest module-collection orders. Without this fix, the new tests
added in this commit perturb the collection profile enough to
flip the failure rate from ~10% to nearly 100%. Replaced with
exact-name lookup ``name == "callback.worker.healthcheck"``.
Identical fix already landed on the UN-3513 branch (see #2020).

Test count: 31 -> 42 on the UN-3508-touched modules. Full workers
suite: 6 failures pre-existing baseline, unchanged by this commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* UN-3508 [FEAT] Plumb fairness through ExecutionDispatcher

Phase 5.2 of the PG Queue rollout (epic UN-3445). Adds fairness-header
support to the third dispatch path (sdk1's ExecutionDispatcher) so
``execute_extraction`` tasks emitted by file_processing carry the same
routing metadata as workflow-execution dispatches that go through
queue_backend.

What

* sdk1/execution/dispatcher.py: ``dispatch``, ``dispatch_async``,
  ``dispatch_with_callback`` all accept an optional ``headers`` kwarg.
  When non-None, forwarded to Celery's send_task; when None, omitted
  so the call shape stays identical to pre-Phase-5.2 for callers that
  don't opt in (sdk1's existing tests remain green unchanged).
* queue_backend/fairness.py: new ``FairnessKey.as_header()`` method
  returns the wire-ready ``{"x-fairness-key": ...}`` dict. Producers
  no longer need to reference ``FAIRNESS_HEADER_NAME`` directly —
  keeps the additive-only canary in test_fairness_key.py happy.
* file_processing/structure_tool_task.py: small ``_fairness_headers``
  helper builds the header (defaulting workload_type to NON_API;
  propagating the real type is Phase 6 work). All three
  ``dispatcher.dispatch(...)`` sites (lines 468, 507, 720) now pass
  ``headers=_fairness_headers(organization_id)``.
* tests/test_executor_dispatch.py: new file. Covers header forwarding
  through all three dispatcher methods (including the "omit when
  None" pre-existing shape preservation), the FairnessKey.as_header()
  shape, and an AST inventory canary that forbids raw
  ``*.send_task("execute_extraction", ...)`` outside
  ExecutionDispatcher.

Why

UN-3501 plumbed fairness on bare dispatch() call sites. The
``execute_extraction`` task is the most workflow-execution-y dispatch
in the codebase but bypasses queue_backend (uses ExecutionDispatcher
directly), so it had no fairness header. The canary in
test_fairness_key.py audits only bare-name dispatch() and missed it.

No regression risk

* Additive: ``headers`` is optional and defaults to None on all three
  dispatcher methods; the existing 78 sdk1 tests pass unchanged.
* Producer-side only — no consumer reads ``x-fairness-key`` yet.
* No queue routing, task name, or args/kwargs change.

Test count: workers seam suite 53 -> 60 (new test_executor_dispatch.py
with 7 tests). sdk1 dispatcher suite 80/80 green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* UN-3508 [REFACTOR] Extract shared canary helpers to drop SonarCloud duplication

SonarCloud flagged 7.8% duplicated lines in the new
test_executor_dispatch.py — the file-walking helper and skip-dir
constants were copy-pasted from test_fairness_key.py.

Move them into tests/canary_helpers.py:

* WORKERS_ROOT, DEFAULT_SKIP_TOP_DIRS constants.
* iter_production_trees(skip_top_dirs=…) generator.

Both canary tests use relative imports (from .canary_helpers import …)
to keep one canonical import path — tests/ is already a package via
__init__.py, no pyproject change needed. (An earlier attempt added
pythonpath = ["tests"], reverted — it would have created a second
top-level import path for every test file and a dual-module-object
hazard.)

The fairness canary widens its skip set with ``queue_backend`` (where
the seam legitimately defines fairness constants); the executor canary
keeps the default. Tests stay at 60/60 — pure dedup, no behavioural
change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* UN-3508 [FIX] Address 14 PR review findings (HIGH/MED/NIT)

* dispatcher.py: factor _build_send_kwargs helper; document headers kwarg on dispatch_with_callback; reference FAIRNESS_HEADER_NAME symbol instead of bare string; document empty-dict caller-bug semantic
* structure_tool_task.py: narrow _fairness_headers return type; replace 'Phase 6 work' with TODO(UN-3504) anchor
* fairness.py: concrete as_header() docstring with explicit shape
* canary_helpers.py: surface SyntaxError via UserWarning (real silent-failure bug; canaries no longer pass vacuously on unparseable files)
* test_executor_dispatch.py: switch to dict[str, Any] dropping type-ignore; use WorkloadType.NON_API.value instead of invalid 'etl' literal; new test_dispatch_async_omits_headers_when_none; tighten canary docstring + note blind spots; drop plan-stage vocab; reorder relative import; new test_fairness_header_shape_orgless for org_id=None case

Tests: workers 60 -> 62, sdk1 dispatcher 80/80 green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* UN-3508 [DOCS] Fix iter_production_trees docstring: 'Yield' -> 'Return a list'

Greptile P2: function builds and returns a list — it is not a
generator — but the docstring opened with 'Yield ...', which would
mislead a reader into expecting lazy consumption / generator semantics
(early break, send(), etc.).

Pure docstring fix, no behaviour change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* UN-3508 [FIX] Address vishnuszipstack review (7 real fixes + bundled test-infra fix)

Seven of Vishnu's findings against ``524ae9184`` addressed. Three
flagged IMPORTANT (silent-failure + missing test coverage), four
SUGGESTION (drift hazard, comment/behaviour mismatch, hollow canary,
duplicate-test cleanup). The other three (hand-built fixture,
SDK1 ``dict[str, Any]`` boundary, ``as_header`` TypedDict refactor)
deferred — see PR thread acknowledgments.

* **#11a (SUGGESTION, drift)** — ``workers/queue_backend/dispatch.py``
  still hand-built the fairness header instead of calling
  ``fairness.as_header()``. Wire-format encoding now has a single
  source so the two sites can't drift. ``FAIRNESS_HEADER_NAME``
  import dropped (no longer used here).

* **#9 (SUGGESTION, comment/behaviour mismatch)** —
  ``ExecutionDispatcher._build_send_kwargs`` was forwarding ``{}``
  as-is despite the docstring calling it "likely indicates a
  producer-side build bug". Changed ``if headers is not None`` ->
  ``if headers``: falsy is dropped so the on-wire shape matches the
  no-headers baseline and a miswired producer surfaces immediately.
  Docstring rewritten to describe the new contract.

* **#3 (SUGGESTION, wording)** — ``canary_helpers`` docstring
  overstated adoption. Softened to "intended single home" and
  named the two characterisation walkers still inlining the logic
  as a known follow-up.

* **#5 + #6 (IMPORTANT cleanup + SUGGESTION fixture)** — six
  structurally-identical ``*_forwards_headers`` /
  ``*_omits_headers_when_none`` tests collapsed to three
  parametrized methods over the three dispatch entry points.
  Fixture now uses ``FairnessKey(...).as_header()`` rather than
  hand-built dicts, so the wire shape exercised matches what real
  producers emit (including ``pipeline_priority``). Net: ~60 LOC
  removed, per-method failure granularity preserved via parametrize
  IDs. Also added empty-dict drop assertions covering #9.

* **#7 (SUGGESTION, missing combined test)** — new
  ``test_dispatch_with_callback_combines_headers_and_callbacks``
  passes ``on_success``, ``on_error``, ``task_id``, and ``headers``
  together and asserts all four land on the same ``send_task``
  call. A key-merge regression in ``_build_send_kwargs`` would
  have slipped through the single-kwarg forwarding tests.

* **#8 (SUGGESTION, hollow canary)** — the
  ``execute_extraction`` dispatch canary only ever asserted the
  empty (passing) case against the live tree. Added a positive-
  detection unit test feeding ``ast.parse`` of a known-bad snippet
  and a blind-spot lock test (constant ref, f-string,
  ``apply_async`` all evade the detector — documenting the scope
  so a future widening intentionally trips the asserts).

* **#4 (IMPORTANT, untested helper)** — ``_fairness_headers`` in
  ``structure_tool_task`` was untested; a regression flipping
  ``NON_API`` -> ``API`` or dropping ``headers=`` at any of the
  three call sites would have stayed green. Added focused unit
  tests in new ``test_structure_tool_task.py`` (wire shape,
  org_id propagation, ``NON_API`` not ``API``) and extended
  ``test_sanity_phase5.TestStructureToolSingleDispatch`` to assert
  ``dispatch.call_args.kwargs["headers"]`` carries the expected
  shape.

* **#2 (IMPORTANT, vacuous-pass)** — ``iter_production_trees``
  warned-and-continued on ``SyntaxError`` but neither canary
  module promoted the warning to error. A botched merge in a
  production file would have dropped silently from the audit set
  and every canary would have passed vacuously over a smaller
  tree. Added ``pytestmark = pytest.mark.filterwarnings(
  "error::UserWarning")`` on both ``test_executor_dispatch`` and
  ``test_fairness_key``, plus a new ``test_canary_helpers.py``
  that unit-tests both the warn-on-broken behaviour and the
  promote-to-error contract the canary modules rely on.

**Bundled test-infra fix (unrelated but unblocks CI):** the
``test_callback_sanity.TestEagerHealthcheckRoundTrip`` tests
selected the healthcheck task via ``endswith(".healthcheck")``
against ``eager_app.tasks``, which is a shared celery global
registry containing ``callback.worker.healthcheck``,
``executor.worker.healthcheck``,
``file_processing.worker.healthcheck`` etc. The bare ``next(...)``
returned whichever was inserted first — non-deterministic across
pytest module-collection orders. Without this fix, the new tests
added in this commit perturb the collection profile enough to
flip the failure rate from ~10% to nearly 100%. Replaced with
exact-name lookup ``name == "callback.worker.healthcheck"``.
Identical fix already landed on the UN-3513 branch (see #2020).

Test count: 31 -> 42 on the UN-3508-touched modules. Full workers
suite: 6 failures pre-existing baseline, unchanged by this commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…ing nit)

Seven of Vishnu's PR review findings addressed, all backward-compat with
main-branch consumers. The three [Important] design-redesign findings
(#1 status __post_init__, #2 alias-pair invariant, #3 to_api_dict/to_json
dead code) are deferred to a follow-up shared-infra dataclass ticket
because they would either fire warning noise on existing call sites
(``worker_base.py:211/222``, ``worker_patterns.py:241`` pass wrong-enum
status) or change the wire/cache contract — neither acceptable
mid-flight while keeping zero regression on main.

Changes in this commit are either:
  * Pure additive (test methods, docstrings, observability)
  * Or provably equivalent wire output (the typed-count refactor)

So a rolling deploy where old workers and new workers run concurrently
sees identical wire shapes and identical behaviour for all current
valid data; the only observable differences are log content (better
context on the existing warning) and the presence of a new opt-in
classmethod that nothing currently calls.

* **Vishnu #8 [Suggestion]** — ``SkipReason`` docstring claimed
  "StrEnum semantics" but the class is ``(str, Enum)``, not
  ``enum.StrEnum``. The two differ on ``__str__``. Rewrote the
  docstring to describe the actual behaviour.

* **Vishnu #4a [Important — log context]** — ``_parse_skipped``
  now accepts an optional ``file_execution_id`` kwarg that
  ``from_dict`` threads through. The warning emitted for unknown
  wire values now carries the file identifier, so a real
  rolling-deploy incident is debuggable rather than a context-free
  warning. Optional kwarg with default — any existing caller passing
  one positional arg still works.

* **Vishnu #9 [Suggestion]** — added
  ``BatchExecutionResult.from_file_results(...)`` classmethod that
  derives counters from typed file results. Purely additive: no
  existing caller uses it; the constructor signature is unchanged
  so producers that need their own counter semantics keep working.

* **Vishnu #11 [Suggestion]** — ``process_file_batch_api`` was
  computing ``skipped_already_completed`` by string-matching the
  wire dicts AFTER already calling ``from_dict`` on them. Refactored
  to count from the typed list (single ``from_dict`` pass, enum
  compare). Provably equivalent for all current wire data.

* **Vishnu #4 [Important — test gap]** — added
  ``test_from_dict_unknown_skipped_is_lenient`` covering the one
  documented crash-prevention path. A regression to bare
  ``SkipReason(raw)`` would have re-introduced the rolling-deploy
  crash and kept every other test green.

* **Vishnu #5 [Important — failure-aggregation gap]** — added
  ``test_process_file_batch_api_batch_wrapper_failure_aggregation``
  that drives one success + one failure through the batch wrapper.
  The existing success-only test never exercised
  ``failed_files += 1``.

* **Vishnu #6 [Important — populated round-trip gap]** — added
  ``test_round_trip_with_populated_file_results`` and
  ``test_from_file_results_derives_counters``. The existing
  ``BatchExecutionResult`` round-trip test used
  ``file_results=[]``, so the list-comprehension in ``from_dict``
  that rebuilds nested ``FileExecutionResult`` objects was never
  executed with a populated list.

* **Vishnu #13 [Suggestion]** — replaced hardcoded line reference
  in test docstring with a symbol reference.

Deferred to follow-up shared-infra dataclass-redesign ticket:
  * #1 ``__post_init__`` status clobber — would emit warning noise
    on every existing wrong-enum call site
  * #2 alias-pair invariant — back-fill via __post_init__ would
    change the wire shape (file_name no longer None → no longer
    stripped at the top level)
  * #3 ``to_api_dict``/``to_json`` dead code — looks like a public
    SDK surface; changing the body could surprise external consumers
  * #7 recursive ``None``-strip in ``serialize_value`` — touches
    every dataclass in the codebase
  * #10 ``Any`` typing tightening — low value, mypy tightening could
    trip downstream
  * #12 producer redundant kwargs — depends on #2's reconciliation

Tests: workers chord-callback boundary suite 21 -> 25; full workers
suite 622 -> 627 (no new failures; 6 pre-existing baseline
unchanged). Five deterministic-order runs of the full suite returned
exactly 627 passed / 6 pre-existing failed — zero flakiness from
this change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

… / FileExecutionResult (#2020)

* UN-3513 [FEAT] Type chord-callback boundary with BatchExecutionResult / FileExecutionResult

Producers in workers/file_processing/tasks.py now build typed
dataclasses (from unstract.core.worker_models) and emit their
``.to_dict()`` instead of hand-rolled dicts. Locks the wire shape to
the dataclass schema so downstream refactors fail loud.

Scope

Producer-side typing only. Consumer (workers/callback/tasks.py +
aggregate_file_batch_results) already reads via ``.get(..., default)``
— tolerant by construction — so no consumer-side change needed.

Dataclass extensions (unstract.core.worker_models, additive only)

* BatchExecutionResult gains 3 optional fields: skipped_already_completed,
  skipped_active_duplicate, organization_id.
* FileExecutionResult gains 3 optional fields for the API path's legacy
  dict vocabulary: file_name (alias for file), result_data (alias for
  result), skipped (marker like "already_completed").
* Both from_dict updated to populate the new fields.

Producer migrations (workers/file_processing/tasks.py)

* L901 (general path, process_file_batch return):
  BatchExecutionResult(...).to_dict(). Wire dict gains file_results: []
  and errors: [] defaults — strictly additive.
* L1706, L1798, L1823 (API path returns from _process_file_batch_api_core
  helpers): FileExecutionResult(...).to_dict(). L1798 preserves the
  legacy storage_result field via dict-spread merge.

Domain-vocabulary correction on the API path

API-path producers previously returned status="completed" / "failed" —
lowercase strings matching neither ExecutionStatus (workflow-level,
uppercase) nor ApiDeploymentResultStatus (per-file, Success/Failed,
the canonical per-file vocab). Producers now emit "Success" / "Failed"
via FileExecutionResult.

Audit: no Python equality consumer was found reading the lowercase
variants (grep clean). Observability tooling pattern-matching the
old strings would need updating; this is a domain-correctness fix.

Tests

New tests/test_chord_callback_boundary.py — 14 tests, 3 classes:
* Wire-shape characterisation for BatchExecutionResult.
* Wire-shape characterisation for FileExecutionResult with alias
  fields and canonical Success/Failed vocab.
* Consumer tolerance: aggregate_file_batch_results-style .get() reads
  return expected values from the new wire shape.

sdk1's 80 worker_models tests still pass — the dataclass extensions
are strictly additive.

Regression risk: zero on consumer side, zero on backend
(doesn't import these classes; has its own FileExecutionResult in
dto.py — untouched). Status-vocab shift on API path is a deliberate
domain correction.

Test count: workers boundary suite +14 (new); sdk1 dispatcher 80/80.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* UN-3513 [FIX] Address PR review (toolkit + SkipReason enum + producer-binding tests)

A+B from the triage on PR #2020:

* tasks.py:1659 (API-path BATCH return) — migrated to
  BatchExecutionResult.to_dict(). Fixes the half-typed boundary the
  reviewer flagged. file_results, total_files, skipped_already_completed
  and organization_id are now on the wire. Successful/skipped counter
  semantic preserved (separating them is deferred to a follow-up).

* New SkipReason StrEnum (worker_models.py) with ALREADY_COMPLETED +
  ACTIVE_DUPLICATE — mirrors the batch-level skip counters on
  BatchExecutionResult. FileExecutionResult.skipped is now
  SkipReason | None. from_dict coerces. Producer uses the enum;
  the ACTIVE_DUPLICATE value has no current per-file producer but
  is exercised end-to-end via a round-trip test.

* TODO(UN-3516) marker on the three alias fields (file_name,
  result_data, skipped) — sunset ticket filed.

* Tests strengthened:
  - TestProducerBinding drives real _compile_batch_result with a
    minimal SimpleNamespace context, and drives _process_single_file_api
    via mocked api_client for the already-completed branch.
  - TestRealConsumerTolerance imports the real
    aggregate_file_batch_results — producer-consumer contract driven
    end-to-end.
  - test_none_valued_optional_fields_stripped_from_wire documents
    serialize_dataclass_to_dict's None-strip behaviour.
  - test_active_duplicate_skip_reason_round_trips proves the second
    enum value isn't dead.
  - SonarCloud python:S1244 fixed — pytest.approx.
  - skipped_files==0 NIT assertion removed.

Test count: workers boundary suite 14 -> 18; sdk1 worker_models 80/80
still green.

Deferred (separate tickets to follow): __post_init__ silent status
clobber, from_dict status discard, BatchExecutionResult invariant,
storage soft-failure, dead aggregator branch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* UN-3513 [FIX] Address second-pass review (storage_result + lenient skipped + missing producer tests)

Three findings from the second review round on PR #2020:

* HIGH — storage_result silent data loss at batch boundary. The
  per-file dict-spread at tasks.py:1816 preserved storage_result on
  the immediate return, but the value was dropped when wrapped into
  BatchExecutionResult.file_results (from_dict didn't know the key).
  Promoted to a typed FileExecutionResult.storage_result: Any | None
  field; producer now emits via the constructor; from_dict reads it
  back. The round-trip preserves it end-to-end.

* HIGH — strict SkipReason parsing would crash entire batches during
  rolling deploys if a newer producer ever emitted an unknown value.
  Added FileExecutionResult._parse_skipped, which catches ValueError
  + logs a warning + falls back to None. Standard "strict on emit,
  lenient on receive" posture for wire compat.

* MEDIUM — TestProducerBinding only covered 2 of 5 producer branches.
  Added three more tests:
  - _process_single_file_api success branch (asserts storage_result
    survives the typed wire — would catch the dict-spread revert).
  - _process_single_file_api failure branch (asserts canonical
    "Failed" vocab — catches reverts to the legacy lowercase
    "failed").
  - process_file_batch_api batch wrapper via task.apply() with an
    in-memory result_backend (asserts BatchExecutionResult shape +
    skipped_already_completed counter derived from
    SkipReason.ALREADY_COMPLETED.value).
  Strengthened the existing already-completed branch test to assert
  result_data + metadata propagation.

Bug caught by the new batch-wrapper test: process_file_batch_api was
missing execution_time on its BatchExecutionResult(...) call —
BatchExecutionResult.execution_time is a required positional, so the
API-path batch task would have crashed with TypeError on every run.
Introduced batch_start_time = time.time() at task entry and pass
execution_time = time.time() - batch_start_time. The new test would
have caught this immediately at PR time; logging it here as the
exact value of producer-binding coverage.

Test count: 18 -> 21; all green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* UN-3513 [FIX] Symmetric None-stripping for nested file_results + deterministic callback healthcheck picker

Greptile P2 #2 — None-stripping was asymmetric for nested
FileExecutionResult objects. ``serialize_dataclass_to_dict`` only
filters None at the outermost level, so a standalone
``FileExecutionResult.to_dict()`` would omit unset optional fields
while ``batch.to_dict()["file_results"][i]`` would carry explicit
``"file_name": None`` etc. for the same input. A consumer doing
``"x" in result`` membership checks would behave differently
depending on whether it read the standalone wire or the nested-in-
batch wire — a real contract divergence.

Fixed locally on ``BatchExecutionResult.to_dict()`` (not by touching
the shared ``serialize_dataclass_to_dict`` infra): post-process
``wire["file_results"]`` to drop None-valued keys, mirroring the
top-level strip. ``BatchExecutionResult.from_dict`` was already
tolerant via ``.get(...)`` so the round-trip stays clean.

Greptile P2 #1 (``status`` constructor parameter clobbered by
``__post_init__``) is the same pathology I flagged as BLOCKER #1 in
the first review round — deferred to a separate ticket with the
shared-infra dataclass redesign.

Test coverage: extended the existing
``test_none_valued_optional_fields_stripped_from_wire`` to also
assert nested symmetry — same test method, no new method added.
This keeps the pytest collection profile stable (a separate test
method would perturb celery's shared task-registry insertion
order during pytest collection and amplify a pre-existing flake
in ``test_callback_sanity.py``).

Test infra fix (bundled because it would have flaked CI on this
PR's HEAD): ``test_callback_sanity.TestEagerHealthcheckRoundTrip``
selected the healthcheck task via
``endswith(".healthcheck")`` against ``eager_app.tasks``. That
registry is a shared celery global with at least 5 worker modules
registering ``healthcheck`` (callback, executor, file_processing,
log_consumer, scheduler). ``next(...)`` returned whichever was
inserted first, which depends on pytest module-collection order
across the whole suite. The test would assert
``worker_type == "callback"`` and intermittently get ``"executor"``
or ``"file_processing"`` instead — empirically a ~10% flake rate
on this branch's HEAD, climbing to ~90% with any test-collection
perturbation. Replaced with an exact-name lookup
(``name == "callback.worker.healthcheck"``); 30/30 green across
deterministic + randomised probes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* UN-3513 [FIX] Address vishnuszipstack review (7 real fixes + 1 docstring nit)

Seven of Vishnu's PR review findings addressed, all backward-compat with
main-branch consumers. The three [Important] design-redesign findings
(#1 status __post_init__, #2 alias-pair invariant, #3 to_api_dict/to_json
dead code) are deferred to a follow-up shared-infra dataclass ticket
because they would either fire warning noise on existing call sites
(``worker_base.py:211/222``, ``worker_patterns.py:241`` pass wrong-enum
status) or change the wire/cache contract — neither acceptable
mid-flight while keeping zero regression on main.

Changes in this commit are either:
  * Pure additive (test methods, docstrings, observability)
  * Or provably equivalent wire output (the typed-count refactor)

So a rolling deploy where old workers and new workers run concurrently
sees identical wire shapes and identical behaviour for all current
valid data; the only observable differences are log content (better
context on the existing warning) and the presence of a new opt-in
classmethod that nothing currently calls.

* **Vishnu #8 [Suggestion]** — ``SkipReason`` docstring claimed
  "StrEnum semantics" but the class is ``(str, Enum)``, not
  ``enum.StrEnum``. The two differ on ``__str__``. Rewrote the
  docstring to describe the actual behaviour.

* **Vishnu #4a [Important — log context]** — ``_parse_skipped``
  now accepts an optional ``file_execution_id`` kwarg that
  ``from_dict`` threads through. The warning emitted for unknown
  wire values now carries the file identifier, so a real
  rolling-deploy incident is debuggable rather than a context-free
  warning. Optional kwarg with default — any existing caller passing
  one positional arg still works.

* **Vishnu #9 [Suggestion]** — added
  ``BatchExecutionResult.from_file_results(...)`` classmethod that
  derives counters from typed file results. Purely additive: no
  existing caller uses it; the constructor signature is unchanged
  so producers that need their own counter semantics keep working.

* **Vishnu #11 [Suggestion]** — ``process_file_batch_api`` was
  computing ``skipped_already_completed`` by string-matching the
  wire dicts AFTER already calling ``from_dict`` on them. Refactored
  to count from the typed list (single ``from_dict`` pass, enum
  compare). Provably equivalent for all current wire data.

* **Vishnu #4 [Important — test gap]** — added
  ``test_from_dict_unknown_skipped_is_lenient`` covering the one
  documented crash-prevention path. A regression to bare
  ``SkipReason(raw)`` would have re-introduced the rolling-deploy
  crash and kept every other test green.

* **Vishnu #5 [Important — failure-aggregation gap]** — added
  ``test_process_file_batch_api_batch_wrapper_failure_aggregation``
  that drives one success + one failure through the batch wrapper.
  The existing success-only test never exercised
  ``failed_files += 1``.

* **Vishnu #6 [Important — populated round-trip gap]** — added
  ``test_round_trip_with_populated_file_results`` and
  ``test_from_file_results_derives_counters``. The existing
  ``BatchExecutionResult`` round-trip test used
  ``file_results=[]``, so the list-comprehension in ``from_dict``
  that rebuilds nested ``FileExecutionResult`` objects was never
  executed with a populated list.

* **Vishnu #13 [Suggestion]** — replaced hardcoded line reference
  in test docstring with a symbol reference.

Deferred to follow-up shared-infra dataclass-redesign ticket:
  * #1 ``__post_init__`` status clobber — would emit warning noise
    on every existing wrong-enum call site
  * #2 alias-pair invariant — back-fill via __post_init__ would
    change the wire shape (file_name no longer None → no longer
    stripped at the top level)
  * #3 ``to_api_dict``/``to_json`` dead code — looks like a public
    SDK surface; changing the body could surprise external consumers
  * #7 recursive ``None``-strip in ``serialize_value`` — touches
    every dataclass in the codebase
  * #10 ``Any`` typing tightening — low value, mypy tightening could
    trip downstream
  * #12 producer redundant kwargs — depends on #2's reconciliation

Tests: workers chord-callback boundary suite 21 -> 25; full workers
suite 622 -> 627 (no new failures; 6 pre-existing baseline
unchanged). Five deterministic-order runs of the full suite returned
exactly 627 passed / 6 pre-existing failed — zero flakiness from
this change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

)

#2 _sanitize_for_bigquery: the UN-3176 comment said the old magnitude-derived
decimal count over-preserved precision for values BELOW 1, citing 0.0053325 as
a value "passed through unchanged". That is inverted. For magnitude m < 0 the
value carries |m| leading zeros after the point, so `15 - m` decimals preserves
exactly 15 significant figures; the old form was already correct for every
m <= 15. It over-preserves only ABOVE 10^15, where the count floors at 0 and
the full integer part is emitted. A 200k-sample sweep over magnitudes 10^-12 to
10^+20 shows old and new differ only at 10^15 and above -- and old(0.0053325)
== new(0.0053325) exactly. The code change is right; the stated cause was not.

#5 _is_value_error: the docstring claimed it "prefers the structured errors
payload" because "invalidQuery covers the value-level rejections". Neither is
true -- the message text is checked first and returns before the payload is
reached, and `reason` is never inspected at all. Restated to describe what the
function does. The e.errors loop is kept and is provably live: str() of a
google.api_core BadRequest is just "400 <message>" and omits the payload, so a
marker present only there is still matched.

#6 serializers.py: drop a stray blank line added by the diff.

No behaviour change.

#1 (High) ManageDocsModal: the 5s index-status poll could never stop on the
path it was written for. `indexDocs` is emptied only by `deleteIndexDoc`,
called from the websocket handlers -- so when socket messages are dropped it
never empties and the interval ran forever, issuing two requests per tick and
raising two failure toasts per tick. The poll now retires a document itself
when the polled status reports it indexed, which stops the spinner and lets
the effect tear its own interval down; poll-driven calls pass `silent` to
suppress the repeating toast. User-initiated calls are unchanged.

#2 (High) ToolIde: drop the `.then()` that merged `{...details, ...res.data}`
into the store. `details` was a closure snapshot from the render that issued
the PATCH, so a prompt added or deleted while the request was in flight was
silently discarded. The single-pass toggle -- the only caller that changes
single-pass mode, and so the only one UN-2900 needs refreshed prompts for --
already applies the response in its own `.then()`.

#3, #4 (Medium) Correct two comments whose stated premises were false: the
poll does not stop "as soon as indexDocs empties", and the single-pass toggle
does not "only call this function" -- it consumes the response too.

#5 (Low) ProfileInfoBar: chunk_size is nullable; render "-" instead of a
dangling " tokens". 0 still renders "0 tokens".

#6 (Low) AddLlmProfile: calcTokenSize no longer calculates anything -- rename
to toTokenSize and drop the now-redundant `> 0` guard at the call site. Also
clamps negatives, which the unguarded second call site previously displayed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NQhqNqCwFcXZZ7cQU6HxUE

…ranches, stale comments

Review findings on #2284, highest first.

#1 (High) — half the rolling-deploy shim had no gating test that runs in the
default lane. The create_workflow_execution response write had none at all, and
the PgBarrier descriptor was pinned only by a test behind the barrier_db fixture,
which skips wherever Postgres is absent. Descriptor construction moves to
build_callback_descriptor(), pinned by a DB-free suite
(workers/tests/test_legacy_transport_shim.py); the response write is pinned by a
new backend case. All four writes now fail a required build if dropped early.

#2 (High) — _process_file_batch_core's Args entries still described
barrier_context=None as the supported Celery chord path while the body treats it
as a malformed payload. Both entries rewritten; the log line now carries
execution/workflow ids and the batch size.

#3 — process_file_batch_django_compat delegates with two arguments, so after this
PR every call landed in that malformed-payload branch: a false ERROR, and a full
batch run with no claim (no pg_batch_dedup marker, so a redelivery re-runs it —
LLM spend twice and a duplicate destination write with use_file_history=False).
Its only producer was the Celery chord this PR removes, so it now refuses, the
same fail-fast choice step_execution makes. Its MRQ helpers are left in place per
the no-removal rule.

#5 — PG_TRANSPORT_CALLBACK_KWARG became unconditional but its consumers still
branched on it, leaving an else branch that swallows a failed finalization and
strands the execution. The marker is now popped for wire compatibility only;
the duplicate guard and the re-raise are unconditional. is_pg on
_update_execution_status_unified becomes raise_on_failure (default True); the one
caller inside an except block opts out so a raise cannot mask the original error.

#6 — the removal checklist now lists all five artefact groups, and
EXECUTION_EXCLUDED_PARAMS names the key through LEGACY_TRANSPORT_KEY instead of a
hardcoded literal grep would miss.

#7 — CallbackDescriptor.transport is Literal["pg_queue"] and required, not
NotRequired[str]: it is mandatory this release, "celery" must never be written,
and the follow-up removal becomes a type error at the write site.

#8 — the Barrier protocol's "two call sites still program against it" was untrue;
both construct PgBarrier concretely. Corrected, with what actually keeps it.

#10 — executor_dispatch_fakes reimplemented the queue-naming rule and accepted
any enqueue signature. queue_for now delegates to a new production helper
queue_for_executor() (also used by the three dispatch sites), and the fake mirrors
the QueueTransport protocol's keyword-only key set.

#11, #14 — stale "PG only" / "No-op on Celery" comments at the sites this PR made
unconditional, a citation of the deleted Celery max_retries=0 wrapper, and a
cross-reference to a chart file that lives in the cloud repo.

Five tests that asserted the removed Celery branches are restated against the
collapsed behaviour rather than deleted.

Verification: workers 1481 passed / 1 skipped; backend shim, dispatch-orchestrator
and async-wait tests pass; pre-commit clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…, backend and SDK (#2284)

* UN-4078 [MISC] Delete the SDK Celery ExecutionDispatcher and retarget its tests

First slice of the Celery transport removal. `ExecutionDispatcher` published
`execute_extraction` to RabbitMQ and blocked on an `AsyncResult`; nothing has
selected it since UN-4046 made `get_executor_dispatcher()` return the PG
request-reply dispatcher unconditionally. It had zero non-test importers.

Deleted: `unstract/sdk1/.../execution/dispatcher.py` (316 lines), its export
from the package `__init__`, and the `TestExecutionDispatcher` suite.

The 11 workers test files that imported it are retargeted rather than dropped —
they encode live product behaviour (which queue each operation lands on, what
the payload carries), and that behaviour survives on `PgExecutionDispatcher`:

- Queue naming moves to `unstract.workflow_execution.executor_rpc.QUEUE_PREFIX`,
  the surviving definition. The `celery_executor_*` names are NOT dead Celery
  surface — `worker-pg-executor` subscribes to exactly those strings — so the
  assertions pin the literal wire names.
- New `tests/executor_dispatch_fakes.py` holds the shared fake transport, an
  eager variant that runs the task in-process (replacing the `send_task`
  monkey-patching four round-trip tests did), and a correctly-shaped callback
  signature builder.
- The header-forwarding suite is replaced, not ported: PG dispatch takes no
  `headers=` by design, so the new tests assert org routing rides the payload
  AND that passing `headers=` still raises — the exact mistake that broke three
  call sites during UN-4046.
- The raw-`send_task` canary is kept and its message updated; it matters more
  now, since a raw send_task publishes to a broker nothing drains.

Two docstrings that claimed executor calls "go through Celery" were corrected —
they became false with this change, not merely dated.

Verification: workers 1517 passed / 1 skipped. sdk1 unchanged at 20 pre-existing
failures (`test_llm_compat`, `test_litellm_cohere_timeout`), reproduced on a
stashed tree to confirm they predate this work. Pre-commit clean on all 17 files.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* UN-4078 [MISC] Delete the Celery execution transport

Removes the Celery fan-out substrate and the per-execution transport field.
Nothing has selected either since UN-4046 made `select_backend()` return PG
unconditionally, but both were still compiled, imported and shipped — and
importable Celery surface has already cost two incidents (UN-3779's hardcoded
dispatcher, and the `headers=` breakage at three call sites during UN-4046).

Deleted outright:
- `queue_backend/redis_barrier.py` (761 lines) — the DECR-counter substrate
- `CeleryChordBarrier` + `BarrierBackend` + `get_barrier()` + the
  `WORKER_BARRIER_BACKEND` env selector
- `WorkflowTransport`, `DEFAULT_WORKFLOW_TRANSPORT`, `normalize_transport`,
  `is_pg_transport` (88 lines of `unstract/core/data_models.py`)
- `barrier_pg_decr_and_check` — a thin @worker_task wrapper reachable only as a
  Celery `.link`; the decrement runs in-body via `_barrier_pg_decrement`

Collapsed: the 10 `is_pg_transport` branch sites, the `transport` parameter
threaded through the general/api/scheduler orchestrators, the `transport` field
on `WorkflowContextData` and `FileProcessingContext`, the `is_pg` flag on the
file-batch path, and the `transport` key the backend wrote into the dispatch
payload. `_barrier_for_transport` becomes `PgBarrier()`.

Two guards become unconditional rather than PG-gated: the terminal-execution
skip and the duplicate-destination-write check. Both were only ever no-ops on
Celery.

This also starts retiring a documented live hazard. `DEFAULT_WORKFLOW_TRANSPORT` was
`"celery"`, so a payload that lost the field during a rolling deploy selected a
Celery fan-out with no consumers. Worth correcting the in-code note while
deleting it: it claimed `CeleryChordBarrier` was selected and no PG rows were
written, so the reaper could not see the strand. Neither held — every PG worker
sets `WORKER_BARRIER_BACKEND=pg`, so `PgBarrier` was selected, and its
`_reset_barrier` UPSERT is unconditional, so the barrier row was written and the
reaper did see it. Real behaviour was a ~2.5h delayed ERROR.

The reads are removed here, but the writes are NOT: a pre-UN-4078 worker still
defaults an absent field to Celery, so during a rolling deploy it would publish to
a RabbitMQ with no consumers. The next commit keeps writing the field for one
release; the hazard is gone only once that shim is removed.

`EXECUTION_EXCLUDED_PARAMS` deliberately keeps its `"transport"` entry: a
rolling deploy can still have an un-upgraded producer emitting the field, and
without the exclusion it reaches the legacy `execute_workflow` signature and
raises.

Tests: four whole suites deleted with their substrate (redis barrier, barrier
backend selection, barrier differential, workflow-context transport). The rest
are retargeted rather than dropped — the chord canary now asserts *zero* chord
call sites (with a non-vacuity lock, since a zero-expectation assertion is
exactly what a path bug passes silently), and the Celery-gated guard tests are
replaced by the unconditional behaviour they now have.

Verification: workers 1438 passed / 1 skipped / 0 failed. Pre-commit clean.
The backend suite cannot start in this environment (`AppRegistryNotReady`,
reproduced on a stashed tree — pre-existing), so the four backend edits are
CI-verified only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* UN-4078 [MISC] Keep writing the transport field for one release as a rolling-deploy shim

The previous commit removed every read of the `transport` payload field and
every write. Removing the writes in the same release is unsafe: a pre-UN-4078
worker still treats an absent field as "celery" (DEFAULT_WORKFLOW_TRANSPORT)
and, during a Kubernetes RollingUpdate, can receive a payload from an upgraded
producer. It then publishes to RabbitMQ, which has no Celery consumers, so:

- general workflows are marked ERROR by the reaper after ~2.5h, even with every
  file processed;
- API deployments take no orchestration claim, so nothing sweeps them and they
  stay EXECUTING.

The window is real: old file-processing pods keep draining claimed batches for
their full grace period (up to 9120s).

The four writes are restored as `transport: "pg_queue"`, all via one constant,
LEGACY_TRANSPORT_KEY / LEGACY_TRANSPORT_VALUE in unstract.core.data_models, so
the follow-up removal is a single grep:

- backend WorkflowHelper orchestrator dispatch payload
- backend create_workflow_execution response
- workers scheduler async_execute_bin kwargs
- workers PgBarrier callback descriptor (CallbackDescriptor gains
  `transport: NotRequired[str]`)

Nothing on the new side reads the field. Tests pin each write so the shim cannot
be dropped early; the scheduler cases also assert a stale "celery" in the
backend response is never forwarded.

Remove in the release after UN-4078: the writes, the constants, the
`transport` entry in EXECUTION_EXCLUDED_PARAMS, and test_legacy_transport_shim.py.

Verification: workers 1478 passed / 1 skipped / 0 failed; backend dispatch
tests 5 passed (with backend/sample.env loaded); pre-commit clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* UN-4078 [MISC] Address standardized review: shim gating tests, dead branches, stale comments

Review findings on #2284, highest first.

#1 (High) — half the rolling-deploy shim had no gating test that runs in the
default lane. The create_workflow_execution response write had none at all, and
the PgBarrier descriptor was pinned only by a test behind the barrier_db fixture,
which skips wherever Postgres is absent. Descriptor construction moves to
build_callback_descriptor(), pinned by a DB-free suite
(workers/tests/test_legacy_transport_shim.py); the response write is pinned by a
new backend case. All four writes now fail a required build if dropped early.

#2 (High) — _process_file_batch_core's Args entries still described
barrier_context=None as the supported Celery chord path while the body treats it
as a malformed payload. Both entries rewritten; the log line now carries
execution/workflow ids and the batch size.

#3 — process_file_batch_django_compat delegates with two arguments, so after this
PR every call landed in that malformed-payload branch: a false ERROR, and a full
batch run with no claim (no pg_batch_dedup marker, so a redelivery re-runs it —
LLM spend twice and a duplicate destination write with use_file_history=False).
Its only producer was the Celery chord this PR removes, so it now refuses, the
same fail-fast choice step_execution makes. Its MRQ helpers are left in place per
the no-removal rule.

#5 — PG_TRANSPORT_CALLBACK_KWARG became unconditional but its consumers still
branched on it, leaving an else branch that swallows a failed finalization and
strands the execution. The marker is now popped for wire compatibility only;
the duplicate guard and the re-raise are unconditional. is_pg on
_update_execution_status_unified becomes raise_on_failure (default True); the one
caller inside an except block opts out so a raise cannot mask the original error.

#6 — the removal checklist now lists all five artefact groups, and
EXECUTION_EXCLUDED_PARAMS names the key through LEGACY_TRANSPORT_KEY instead of a
hardcoded literal grep would miss.

#7 — CallbackDescriptor.transport is Literal["pg_queue"] and required, not
NotRequired[str]: it is mandatory this release, "celery" must never be written,
and the follow-up removal becomes a type error at the write site.

#8 — the Barrier protocol's "two call sites still program against it" was untrue;
both construct PgBarrier concretely. Corrected, with what actually keeps it.

#10 — executor_dispatch_fakes reimplemented the queue-naming rule and accepted
any enqueue signature. queue_for now delegates to a new production helper
queue_for_executor() (also used by the three dispatch sites), and the fake mirrors
the QueueTransport protocol's keyword-only key set.

#11, #14 — stale "PG only" / "No-op on Celery" comments at the sites this PR made
unconditional, a citation of the deleted Celery max_retries=0 wrapper, and a
cross-reference to a chart file that lives in the cloud repo.

Five tests that asserted the removed Celery branches are restated against the
collapsed behaviour rather than deleted.

Verification: workers 1481 passed / 1 skipped; backend shim, dispatch-orchestrator
and async-wait tests pass; pre-commit clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>