fix(huggingface_hub): tolerate non-mapping responses - #842

ELI5

The huggingface_hub integration writes a trace span after every model call. Six code paths that read the model's response assume it came back as a dictionary, but the generative-media methods (text_to_image, text_to_video, text_to_speech, and friends) return raw bytes. The first wrapper added for any of them would turn a successful call into an AttributeError raised from inside its own tracing code — the customer's request succeeds, then the receipt for it crashes.

This PR replaces x is None with isinstance(x, Mapping) at those six sites. Net production change is +1 line. Nothing changes for the four methods currently instrumented; their responses are always dict-like.

The one decision worth review: Mapping, not dict — see Why Mapping.

What Changed

Six response-shaping sites in py/src/braintrust/integrations/huggingface_hub/tracing.py now guard with isinstance(x, Mapping):

A non-mapping response now yields empty metrics/output/metadata rather than raising.

The four currently-wrapped methods — chat_completion, text_generation, feature_extraction, sentence_similarity — are unaffected.

This is prerequisite work for #487 (text_to_image, image_to_text, text_to_speech). Those three return image/audio bytes, and without this guard the first wrapper for any of them raises from its own tracing code. Happy to follow up with the wrappers themselves if useful.

Why

InferenceClient exposes 28 unwrapped methods — of 32 public methods on the client, the integration instruments 4. Twelve of the unwrapped set are media or audio (text_to_video, text_to_image, image_to_video, automatic_speech_recognition, text_to_speech, audio_to_audio, and others), and none of them return dicts.

(Counted against huggingface_hub==1.32.0, pinned in [tool.braintrust.matrix.huggingface-hub].latest.)

So the hazard is not hypothetical and not scoped to one method — it sits on the shared path every future wrapper will walk. Fixing it once here is cheaper than rediscovering it per-wrapper.

Why Mapping and not dict

huggingface_hub's BaseInferenceType subclasses dict today, but its own docstring describes that base as kept "for backward compatibility" with a plan to remove it. A dict guard would then fail silently: no exception, just missing token counts. It also rejects mapping types that are not dict subclasses.

The sibling integrations already take the permissive route: cohere reads through _get_field, mistral normalizes via _normalized_mistral_dict. Neither does a concrete-type check, so Mapping matches local convention rather than departing from it.

The alternative considered was hasattr(result, "get") — looser still, but it admits objects whose get is not mapping semantics. Mapping keeps the actual contract.

Linked Issue

Refs #487 — prerequisite hardening, not reachable in production today.

Visual Proof

N/A — response-shaping helpers inside a tracing library; no user-facing surface.

Testing

• nox -s "test_huggingface_hub(latest)" → 39 passed. (0.32.0) → 37 passed, 2 skipped (pre-existing numpy pin at the floor).
• nox -s pylint (whole SDK) and nox -s test_types (pyright + mypy + 36 type tests) → clean.
• pre-commit run on both changed files → format, ruff, codespell, EOF, whitespace pass.
• Python 3.10.21 / 3.11.15 / 3.12.13 / 3.13.15 / 3.14.6 → identical results, no version-specific regressions.
• Every test mutation-verified — each confirmed to fail with the fix reverted: unguard the sibling shapers → 5 failures; Mapping→dict → the UserDict case fails; revert line 726 only → the full-path test fails.
• Baseline on each interpreter: 16 failed / 4 passed before, 16 failed / 23 passed after. The 16 are pre-existing and environmental — huggingface_hub resolves provider↔model over the network before VCR engages (ValueError: Model ... not supported by provider cerebras, HTTP 401).
• Hosted CI across the 3.10–3.14 matrix — not yet run; fork PRs await workflow approval.

Review

Two things worth a second opinion:

1. Mapping vs duck-typed get — the tradeoff above.
2. Scope. This deliberately does not decide what a bytes span output should look like. Existing precedent is images logging b64_json_present rather than the payload (tracing.py:1791), and embeddings reducing an array to {embedding_length} (tracing.py:304), so length plus media type is the obvious shape — but that belongs to whoever writes the wrapper.

Process note: an earlier draft of this fix touched only one of four call sites, which relocated the crash rather than removing it. Review caught it; the full-path tests and the mutation runs above exist because of that.

Agent skill upstream boundary

• Not applicable — no agent skill, harness, or orchestration surface is touched. Confined to response shaping in one integration.

Checklist

• Explained before/after, mechanism and alternatives.
• N/A for visual proof, with reason.
• Cross-platform impact considered — verified 3.10–3.14; no runtime behavior change for currently-wrapped methods.
• Focused production change: six guards, +1 net production line, no shared-code changes.
• Final self-review and hosted CI — hosted CI pending.