Skip to content

[DOC-FIX] Custom Trace Views (3.16.0) have no documentation page #26335

Description

@rrioh

Warning

Before submitting a PR, please make sure that:

  • A maintainer has triaged this issue and applied the ready label
  • This issue has no assignee
  • No duplicate PR exists

PRs not meeting these requirements may be automatically closed.

URL(s) with the issue

https://mlflow.org/docs/latest/genai/tracing/observe-with-traces/ui/
https://mlflow.org/docs/latest/genai/getting-started/try-assistant/
https://mlflow.org/releases/3.16.0/

Description of proposal (what needs changing)

Custom Trace Views are the headline feature of the 3.16.0 release post, but neither the live docs nor the docs source on master has a page or section about them.
The release post says "no config files, no custom code, just a prompt", and the "View Traces" page (genai/tracing/observe-with-traces/ui) does not mention the feature.
The "MLflow AI Assistant" page (genai/getting-started/try-assistant) describes the assistant chat but not the view builder.

As a user I had to read the source to answer basic questions:

  • What can a view contain? The Assistant does not generate arbitrary UI. It returns an A2UI spec built from a fixed catalog of 14 components (mlflow/server/js/src/shared/web-shared/model-trace-explorer/custom-view/catalog.json: Text, Row, Column, Card, Icon, StatCard, Markdown, KeyValueViewer, AssessmentCard, AssessmentBoard, FeedbackThumbsUpDownButtons, RadioGroup, FeedbackInputText, FeedbackSubmit).
  • Where does "Submit" write to? FeedbackSubmit calls the existing assessment API, so values land on the trace as feedback assessments (the same thing mlflow.log_feedback produces) and show up in the Assessments column of the trace table. The view itself does not store anything.
  • Where are saved views stored, and are there limits? One experiment tag per view (mlflow.customView.view.v1.<id>), at most 50 views per experiment (MAX_CUSTOM_VIEWS_PER_EXPERIMENT in mlflow/utils/validation.py), and a view spec is capped at about 20 KB (CUSTOM_VIEW_TAG_VALUE_SAFE_MAX_BYTES).
  • Which models can build a view? The docs do not say. For OpenAI-compatible and Gateway providers the view is produced by the model calling the render_custom_view tool, so the model has to support tool calling. With llama3.1 8B through Ollama, the model answered in prose instead of calling the tool, and the UI stayed on "Building this view..." with no error.

Proposal: add a "Custom Trace Views" section to the View Traces page (or a new page under observe-with-traces) covering:

  1. What a custom view is: a layout the Assistant builds from a description, saved per experiment (50 per experiment at most)
  2. The component catalog, with one sentence per component
  3. That feedback widgets write trace assessments, and where to see them
  4. Which Assistant providers and models can build a view (tool calling is required for OpenAI-compatible and Gateway providers)

I can open a PR for this.
Note: #26297 adds a TraceImage component to the catalog, so the component list should be written against master once that lands.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    AcknowledgedThis issue has been read and acknowledged by the MLflow admins.area/docsDocumentation issuesbugSomething isn't workingreadyTriaged and ready for implementation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions