You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Custom Trace Views are the headline feature of the 3.16.0 release post, but neither the live docs nor the docs source on master has a page or section about them.
The release post says "no config files, no custom code, just a prompt", and the "View Traces" page (genai/tracing/observe-with-traces/ui) does not mention the feature.
The "MLflow AI Assistant" page (genai/getting-started/try-assistant) describes the assistant chat but not the view builder.
As a user I had to read the source to answer basic questions:
What can a view contain? The Assistant does not generate arbitrary UI. It returns an A2UI spec built from a fixed catalog of 14 components (mlflow/server/js/src/shared/web-shared/model-trace-explorer/custom-view/catalog.json: Text, Row, Column, Card, Icon, StatCard, Markdown, KeyValueViewer, AssessmentCard, AssessmentBoard, FeedbackThumbsUpDownButtons, RadioGroup, FeedbackInputText, FeedbackSubmit).
Where does "Submit" write to? FeedbackSubmit calls the existing assessment API, so values land on the trace as feedback assessments (the same thing mlflow.log_feedback produces) and show up in the Assessments column of the trace table. The view itself does not store anything.
Where are saved views stored, and are there limits? One experiment tag per view (mlflow.customView.view.v1.<id>), at most 50 views per experiment (MAX_CUSTOM_VIEWS_PER_EXPERIMENT in mlflow/utils/validation.py), and a view spec is capped at about 20 KB (CUSTOM_VIEW_TAG_VALUE_SAFE_MAX_BYTES).
Which models can build a view? The docs do not say. For OpenAI-compatible and Gateway providers the view is produced by the model calling the render_custom_view tool, so the model has to support tool calling. With llama3.1 8B through Ollama, the model answered in prose instead of calling the tool, and the UI stayed on "Building this view..." with no error.
Proposal: add a "Custom Trace Views" section to the View Traces page (or a new page under observe-with-traces) covering:
What a custom view is: a layout the Assistant builds from a description, saved per experiment (50 per experiment at most)
The component catalog, with one sentence per component
That feedback widgets write trace assessments, and where to see them
Which Assistant providers and models can build a view (tool calling is required for OpenAI-compatible and Gateway providers)
I can open a PR for this.
Note: #26297 adds a TraceImage component to the catalog, so the component list should be written against master once that lands.
Warning
Before submitting a PR, please make sure that:
readylabelPRs not meeting these requirements may be automatically closed.
URL(s) with the issue
https://mlflow.org/docs/latest/genai/tracing/observe-with-traces/ui/
https://mlflow.org/docs/latest/genai/getting-started/try-assistant/
https://mlflow.org/releases/3.16.0/
Description of proposal (what needs changing)
Custom Trace Views are the headline feature of the 3.16.0 release post, but neither the live docs nor the docs source on master has a page or section about them.
The release post says "no config files, no custom code, just a prompt", and the "View Traces" page (genai/tracing/observe-with-traces/ui) does not mention the feature.
The "MLflow AI Assistant" page (genai/getting-started/try-assistant) describes the assistant chat but not the view builder.
As a user I had to read the source to answer basic questions:
mlflow.customView.view.v1.<id>), at most 50 views per experiment (MAX_CUSTOM_VIEWS_PER_EXPERIMENT in mlflow/utils/validation.py), and a view spec is capped at about 20 KB (CUSTOM_VIEW_TAG_VALUE_SAFE_MAX_BYTES).Proposal: add a "Custom Trace Views" section to the View Traces page (or a new page under observe-with-traces) covering:
I can open a PR for this.
Note: #26297 adds a TraceImage component to the catalog, so the component list should be written against master once that lands.