Skip to content

dashboard.md

Latest commit

 

History

History
59 lines (45 loc) · 2.72 KB

dashboard.md

File metadata and controls

59 lines (45 loc) · 2.72 KB
Raw

Evaluation studio

A static dashboard with no framework, build step, fonts, analytics, or CDN dependencies. Start it with make dashboard and visit http://localhost:8080. Python 3 is the only runtime requirement. Opening the HTML as a file cannot load the JSON feed.

Explore a report

  • Use the sidebar to move between the overview, performance, checkpoint table, and generations.
  • Toggle metric chips to compare curves. At least one metric stays selected.
  • Hover a chart to inspect values; the full table provides the same data for keyboard and touch users.
  • Search checkpoints by name and select a table heading to sort. Missing values sort last.
  • Import a report JSON from llmft eval directly in the browser (up to 10 MB). Imported reports stay in memory and are cleared on reload. They are never uploaded.
  • Export all checkpoints as CSV, independent of the table filter.
  • Switch between dark and light themes; the browser remembers the choice.

The bundled report is explicitly marked Demo report. It contains reference values from the existing seed script, not results generated by opening the page. Generate your own report with llmft eval --config configs/eval.yaml, or import one. To restore the bundled demo after a local evaluation, run:

python -m scripts.seed_dashboard

Visual design and accessibility

Translucent panels, soft ambient colour, subtle hover motion, and emerald accents frame the report. Backdrop blur is decorative: the layout and data remain readable without it. prefers-reduced-motion disables animation and smooth scrolling.

Controls have visible focus rings, metric toggles expose their pressed state, table headers expose sort direction, and search/import messages use live status regions. The checkpoint table scrolls independently on narrow screens. Lines have stable metric colours, direct end labels where space permits, and a text legend.

Data interpretation

Quality metrics and loss use separate charts. Length ratio is hidden by default because it is not a quality score and is not bounded by 1. Enabling it rescales the metric chart. Bar charts start at zero. The primary metric is selected by the Python report builder; the dashboard uses its best-checkpoint summary.

Generation comparison selects the base and best checkpoint, matching by prompt. When no shared prompt is available, it displays one sample rather than presenting unrelated prompts as a before/after comparison.

Verification

npm ci
npx playwright install chromium
npm test

Playwright checks desktop and mobile rendering, filtering, sorting, themes, import/export, unsafe text, network failure, missing losses, and a single-checkpoint report. JavaScript requires a modern browser (Chrome, Edge, Firefox, or Safari).