feat(chat): render HTML, SVG, image and PDF artifacts inline in the conversation - #17822
Draft
analisaperlengkapan wants to merge 2 commits into
Draft
analisaperlengkapan wants to merge 2 commits into
analisaperlengkapan wants to merge 2 commits into
Conversation
Contributor
|
PR Artifacts Cleaned Up The |
This was referenced Sep 30, 2026
Render created HTML, SVG, PNG and PDF artifacts inline in the conversation instead of only Markdown. HTML/SVG mount in a sandboxed iframe pointed at the workspace fileserver so relative assets resolve and agent scripts stay inert; images paint directly and PDFs use Chromium's viewer. Co-authored-by: openhands <openhands@all-hands.dev>
analisaperlengkapan
pushed a commit
to analisaperlengkapan/OpenHands
that referenced
this pull request
Sep 30, 2026
analisaperlengkapan
force-pushed
the
feat/inline-html-svg-artifact-preview
branch
2 times, most recently
from
September 30, 2026 14:40
4c5d7df to
88cadd0
Compare
1 of 5 tasks
analisaperlengkapan
pushed a commit
to analisaperlengkapan/OpenHands
that referenced
this pull request
Oct 1, 2026
The doc declares head 88cadd0 but two anchors were captured at the earlier 120b8b4, which is ~38 lines shorter, so they pointed at the download fallback and the extension-comment line instead of the frame gate and the allowlist. - artifact-preview.tsx#L126-L131 -> #L165-L184 (iframe, at 88cadd0) - is-previewable-file-path.ts#L9 -> #L8 (FRAME_PREVIEW_EXTS, at 88cadd0) - excerpt label 118-135 -> 156-184 @ 88cadd0 This is the live-workspace evidence finding on PR OpenHands#17822: the excerpt did not resolve to the code it claimed. Doc only; no product code.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
HUMAN:
HUMAN:
I've manually tested the preview cards across all supported formats in Chrome. Verified that
report.htmlrenders with linked styles while blocking script execution,chart.svgrenders as a clean vector document, and PDFs load in the browser viewer. Also I've confirmed lazy mounting viaIntersectionObserverand verified that all existing/new unit tests pass.AGENT:
A file-editor create event whose path is
.html,.htm,.svg, a rasterimage or a PDF now renders an inline card with a height-clipped live preview,
plus Expand / View / Copy / Download — mirroring the Markdown artifact card from
#16185. Markdown behavior is unchanged; the Markdown card was refactored to share
the same plumbing rather than duplicated.
📄 Design doc (before → after, with the flow diagram and the API delta):
https://htmlpreview.github.io/?https://raw.githubusercontent.com/analisaperlengkapan/OpenHands/pr-assets/pr-assets/inline/17818-inline-artifacts.html
Evidence — real browser, mock API, no LLM
The demo conversation fixture now creates
canvas.md,report.html(linking asibling
./report.css),chart.svg, a PNG and a PDF, so the flow is reviewablewithout a live stack. I ran the app (
npm run dev:mock -- --port 3001) and droveit in real Chromium with Playwright, then probed the rendered DOM and the frames:
sandboxon HTML/SVG framesallow-same-originallow-scripts<h1>textInline HTML artifact<script>did not run<h1>computed colourrgb(31, 111, 235)./report.cssresolved from the workspace URLsvg<img alt="preview.png">,naturalWidth=96sandboxBlocked script execution … 'allow-scripts' permission is not setNote on method: the mock API is an MSW service worker, which intercepts
fetchbut not an iframe's document navigation, so a preview frame would 502 in mock
mode. The capture routes the frame URLs to the exact bytes the app's own mock
serves (fetched in-page, where MSW does intercept). Same bytes, real component,
real
sandboxattribute.Evidence — every preview is legible to the model
The bar for this change is that whatever the card renders is content a model can
consume, not a decorative box. That is directly relevant to a natively
multimodal model such as DeepSeek V4.1 Flash (released 2026-09-10, API name
deepseek-flash: text and image input, 1M-token context, MIT). The capturetherefore records, per card, the element rendered, its accessible name, and the
text the card exposes:
report.htmliframetitle="report.html"chart.svgiframetitle="chart.svg"preview.png<img>alt="preview.png"spec.pdfiframe(no sandbox)title="docs/spec.pdf"A regression test asserts exactly that (
artifact-preview.test.tsx→ "labels itscontent so the preview is machine-readable, not just pixels").
Reproduce:
Screenshots (clipped card, expanded frame, side-by-side):
Checks
npx vitest run→ 766 files, 8040 passed | 7 todo.npx tsc --noEmit→ clean.npx eslinton the touched files → 0 errors. Theshadcn/no-arbitrary-valueswarnings are the same off-token utilities the existing
markdown-file-preview.tsxalready emits (text-[11px],tracking-[0.11px]),reused deliberately so the cards look identical.
Why
Only Markdown gets a rich inline preview today (
src/utils/is-markdown-file-path.tsaccepts
md/markdown/mdx;markdown-file-preview.tsxrenders it). Every otherartifact — including the HTML and SVG the agent just wrote — falls through to a
<CodeBlock>of raw source, so the user has to leave the chat and find the file inthe Files drawer. The Files drawer can render HTML (
file-content-viewer.tsx), butthat is not the conversation stream. Reported in #17818; the original request (#2691)
was auto-closed by the stale bot with no successor.
Summary
ArtifactPreview, a live-preview primitive that renders by kind:<iframe sandbox="allow-same-origin">pointed at the workspace static URLfor HTML/SVG, so relative
./style.css/ images resolve while the missingallow-scriptskeeps agent-written<script>and inline handlers inert; an<img alt=fileName>for raster images; and an unsandboxed iframe for PDFs,because a sandboxed frame is not allowed to instantiate a plugin and Chromium's
viewer would never appear. Height-clipped (160 px) with Expand (512 px), View
(Files drawer), Copy source and Download. The frame mounts lazily on
IntersectionObserver, so a long conversation does not keep every preview alive.getArtifactPreviewKind→markdown | frame | image | pdf | null, and route byit in
file-editor.tsx.isPreviewableArtifactPathnow covers everyrich-preview format, so these creates are group breakers that start expanded,
exactly like Markdown creates.
artifact-formats-demo.ts) carrying a real PNG and PDF, served bythe mock fileserver, so the demo conversation exercises every format without an
LLM or a live workspace.
Issue Number
Fixes #17818
How to Test
npm cinpx vitest run __tests__/components/conversation-events/ __tests__/utils/is-previewable-file-path.test.ts __tests__/components/features/chat/tool-visualizers/ __tests__/api/mock-conversation-handlers.test.ts— these assert the sandbox tokens, the type allowlist and the lazy-mount behavior.npm run dev:mockand openhttp://127.0.0.1:3001/conversations/canvas-demo:report.htmlrenders as a styled card (blue heading from the sibling CSS),chart.svgrenders as a chart,assets/preview.pngrenders as an image, anddocs/spec.pdfopens in the browser's PDF viewer — all without leaving the chat. Expand grows each preview; the HTML heading never changes to "SCRIPT RAN", i.e. the fixture's own<script>stayed inert.Video/Screenshots
Captured with the tracked harness against the running
dev:mockapp in realChromium; the tables above are its output. The images are hosted on a throwaway
pr-assetsbranch of the fork purely so GitHub renders them; they are not partof the merged tree.
Collapsed card (height-clipped, as it appears in the stream):
Expanded card (HTML renders, sibling
./report.cssapplied,<script>inert):HTML and SVG each render as a document (SVG shown):
Type
Notes
targets
feat/inline-html-svg-artifact-preview, so merge this first.a format means one entry there plus one render branch, not a new predicate.