Skip to content

Ngfp/sprite lab package - #75686

Draft
wilkie wants to merge 39 commits into
ngfp/music-lab-packagefrom
ngfp/sprite-lab-package
Draft

wilkie wants to merge 39 commits into
ngfp/music-lab-packagefrom
ngfp/sprite-lab-package

Conversation

@wilkie

@wilkie wilkie commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

Links

  • Jira:

Testing story

Deployment notes

Privacy and security

@wilkie
wilkie changed the base branch from staging to ngfp/music-lab-package October 1, 2026 18:33
@wilkie
wilkie force-pushed the ngfp/music-lab-package branch from a90f96b to 495c7e5 Compare October 1, 2026 20:27
@wilkie
wilkie force-pushed the ngfp/sprite-lab-package branch 2 times, most recently from 4654ab5 to fbaa697 Compare October 2, 2026 03:08
@wilkie
wilkie force-pushed the ngfp/music-lab-package branch from 495c7e5 to 86e0706 Compare October 2, 2026 03:08
wilkie and others added 25 commits October 1, 2026 20:10
Scaffolds labs/sprite with the lab generator and ports the parts of
apps' Lab2 Sprite Lab that stand alone: platform physics, character
animation, the camera, audio feedback and the image pixel helpers,
with their apps tests. docs/port-plan.md lays out the remaining
layers and the decisions they wait on.

Exports lab-base's local-storage helpers, which the audio settings
use.
Brings over the lab's types, world grid, image references, trimming,
level modes, guide steps, toolbox and added-block handling, scene
sharing and thumbnails, and UI state, with their apps tests. Apps'
Blockly and Lab2 types map to their frontend equivalents.

Adds a spriteLab module to core's API client for the two cross-project
scene endpoints, replacing apps' HttpClient. Scene fingerprints stay
MD5 so thumbnails saved by apps still match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Brings over the lab-owned block definitions, their fields, toolbox
helpers and placeholder rendering, and apps' image-dropdown and bitmap
fields they build on. Fields travel as plugins on the definitions, and
SPRITELAB2_BLOCKS collects them for a workspace. A new test registers
and builds every block headlessly.

The image pickers read images through a seam the engine will supply;
the legacy block pool waits on the engine decision.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
The views are scene picking and editing, the tab shell, the play bar and
the image-pane pieces that need neither the engine nor image generation.

The sprite library (src/sprites, exported as
@code-dot-org/sprite-lab/sprites) reproduces the part of p5.play that
Sprite Lab uses, over upstream p5 2.x, without Code.org's LGPL fork. It
was written clean-room: behavioral specifications (docs/sprites) were
written from the fork, and the library was implemented from them alone.
Every requirement and test vector is a test. Fork bugs are kept on
purpose and listed in docs/sprites/compatibility.md, each pinned by a
test that names it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
The library no longer redefines p5's mouse fields or wraps its pointer,
key and button handlers. Its own window listeners keep the pointer state,
which World.mouseX, the edge states, the hit tests and the camera read;
p5's fields are left to p5. Hosts press keys through pressKey and
releaseKey.

Registers the departures this makes from the fork: unrounded positions,
a window blur releasing the button, the public key calls, and the
library-owned pointer state (CMP-43..CMP-46).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
CoreLibrary and the Sprite Lab commands, typed, taking the sprite
library's p5 instance where apps took P5Wrapper's. What apps reached
through Redux, audioApi and the Blockly globals goes through a
RuntimeHost the engine supplies. drawBackground uses p5 2.x's image(),
getP5Color uses setAlpha, and only an image can be a background.

Apps' CoreLibraryTest and command tests are ported and pass. CMP-9 is
now pinned, and CMP-47, the first spiral sprite never moving, is new
to the register.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
customMarshaler and asyncFunctionList were class fields, which the
compiled class resets to undefined after the base constructor set
them, so no custom-marshalled object ever reached a program.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Project images are normalized, fetched and built into the predefined
animation table, clean-room from ANI-27 and ANI-35..ANI-38.
SpriteLabEngine runs programs over one p5 instance on a port of
JSInterpreter's non-debugger core, with Sprite Lab 2's scene,
platform and camera commands. engine-demo.html runs it in a browser,
and e2e/engine.spec.ts drives it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
ProjectSourcesSchema stripped every key but source and labConfig, so
Sprite Lab's scenes and animations were lost when a project loaded, and
took source only as a string or a multi-file source, so saving a Blockly
workspace kept as an object threw before the request was sent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
App is BlocklyLab around SpriteLabView: the Code tab's workspace on
@code-dot-org/blockly, the World tab, the playspace the engine draws
in, scenes and their pictures, Play and the live preview. The
project's images are a Redux slice. The build now ships its CSS. The
block pool, external scenes and the Images tab are still to come; see
docs/port-plan.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Each sharedBlocks entry becomes an @code-dot-org/blockly definition
that builds itself in init as block_utils' blocks did, with its
generator, so pool blocks look and compile as in apps. Custom-typed
sockets meet only their own type, as apps' connection checker had
them. The location field picks a point on the playspace, with apps'
costume ghost. The simple fixture carries the GamelabJr pool.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
The go-to-external-scene block lists the scenes of section-mates'
projects on the level, and a jump in Play fetches the project and runs
the scene on its owner's images. A project whose saved blocks name an
external scene loads its workspace once the list is in, as in apps;
the list follows the level instead of loading once behind a ref, which
strict mode's second mount left waiting forever.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
…ent.

libraries.getSource reads a helper library's source; animations.remove
deletes a project's animation; media.getImage reads an image by URL,
through the media proxy for a site that does not allow cross-origin
reads, as apps' assetPrefix.fixPath arranged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Helper libraries, project images and image deletes called fetch
directly; they now use core's libraries, media and animations modules.
A data URL source is taken as the image, with no request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Only Sprite Lab 2 uses it. The image being edited is read through the
media API, or from memory when it is a data or object URL. The
tooltip's data-theme slot prop needs the design system's MUI
augmentation, now mirrored in the package.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
The ky transport sent X-CSRF-Token with every write, including an
absolute URL on another host, such as the AI gateway. It now goes only
to dashboard's own origin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Ported from apps/src/aiGateway: a fresh dashboard token per call, then
generateText against the gateway, validated with the gateway contract.
The contract is copied from apps, which stays its source of truth; a
test compares the copy with apps' snapshots. Turnstile is not ported, so
a call the gateway would demand it for fails before it is sent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Text calls get a canned sentence, structured calls the first value
their schema admits, and image calls a prepared pixel image on magenta
at the model's output size. A scenario can register its own answer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
A Vite plugin at /__ai-gateway holds GOOGLE_GENERATIVE_AI_API_KEY on
the server and answers gateway calls from the Gemini API, running none
of the gateway's checks. It mounts only under vite serve, with a key,
and outside production mode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
The blank paint canvas, background removal, the image model's size
constants without the AI SDK provider, the generated-image types and
the image session hook.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
images.moderate posts an image's bytes to /v3/images/moderate and
imageModerationVerdict judges Azure's severities, as apps'
moderateImage did. The transport now sends a Blob body as its bytes.
The mock answers every image as safe, and the gateway client takes
the llm_safety_judge phase.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
@code-dot-org/lab/ai carries aichat's text and image judges and the
moderation-plus-judge check for generated images, over core's gateway
and moderation APIs. Apps put the image judges behind DCDO switches
that default on; with no DCDO here they are on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Single images and character sets generate through core's gateway
client and are checked by lab-base's safety module, with apps' tests.
Saving to a level's starter assets waits on a core API.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
The gallery, the new-image form and the details dialog, with
generation, painting, rename and delete; a rename or delete reaches
every scene, World grid and the open workspace. A freeplay fixture
offers the paint tools. The image-prompt event is a metric without
the student's prompt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Guide takes collapsed, a panelRef and a width in px, and exports the
offsets siblings lay out around, as apps' Guide gained after the port.
NavigationArea is exported from ./instructions for a lab's own guide.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
wilkie and others added 14 commits October 1, 2026 20:10
The guide floats over a level with a mode, stepping through its
guideSteps, narrowing on Play and collapsing on freeplay. An image
level is the image pane alone, with the guide over it and no engine.
An image fixture shows the guide moving on once a character is made.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
showStartOverDialog was declared as (type, message) but implemented as
(message), and StartOverDialog ignored the message it was passed, so a
lab's own warning, such as Sprite Lab's that images go too, never
appeared. A custom request now shows its message, and a text or blocks
request the standard one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
A level toolbox listing a blocksCommon block now keeps it, where the
registered-block filter dropped it. The lab's image dropdowns take the
config helpers from @code-dot-org/blockly. The behavior picker's empty
list matches apps' Lab2, which never mounts the behavior editor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
projects.getPersonal reads GET /api/v1/projects/personal, the signed-in
user's own projects; the mock lists none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
registerLabFixtures cleared a lab's routes before registering its
fixtures, so a route a fixture module registered itself was gone by the
first request. A fixture's routes field is registered under its own
scenario instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
The play-music block offers the user's Music Lab projects on the default
pack, and in Play a scene loops its song through Music's ProjectPlayer,
loaded on the first song. A project whose saved blocks name a song loads
its workspace once the list is in. A music fixture plays an electro loop
from the real library.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Legacy (non-Lab2) GamelabJr levels, 3,682 of them in 142 units, run on
the package's own interface; everything but the Piskel editor is in
scope. The plan records the scale, what carries over, the decisions
made, two fixes that also affect Lab2 data, and the work in twelve
layers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
`yarn dev` in the package now hosts the lab in LabHost under the mock
service worker and the CodeAI theme, as Studio does, so the fixtures load
at /app/projects/sprite/<fixture>/edit without Studio or dashboard. The
dev server also takes a developer's own AI key through the gateway proxy.

The README describes the lab as it stands and how to run it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
A LabFixture's aiImages rules, tested in order against the request's
text, pick the image the mock AI gateway returns. An edit request no rule
matches gets back the image it sent, so an animation's frames stay one
character; anything else still gets the canned character.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
The fixtures' aiImages rules tell a background, a ground block and a
smooth sprite apart by the wording the lab adds to each prompt, and
answer with a prepared image: a meadow, or a night sky when the prompt
says night, and a grass tile, each in pixel art when that style is
chosen. The wording moves into promptClauses.ts, which both the prompts
and the rules use, so a rewording cannot strand the rules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
The fixture reads the level's .level file for its toolbox, block pools,
helper libraries, image roles and guide, so it follows the course. Its
project has a story and a platform scene, which is what splits the
authored toolbox, and a world with a floor to stand on. The pool loader,
renamed blockPools.ts, now loads GameDev beside GamelabJr.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Served from an unknown host, core falls back to the development API
origin, http://localhost-studio.code.org:3000. On an HTTPS host such as
GitHub Pages the browser blocks that as mixed content before the mock
worker can answer. The pages build now carries core's cdo-api-url meta
tag set to same-origin; the dev server is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
The page object waited for the placeholder "Sprite Lab" heading the
package started with; the harness renders the lab itself, so it now
waits for the Code tab and checks that the scene's workspace shows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
@wilkie
wilkie force-pushed the ngfp/sprite-lab-package branch from fbaa697 to c1208cb Compare October 2, 2026 03:21
@wilkie
wilkie force-pushed the ngfp/music-lab-package branch from 86e0706 to 0c6fe6f Compare October 2, 2026 03:21

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant