Ngfp/sprite lab package - #75686
Draft
wilkie wants to merge 39 commits into
Draft
Ngfp/sprite lab package#75686wilkie wants to merge 39 commits into
wilkie wants to merge 39 commits into
Conversation
wilkie
force-pushed
the
ngfp/music-lab-package
branch
from
October 1, 2026 20:27
a90f96b to
495c7e5
Compare
wilkie
force-pushed
the
ngfp/sprite-lab-package
branch
2 times, most recently
from
October 2, 2026 03:08
4654ab5 to
fbaa697
Compare
wilkie
force-pushed
the
ngfp/music-lab-package
branch
from
October 2, 2026 03:08
495c7e5 to
86e0706
Compare
Scaffolds labs/sprite with the lab generator and ports the parts of apps' Lab2 Sprite Lab that stand alone: platform physics, character animation, the camera, audio feedback and the image pixel helpers, with their apps tests. docs/port-plan.md lays out the remaining layers and the decisions they wait on. Exports lab-base's local-storage helpers, which the audio settings use.
Brings over the lab's types, world grid, image references, trimming, level modes, guide steps, toolbox and added-block handling, scene sharing and thumbnails, and UI state, with their apps tests. Apps' Blockly and Lab2 types map to their frontend equivalents. Adds a spriteLab module to core's API client for the two cross-project scene endpoints, replacing apps' HttpClient. Scene fingerprints stay MD5 so thumbnails saved by apps still match. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Brings over the lab-owned block definitions, their fields, toolbox helpers and placeholder rendering, and apps' image-dropdown and bitmap fields they build on. Fields travel as plugins on the definitions, and SPRITELAB2_BLOCKS collects them for a workspace. A new test registers and builds every block headlessly. The image pickers read images through a seam the engine will supply; the legacy block pool waits on the engine decision. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
The views are scene picking and editing, the tab shell, the play bar and the image-pane pieces that need neither the engine nor image generation. The sprite library (src/sprites, exported as @code-dot-org/sprite-lab/sprites) reproduces the part of p5.play that Sprite Lab uses, over upstream p5 2.x, without Code.org's LGPL fork. It was written clean-room: behavioral specifications (docs/sprites) were written from the fork, and the library was implemented from them alone. Every requirement and test vector is a test. Fork bugs are kept on purpose and listed in docs/sprites/compatibility.md, each pinned by a test that names it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
The library no longer redefines p5's mouse fields or wraps its pointer, key and button handlers. Its own window listeners keep the pointer state, which World.mouseX, the edge states, the hit tests and the camera read; p5's fields are left to p5. Hosts press keys through pressKey and releaseKey. Registers the departures this makes from the fork: unrounded positions, a window blur releasing the button, the public key calls, and the library-owned pointer state (CMP-43..CMP-46). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
CoreLibrary and the Sprite Lab commands, typed, taking the sprite library's p5 instance where apps took P5Wrapper's. What apps reached through Redux, audioApi and the Blockly globals goes through a RuntimeHost the engine supplies. drawBackground uses p5 2.x's image(), getP5Color uses setAlpha, and only an image can be a background. Apps' CoreLibraryTest and command tests are ported and pass. CMP-9 is now pinned, and CMP-47, the first spiral sprite never moving, is new to the register. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
customMarshaler and asyncFunctionList were class fields, which the compiled class resets to undefined after the base constructor set them, so no custom-marshalled object ever reached a program. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Project images are normalized, fetched and built into the predefined animation table, clean-room from ANI-27 and ANI-35..ANI-38. SpriteLabEngine runs programs over one p5 instance on a port of JSInterpreter's non-debugger core, with Sprite Lab 2's scene, platform and camera commands. engine-demo.html runs it in a browser, and e2e/engine.spec.ts drives it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
ProjectSourcesSchema stripped every key but source and labConfig, so Sprite Lab's scenes and animations were lost when a project loaded, and took source only as a string or a multi-file source, so saving a Blockly workspace kept as an object threw before the request was sent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
App is BlocklyLab around SpriteLabView: the Code tab's workspace on @code-dot-org/blockly, the World tab, the playspace the engine draws in, scenes and their pictures, Play and the live preview. The project's images are a Redux slice. The build now ships its CSS. The block pool, external scenes and the Images tab are still to come; see docs/port-plan.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Each sharedBlocks entry becomes an @code-dot-org/blockly definition that builds itself in init as block_utils' blocks did, with its generator, so pool blocks look and compile as in apps. Custom-typed sockets meet only their own type, as apps' connection checker had them. The location field picks a point on the playspace, with apps' costume ghost. The simple fixture carries the GamelabJr pool. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
The go-to-external-scene block lists the scenes of section-mates' projects on the level, and a jump in Play fetches the project and runs the scene on its owner's images. A project whose saved blocks name an external scene loads its workspace once the list is in, as in apps; the list follows the level instead of loading once behind a ref, which strict mode's second mount left waiting forever. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
…ent. libraries.getSource reads a helper library's source; animations.remove deletes a project's animation; media.getImage reads an image by URL, through the media proxy for a site that does not allow cross-origin reads, as apps' assetPrefix.fixPath arranged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Helper libraries, project images and image deletes called fetch directly; they now use core's libraries, media and animations modules. A data URL source is taken as the image, with no request. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Only Sprite Lab 2 uses it. The image being edited is read through the media API, or from memory when it is a data or object URL. The tooltip's data-theme slot prop needs the design system's MUI augmentation, now mirrored in the package. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
The ky transport sent X-CSRF-Token with every write, including an absolute URL on another host, such as the AI gateway. It now goes only to dashboard's own origin. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Ported from apps/src/aiGateway: a fresh dashboard token per call, then generateText against the gateway, validated with the gateway contract. The contract is copied from apps, which stays its source of truth; a test compares the copy with apps' snapshots. Turnstile is not ported, so a call the gateway would demand it for fails before it is sent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Text calls get a canned sentence, structured calls the first value their schema admits, and image calls a prepared pixel image on magenta at the model's output size. A scenario can register its own answer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
A Vite plugin at /__ai-gateway holds GOOGLE_GENERATIVE_AI_API_KEY on the server and answers gateway calls from the Gemini API, running none of the gateway's checks. It mounts only under vite serve, with a key, and outside production mode. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
The blank paint canvas, background removal, the image model's size constants without the AI SDK provider, the generated-image types and the image session hook. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
images.moderate posts an image's bytes to /v3/images/moderate and imageModerationVerdict judges Azure's severities, as apps' moderateImage did. The transport now sends a Blob body as its bytes. The mock answers every image as safe, and the gateway client takes the llm_safety_judge phase. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
@code-dot-org/lab/ai carries aichat's text and image judges and the moderation-plus-judge check for generated images, over core's gateway and moderation APIs. Apps put the image judges behind DCDO switches that default on; with no DCDO here they are on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Single images and character sets generate through core's gateway client and are checked by lab-base's safety module, with apps' tests. Saving to a level's starter assets waits on a core API. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
The gallery, the new-image form and the details dialog, with generation, painting, rename and delete; a rename or delete reaches every scene, World grid and the open workspace. A freeplay fixture offers the paint tools. The image-prompt event is a metric without the student's prompt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Guide takes collapsed, a panelRef and a width in px, and exports the offsets siblings lay out around, as apps' Guide gained after the port. NavigationArea is exported from ./instructions for a lab's own guide. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
The guide floats over a level with a mode, stepping through its guideSteps, narrowing on Play and collapsing on freeplay. An image level is the image pane alone, with the guide over it and no engine. An image fixture shows the guide moving on once a character is made. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
showStartOverDialog was declared as (type, message) but implemented as (message), and StartOverDialog ignored the message it was passed, so a lab's own warning, such as Sprite Lab's that images go too, never appeared. A custom request now shows its message, and a text or blocks request the standard one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
A level toolbox listing a blocksCommon block now keeps it, where the registered-block filter dropped it. The lab's image dropdowns take the config helpers from @code-dot-org/blockly. The behavior picker's empty list matches apps' Lab2, which never mounts the behavior editor. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
projects.getPersonal reads GET /api/v1/projects/personal, the signed-in user's own projects; the mock lists none. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
registerLabFixtures cleared a lab's routes before registering its fixtures, so a route a fixture module registered itself was gone by the first request. A fixture's routes field is registered under its own scenario instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
The play-music block offers the user's Music Lab projects on the default pack, and in Play a scene loops its song through Music's ProjectPlayer, loaded on the first song. A project whose saved blocks name a song loads its workspace once the list is in. A music fixture plays an electro loop from the real library. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Legacy (non-Lab2) GamelabJr levels, 3,682 of them in 142 units, run on the package's own interface; everything but the Piskel editor is in scope. The plan records the scale, what carries over, the decisions made, two fixes that also affect Lab2 data, and the work in twelve layers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
`yarn dev` in the package now hosts the lab in LabHost under the mock service worker and the CodeAI theme, as Studio does, so the fixtures load at /app/projects/sprite/<fixture>/edit without Studio or dashboard. The dev server also takes a developer's own AI key through the gateway proxy. The README describes the lab as it stands and how to run it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
A LabFixture's aiImages rules, tested in order against the request's text, pick the image the mock AI gateway returns. An edit request no rule matches gets back the image it sent, so an animation's frames stay one character; anything else still gets the canned character. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
The fixtures' aiImages rules tell a background, a ground block and a smooth sprite apart by the wording the lab adds to each prompt, and answer with a prepared image: a meadow, or a night sky when the prompt says night, and a grass tile, each in pixel art when that style is chosen. The wording moves into promptClauses.ts, which both the prompts and the rules use, so a rewording cannot strand the rules. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
The fixture reads the level's .level file for its toolbox, block pools, helper libraries, image roles and guide, so it follows the course. Its project has a story and a platform scene, which is what splits the authored toolbox, and a world with a floor to stand on. The pool loader, renamed blockPools.ts, now loads GameDev beside GamelabJr. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
Served from an unknown host, core falls back to the development API origin, http://localhost-studio.code.org:3000. On an HTTPS host such as GitHub Pages the browser blocks that as mixed content before the mock worker can answer. The pages build now carries core's cdo-api-url meta tag set to same-origin; the dev server is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
The page object waited for the placeholder "Sprite Lab" heading the package started with; the harness renders the lab itself, so it now waits for the Code tab and checks that the scene's workspace shows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qhPT35e41HtRWzPiYxY1
wilkie
force-pushed
the
ngfp/sprite-lab-package
branch
from
October 2, 2026 03:21
fbaa697 to
c1208cb
Compare
wilkie
force-pushed
the
ngfp/music-lab-package
branch
from
October 2, 2026 03:21
86e0706 to
0c6fe6f
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Links
Testing story
Deployment notes
Privacy and security