Repository navigation
chore: migrate to pnpm and Biome, and harden the MCPB bundle build - #37
Conversation
Ports tomtom-internal/tomtom-traffic-analytics-mcp#34 (commits a66b3dd and 8ddce10) to this repository. Replaces npm with pnpm as the package manager, and the ESLint + Prettier stack with Biome, matching the tooling in tomtom-maps-mcp (#78). Why: one tool (Biome) replaces the ESLint + Prettier plugin/config stack and runs in milliseconds instead of seconds; pnpm installs faster from a content-addressed store; both match the tooling the team already uses. - `overrides` and the new `allowBuilds` live in pnpm-workspace.yaml; pnpm 11 no longer reads them from package.json. - Added a .dockerignore. The Dockerfile does `COPY . .` after installing, which would otherwise copy the host's node_modules over the image's. - Dropped the `@rollup/rollup-linux-x64-gnu` optionalDependency: it was pinned to 4.9.5 against rollup ^4.62, and rollup resolves its own platform binary. - CI gains a `pnpm type-check` step, and pnpm/action-setup is pinned to a SHA like the other third-party actions here. - Biome covers `src/`, `scripts/`, `tests/` and the root config scripts, not just `src/` — which nothing was checking before. JSON stays out of scope on purpose: prepare-release.yml rewrites package.json and manifest-binary.json with `yq`, whose formatting would then fail `pnpm format` on the first PR after every release. - A .gitattributes with `* text=auto eol=lf`, because biome.json pins `lineEnding: "lf"` and a Windows checkout with core.autocrlf=true would fail `pnpm format` on every file. CI never sees it; the runners check out LF. - Drops the dead `version:sync` script and scripts/generate-version.cjs. prepare-release.yml has done that work with `yq` since LSI-305. `nodeLinker: hoisted` rather than pnpm's default isolated layout, because that layout is unusable on Windows at an ordinary checkout depth. It inserts `.pnpm/<pkg>@<ver>_<peers>/node_modules/<pkg>/` into every dependency path, putting @rollup/pluginutils/dist/es/package.json at 266 characters. Past MAX_PATH with LongPathsEnabled=0 that read fails *silently*: Node cannot see the `{"type":"module"}` marker, falls back to the package root's "commonjs", and parses the ESM file as CJS — SyntaxError: Named export 'createFilter' not found. The requested module '@rollup/pluginutils' is a CommonJS module which points nowhere near the cause. CI cannot catch it because runner paths are short. `allowBuilds: esbuild` is not strictly needed — current esbuild resolves its platform binary from optionalDependencies and the full suite passes under `--ignore-scripts` — but it keeps a plain `pnpm install` from printing an "ignored build scripts" warning and sending contributors to `pnpm approve-builds`. CI installs with `--ignore-scripts`, because this package's own `prepare` hook is `pnpm run build`: without it every job builds the whole project during install and again in its explicit build step. `publish-npm` needs it twice over, since `prepare` runs on publish as well. `verifyDepsBeforeRun: false` is a deliberate divergence from maps-mcp. pnpm 11 defaults it to `install`, so every `pnpm run <script>` re-verifies node_modules and silently re-installs when it looks out of sync. On `RUN pnpm run build` that re-installed on top of the `--ignore-scripts` install and fired `prepare` — itself `pnpm run build` — so the image built everything twice. maps-mcp avoids the same double build by copying sources before installing, which costs it the Docker layer cache on every source change. This keeps the cache and turns the check off instead. Note for contributors: `pnpm run <script>` no longer auto-syncs node_modules after a branch switch — run `pnpm install` yourself, the same contract npm has always had. Divergences from the upstream PR, which was cut against a `main` that does not carry this repository's later fixes: - The Dockerfile keeps the unprivileged `USER node` step from #25. - Docs and .npmignore keep `manifest-binary.json`, this repository's manifest filename since #18, and the "TomTom Maps MCP Server" companion name from #2. - The upstream lockfile was not reusable: this repository is ahead on vitest (^4.1.8 against ^3.2.4) from #24, so pnpm-lock.yaml is generated here. Biome reports 178 warn-level findings (noExplicitAny, useImportType, noNonNullAssertion, cognitive complexity) that do not fail the build; these are left for follow-up rather than mixed into this migration. The one error, noPrecisionLoss on a verbatim API fixture in routeMonitoringService.test.ts, is suppressed in place with the reason. Verified: pnpm install --frozen-lockfile, type-check, lint (0 errors), format, build, 285 unit tests across 35 files, and docker build + /health returning {"status":"ok","version":"0.2.1"} as uid 1000. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: daniel.forniessoria <daniel.forniessoria@tomtom.com>
Ports tomtom-internal/tomtom-traffic-analytics-mcp#34 (commit e82ef15). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: daniel.forniessoria <daniel.forniessoria@tomtom.com>
Ports the guards from tomtom-internal/tomtom-traffic-analytics-mcp#34 (commit 4394aa4) onto this repository's existing self-contained `type: binary` builder, rather than replacing it. That builder already produces one bundle per platform, each shipping its own Node runtime so the launcher is spawned as `type: binary` — which is what avoids the macOS Hardened Runtime library-validation restriction on dlopen of non-Anthropic-signed native modules inside Claude Desktop's Electron UtilityProcess sandbox. The upstream PR was written against a `main` that never had it, and its `type: node` rewrite would have reintroduced that failure without CI noticing, since the smoke test runs the server under a plain `node`. What upstream adds and this repository lacked is everything around the builder: - `build-mcpb.yml` ran a single `ubuntu-22.04` job and uploaded `dist/mcpb/tomtom-traffic-analytics-mcp.mcpb`, a path the per-platform script has not produced since #15 — so the workflow could only ever fail. It is now a matrix over five native runners: darwin-arm64, darwin-x64, linux-x64, win32-x64 and win32-arm64. It runs on pull requests too, bundles being the one artifact a Linux-only CI run cannot vouch for. - The bundle no longer carries the devDependency tree. It copied the whole of node_modules; it now stages a dedicated `pnpm install --prod --frozen-lockfile` and drops pnpm's `.pnpm`, `.bin` and bookkeeping. Measured here: 9595 files and 321 MB of node_modules against 2842 total bundle entries, with no vitest, typescript, rollup, biome or esbuild left in. Four guards, since a bundle that cannot run still packs perfectly: - the staged tree is rejected if `@duckdb/node-bindings-<platform>-<arch>` is absent, which is the whole reason to build on a native runner - it is also rejected if it holds any symlink. The old copy step called `symlinkSync` for each one it met, and a link only survives an extractor that restores it — otherwise the target is written out as a short text file and the server dies on its first require. pnpm's hoisted layout still leaves 30 in node_modules - `MCPB_EXPECT_TARGET` fails a matrix job whose runner is not the platform its row claims; bundles are named from the matrix but built from the host - `scripts/verify-mcpb.cjs` unpacks the bundle, asserts the launcher and the embedded Node binary kept their executable bit, runs SQL through the bundle's own runtime and duckdb binding, then completes an initialize/tools-list handshake against the extracted server Packing moves from `archiver` to `mcpb pack`, which validates the manifest against the published schema before writing and stores Unix modes — dropping a dependency and letting `mcpb unpack` do the extraction in the verifier. `release.yml` globs `./dist/mcpb/**/*.mcpb` and asserts one bundle per matrix entry, the count read from the matrix with `yq`; otherwise `fail-fast: false` plus one red platform would quietly ship a release missing a bundle. Because `compatibility.platforms` has no arch field, both macOS bundles advertise `["darwin"]` and only the filename separates arm64 from x64, so every release now carries a machine-to-file table in its notes. `.mcpbignore` goes away: nothing packs the repository directory any more. Verified on darwin-arm64: build:mcpb produces a 75.8 MB bundle in 19s and verify:mcpb passes it — unpacked, executable, duckdb running SQL, 8 tools over a real MCP handshake. The target guard was confirmed to fail a deliberate darwin-arm64/win32-x64 mismatch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: daniel.forniessoria <daniel.forniessoria@tomtom.com>
a23b1a7 to
ce71e59
Compare
carlosprietofernandez-tomtom
left a comment
There was a problem hiding this comment.
LGTM, Thanks Dani!
| # *silently*, falls back to the package root's "commonjs" and parses the ESM file as CJS | ||
| # — surfacing as `SyntaxError: Named export 'createFilter' not found` from `pnpm build`, | ||
| # which points nowhere near the cause. CI never sees it because runner paths are short. | ||
| nodeLinker: hoisted |
There was a problem hiding this comment.
minor: don't we lose the pnpm install speed/size benefits by setting this? Maybe the Windows users can enable long paths on their machines instead?
There was a problem hiding this comment.
This is indeed config inherited from before making sure Windows developers can contribute to the MCP.
Quickly using Claude for help and clarification:
Measured it instead of guessing, and we don't lose anything: node_modules is 321 MB either way, and install time is a wash (isolated 4.86/6.65/5.12s vs hoisted 5.82/5.77/5.11s, warm store, frozen lockfile). The dedup we care about comes from packageImportMethod — hoisted still links out of the same content-addressed store, nodeLinker only decides symlinks vs real dirs. It's even marginally fewer files hoisted (9,590 vs 9,713), since this tree has almost no multi-version deps.
You did catch that my justification was wrong though: the silent package.json misread I described was a Node bug, fixed in 22.7.0 (nodejs/node#53294), and we require >=22.9. Comment rewritten.
What keeps me on hoisted is path headroom, not that bug. Longest path from the repo root is 115 chars hoisted vs 190 isolated (with Windows' default virtualStoreDirMaxLength: 60). A plain C:\Users\<first>.<last>\IdeaProjects\tomtom-traffic-analytics-mcp is already 63, so isolated leaves ~6 chars before MAX_PATH — OneDrive-redirected Documents or a worktree subdir busts it, and git/tsc don't namespace long paths the way Node now does. On enabling long paths: it needs an admin HKLM value plus a longPathAware manifest node.exe still doesn't ship (nodejs/node#40641), plus core.longpaths for git — can't really ask that of contributors on a public repo.
The real cost is losing strict isolation (an undeclared dep resolves locally but breaks in the published bundle). There are none today — isolated install + type-check + build + 285 tests all green — happy to add a CI job that installs with --config.node-linker=isolated if you want that guarded going forward.
The old comment blamed a Node bug: package.json read silently failing past
MAX_PATH, so `{"type":"module"}` was missed and ESM parsed as CJS. That was
fixed in Node 22.7.0 (nodejs/node#53294, which namespaces fs, ESM resolution
and package.json reads) and this package requires >=22.9.0, so the mechanism
the comment described cannot be why we need the setting.
The reason that does hold up is headroom, measured on this lockfile: the
longest path relative to the repo root is 115 characters hoisted against 190
isolated (at Windows' default virtualStoreDirMaxLength of 60). An ordinary
C:\Users\<first>.<last>\IdeaProjects\tomtom-traffic-analytics-mcp checkout is
63 characters, leaving isolated ~6 before MAX_PATH — a OneDrive-redirected
Documents folder or a worktree subdirectory exceeds it. Enabling long paths
instead needs an admin registry value plus a longPathAware manifest node.exe
does not ship (nodejs/node#40641), which a public repo cannot require.
Also records what the setting does and does not cost, since that was the
review question. It costs strict isolation: an undeclared dependency resolves
locally but breaks in the published bundle. It costs nothing measurable
otherwise — 321 MB and equal install times either way (isolated 4.86/6.65/
5.12s against hoisted 5.82/5.77/5.11s, warm store, frozen lockfile), because
the store dedup comes from packageImportMethod, not from nodeLinker.
Comment only; no behaviour change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Description
Ports tomtom-internal/tomtom-traffic-analytics-mcp#34
to this repository: npm → pnpm and ESLint + Prettier → Biome, plus the guards that PR
added around the
.mcpbbundle build.Biome replaces the whole ESLint + Prettier plugin/config stack and runs in milliseconds instead
of seconds; pnpm installs faster from a content-addressed store; both match what the team
already uses. Full reasoning is in the commit messages; this is the summary.
1. pnpm + Biome (
1e17407)packageManager: pnpm@11.21.0,pnpm-lock.yaml, andbiome.jsonreplacingeslint.config.js,.prettierrc,.prettierignoreand.eslintignore. The rule set is selected withlinter.rules.preset: "recommended"— the olderrecommended: truefield is deprecated inBiome 2.5 and goes away in the next major.
overridesandallowBuildsmove topnpm-workspace.yaml, which pnpm 11 requires. All workflows, the Dockerfile, VS Code tasks anddocs follow; CI gains
pnpm type-checkand pinspnpm/action-setup. Dropped the@rollup/rollup-linux-x64-gnuoptionalDependency, pinned to 4.9.5 against rollup^4.62.Three settings are load-bearing, each documented in place:
nodeLinker: hoisted@rollup/pluginutils/dist/es/package.jsonat 266 chars; past MAX_PATH that read fails silently, so Node misses{"type":"module"}and parses ESM as CJS.verifyDepsBeforeRun: falseinstall, soRUN pnpm run buildre-installed and firedprepare, itselfpnpm run build, building the image twice. The one deliberate divergence from maps-mcp.--ignore-scriptsprepareispnpm run build, so otherwise every job builds during install and again in its build step.Contributor note:
pnpm run <script>no longer auto-syncsnode_modulesafter a branchswitch. Run
pnpm installyourself, the same contract npm always had.Biome now covers
src/,scripts/,tests/and the root config scripts, whereeslint srccovered only
src/— 5 files outsidesrc/reformat mechanically as a result. JSON stays out ofscope on purpose:
prepare-release.ymlrewritespackage.jsonandmanifest-binary.jsonwithyq, whose formatting would then failpnpm formaton the first PR after every release. A.gitattributeswith* text=auto eol=lfkeepspnpm formatfrom failing on every file on aWindows checkout. Also drops the dead
version:syncandscripts/generate-version.cjs,superseded by
prepare-release.ymlsince LSI-305.2. The
.mcpbbuild was hardened, not replaced (ce71e59)Upstream #34 also rewrote
scripts/build-mcpb.cjs, because on internalmainthe bundle waspacked by
mcpb pack .on a Linux runner and only ever worked on Linux. That does not applyhere: this repository has produced one self-contained bundle per platform since #15, each
shipping its own Node runtime so the launcher is spawned as
type: binary— which is what avoidsthe macOS Hardened Runtime library-validation restriction on
dlopenof non-Anthropic-signednative modules inside Claude Desktop's Electron UtilityProcess sandbox. Upstream's
type: noderewrite would reintroduce that failure, and its CI smoke test would not catch it, since it runs
the server under a plain
node.So the builder stays and everything around it comes across:
build-mcpb.ymlcould only ever fail. It ran a singleubuntu-22.04job and uploadeddist/mcpb/tomtom-traffic-analytics-mcp.mcpb, a path the per-platform script has not producedsince Tool description audit, pipeline fixes, and per-OS .mcpb packaging #15. It is now a matrix over five native runners —
darwin-arm64,darwin-x64,linux-x64,win32-x64,win32-arm64— and runs on pull requests too, bundles being the oneartifact a Linux-only CI run cannot vouch for.
node_modules; it nowstages a dedicated
pnpm install --prod --frozen-lockfileand drops pnpm's.pnpm,.binandbookkeeping. Measured here: 9595 files / 321 MB of
node_modules, against 2842 total bundleentries with no vitest, typescript, rollup, biome or esbuild left in.
Four guards, since a bundle that cannot run still packs perfectly:
@duckdb/node-bindings-<platform>-<arch>is absent, which is thewhole reason to build on a native runner
symlinkSyncfor each oneit met, and a link only survives an extractor that restores it — otherwise the target is written
out as a short text file and the server dies on its first
require. pnpm's hoisted layout stillleaves 30 in
node_modulesMCPB_EXPECT_TARGETfails a matrix job whose runner is not the platform its row claims; bundlesare named from the matrix but built from the host
scripts/verify-mcpb.cjsunpacks the bundle, asserts the launcher and the embedded Node binarykept their executable bit, runs SQL through the bundle's own runtime and duckdb binding, then
completes an initialize/tools-list handshake against the extracted server
Packing moves from
archivertomcpb pack, which validates the manifest against the publishedschema before writing and stores Unix modes — one dependency fewer, and
mcpb unpackdoes theextraction in the verifier.
.mcpbignoregoes away; nothing packs the repository directory now.release.ymlglobs./dist/mcpb/**/*.mcpband asserts one bundle per matrix entry, the count readfrom the matrix with
yq; otherwisefail-fast: falseplus one red platform would quietly ship arelease missing a bundle. Because
compatibility.platformshas no arch field, both macOS bundlesadvertise
["darwin"]and only the filename separates arm64 from x64, so every release now carriesa machine-to-file table in its notes.
Other divergences from upstream #34
Upstream was cut against a
mainthat does not carry this repository's later fixes, so:USER nodein the Dockerfilebuild-mcpb.yml(multi-line commit messages break inline interpolation inside[[ ]])manifest-binary.jsonas the manifest filename, in docs,.npmignoreand both workflows^4.1.8over upstream's^3.2.4, sopnpm-lock.yamlis generated here rather than copiedUpstream commits
a66b3ddand8ddce10are combined into1e17407here, since both are Biomeand pnpm configuration and splitting them would have meant two lockfile states for no reviewer
benefit.
Type of change
build-mcpb.ymluploading a path thebuild script does not produce
and
verify:mcpbHow Has This Been Tested?
pnpm install --frozen-lockfilefrom a clean tree;type-check,lint(0 errors, 178pre-existing warnings),
format,build, and 285/285 unit tests across 35 filesdocker build+docker run:/healthreturns{"status":"ok","version":"0.2.1"}, andidinside the container reportsuid=1000(node)build:mcpbon darwin-arm64: 75.8 MB bundle in 19s, 2842 entries, 0 symlinks, nodevDependencies and no pnpm bookkeeping in the archive
verify:mcpbon that bundle: unpacked, launcher and embedded Node still executable, duckdbexecuting SQL, 8 tools over a real MCP handshake
MCPB_EXPECT_TARGETguard confirmed to fail a deliberatedarwin-arm64/win32-x64mismatch
The other four platform bundles are built and verified by this PR's own
build-mcpb.ymlrun — thatis the point of the matrix. Integration tests (
test:comprehensive,test:http) run in CI wherethe API keys are available.
Reviewer notes
on every PR. Deliberate, but the
pull_requesttrigger can be narrowed to bundle-affectingpaths.
tsconfig.jsonstill type-checks onlysrc/, soscripts/andtests/getlint and formatting but no types. The 178 Biome warnings are untouched.
design. Whoever reconciles internal feat: MCP apps for traffic tools (incidents, area analytics, flow, junctions, routes) #26 (which carries the
type: binarybuilder there) shouldland this shape, not upstream chore(deps): bump hono from 4.12.27 to 4.13.1 #34's.
Checklist:
verify-mcpb.cjsruns per platform in CI
🤖 Generated with Claude Code