Skip to content

Ship the turbo extension as a shared core plus a thin extension per PHP version - #6703

Merged
ondrejmirtes merged 6 commits into
2.3.xfrom
turbo-shared-core
Oct 8, 2026
Merged

ondrejmirtes merged 6 commits into
2.3.xfrom
turbo-shared-core

Conversation

@ondrejmirtes

@ondrejmirtes ondrejmirtes commented Oct 8, 2026 •

Copy link
Copy Markdown
Member

The distributed turbo binaries were one self-contained ~7 MB extension per PHP version and platform: 33 files, 226 MB in phpstan/phpstan, nearly all of it the same machine code repeated. This ships one shared core per platform and a thin (~70 KB) extension per PHP version that loads it from its own directory.

How the core stays correct on every PHP version. Everything except Abi.cpp, Shadow.cpp, TrustedTypes.cpp and main.cpp compiles to identical code against the 8.3, 8.4, 8.5, 8.6 and 8.6 ZTS headers. Whatever moved between minors is read through pt_abi (abi.h, filled per minor by Abi.cpp): EG()/CG() fields, zend_class_entry members after 8.6's ce_flags2, object handler offsets, zend_function_entry, ZPP numbering, zend_arg_info's stride, known-string ids, functions that changed signature or stopped being exported. The new turbo-shared-core-gate job (bin/shared-core/check-linux.sh) proves that on every change with clang in the CI images, and the commit job waits for it.

Build. make SPLIT=1 builds the pair and make SPLIT=1 extension links an extension against an existing core. On Windows, configure --enable-phpstan-turbo=core|extension; the extension delay-loads the core. A plain make, phpize and PIE still build one self-contained extension.

CI.

  • turbo-compile-core* builds one profile-guided core per platform, two on Windows (NTS and TS, because the core imports php8.dll or php8ts.dll). These are what turbo-origins now reuses from earlier runs.
  • turbo-compile* link each PHP version's extension against its platform's core in about a minute and upload both files.
  • turbo-dist-layout.sh maps the artifacts onto turbo-ext/<platform>/phpstan_turbo_core.* next to phpstan_turbo-<minor>.*, and refuses a set where two legs of a platform carry different cores.

Safety. The extension refuses a core of another version: it registers nothing and reports core-mismatch, so the enabler rejects it. TurboExtensionSelector offers no binary when the core is missing.

Verified locally

  • Linux, in the CI images: one PGO+LTO core (6.73 MB stripped) with extensions for 8.3, 8.4, 8.6 and 8.6 ZTS; smoke.php passes ALL OK on each.
  • macOS 8.5: the split passes smoke.php, analysis output is identical, and the benchmark difference is +0.18% (noise).
  • Windows has never been built; this PR's CI run is its first.

turbo-ext/README.md ("Shared core") documents the layout, the mechanisms, the gate and the procedure for supporting a new PHP version.

🤖 Generated with Claude Code

https://claude.ai/code/session_017h1GfB7vfJPgwcTHD9Fth6

Closes phpstan/phpstan#15423

Comment thread .github/workflows/phar.yml Fixed
Comment thread .github/workflows/phar.yml Fixed
Comment thread .github/workflows/phar.yml Fixed
Comment thread .github/workflows/phar.yml Fixed
# it runs on ubuntu-latest without a container and carries that core
# over (runs-on and container cannot read env, hence the lookup of
# ORIGIN_RUN_ID repeated in both).
runs-on: ${{ fromJSON(needs.turbo-origins.outputs.origins || '{}')[format('phpstan_turbo_core-{0}', matrix.target.name)] && 'ubuntu-latest' || matrix.target.runs-on }}
# The arm64 counterpart of the linux-musl-x86_64 leg of
# turbo-compile-core, driving an Alpine container through docker exec
# (see turbo-compile-musl-arm64 for why it cannot be a container leg).
runs-on: ${{ fromJSON(needs.turbo-origins.outputs.origins || '{}')['phpstan_turbo_core-linux-musl-arm64'] && 'ubuntu-latest' || 'ubuntu-24.04-arm' }}
# extension — the tests of the turbo-differential jobs load it from
# there, and the commit job checks that every leg of a platform carries
# the same core.
runs-on: ${{ matrix.target.runs-on }}
# the extension DLL it loads (turbo-compile-windows builds each with its
# minor's), and a core linked with an older toolset runs on every newer
# VC runtime.
runs-on: ${{ fromJSON(needs.turbo-origins.outputs.origins || '{}')[format('phpstan_turbo_core-windows-x86_64{0}', matrix.ts == 'zts' && '-zts' || '')] && 'ubuntu-latest' || 'windows-2022' }}
[ -n "$DLL" ] && [ -n "$LIB" ]
cp "$DLL" "$LIB" core-out/
ls -l core-out
echo "CORE_DIR=core-out" >> "$GITHUB_ENV"

- name: "Point the upload at the reused core"
if: env.ORIGIN_RUN_ID != ''
run: echo "CORE_DIR=turbo-ext" >> "$GITHUB_ENV"
# with a newer toolset generation than the core. The windows-2022 image
# is the one that ships VS2022.
# PHP 8.6's vs18 development packs need the VS2026 toolchain.
runs-on: ${{ matrix.php-version == '8.6' && 'windows-2025-vs2026' || 'windows-2022' }}
# it runs on ubuntu-latest without a container and carries that core
# over (runs-on and container cannot read env, hence the lookup of
# ORIGIN_RUN_ID repeated in both).
runs-on: ${{ fromJSON(needs.turbo-origins.outputs.origins || '{}')[format('phpstan_turbo_core-{0}', matrix.target.name)] && 'ubuntu-latest' || matrix.target.runs-on }}
# Ubuntu 22.04+, Debian 12+) and the jobs run no apt at all — and
# alpine:3.24 builds the musl variant. The linux-musl-arm64 core is in
# turbo-compile-core-musl-arm64 (see turbo-compile-musl-arm64 for why).
container: ${{ !(fromJSON(needs.turbo-origins.outputs.origins || '{}')[format('phpstan_turbo_core-{0}', matrix.target.name)]) && (matrix.target.family == 'gnu' && format('ghcr.io/phpstan/turbo-build:gnu-php{0}', matrix.target.php-version) || matrix.target.container) || '' }}
PHP_MINOR: ${{ matrix.target.php-version }}
PHP_ZTS: "0"
TURBO_ARTIFACT: "phpstan_turbo_core-${{ matrix.target.name }}"
ORIGIN_RUN_ID: ${{ fromJSON(needs.turbo-origins.outputs.origins || '{}')[format('phpstan_turbo_core-{0}', matrix.target.name)] }}

- name: "Build the core"
if: env.ORIGIN_RUN_ID == ''
shell: cmd
# the extension DLL it loads (turbo-compile-windows builds each with its
# minor's), and a core linked with an older toolset runs on every newer
# VC runtime.
runs-on: ${{ fromJSON(needs.turbo-origins.outputs.origins || '{}')[format('phpstan_turbo_core-windows-x86_64{0}', matrix.ts == 'zts' && '-zts' || '')] && 'ubuntu-latest' || 'windows-2022' }}
# uses (ships bison and the unix tools phpize/configure expect).
PHP_SDK_COMMIT: "c73faaf1cce914e2fc04da5587132f8f425996af" # php-sdk-2.7.1
TURBO_ARTIFACT: "phpstan_turbo_core-windows-x86_64${{ matrix.ts == 'zts' && '-zts' || '' }}"
ORIGIN_RUN_ID: ${{ fromJSON(needs.turbo-origins.outputs.origins || '{}')[format('phpstan_turbo_core-windows-x86_64{0}', matrix.ts == 'zts' && '-zts' || '')] }}
# ships an x64 musl Node). Instead the workflow steps run on the arm64
# host and only the build commands run inside a long-lived Alpine
# container via docker exec — native arm64, no QEMU.
runs-on: "ubuntu-24.04-arm"
Comment thread .github/workflows/phar.yml Outdated

- name: "Install PHP (macOS)"
if: matrix.target.family == 'macos'
uses: "shivammathur/setup-php@f3e473d116dcccaddc5834248c87452386958240" # v2.37.2
Comment thread .github/workflows/phar.yml Outdated

- name: "Install PHP"
if: env.ORIGIN_RUN_ID == ''
uses: "shivammathur/setup-php@f3e473d116dcccaddc5834248c87452386958240" # v2.37.2

- name: "Set up MSVC environment"
if: env.ORIGIN_RUN_ID == ''
uses: ilammy/msvc-dev-cmd@0b201ec74fa43914dc39ae48a89fd1d8cb592756 # v1.13.0
@ondrejmirtes
ondrejmirtes force-pushed the turbo-shared-core branch 3 times, most recently from 94dd659 to 62c8041 Compare October 8, 2026 15:06
@ondrejmirtes
ondrejmirtes marked this pull request as ready for review October 8, 2026 15:56
@phpstan-bot

Copy link
Copy Markdown
Collaborator

This pull request has been marked as ready for review.

ondrejmirtes and others added 6 commits October 8, 2026 17:58
Spike towards one shared core library per platform with a thin
per-PHP-version extension. The engine layout that moves between minors
(EG()/CG() fields, zend_function_entry, internal_function.handler,
zend_class_entry.info, object handler offsets, ZSTR_KNOWN ids,
IS_REFERENCE_EX, ZEND_ACC_USE_GUARDS, lazy-object flags) is read through
pt_abi, which Abi.cpp fills in at module startup; engine functions whose
signature or behaviour differs (zend_create_closure, the weakref hash
helpers, zend_dval_to_lval, ZEND_TRY_ASSIGN_REF_*, the call stack size
error) are implemented once per minor in Abi.cpp; the PHP_VERSION_ID gates
in shared code became runtime checks.

With this, every object file except Abi.o, Shadow.o, TrustedTypes.o and
main.o compiles to identical code, data and relocations against the PHP
8.3, 8.4 and 8.5 headers (bin/shared-core/gate.sh, compare-data.sh). A core
dylib linked from the 8.4 objects plus three 76 KB thin extensions
(bin/shared-core/link-split.sh) loads under all three, passes smoke.php on
8.4 and 8.5 (8.3 fails the same five deep-recursion checks a pristine
2.3.x build fails on Homebrew 8.3), produces byte-identical analysis
output, and measured +0.18% (t=1.29, n=10, noise) against 2.3.x.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017h1GfB7vfJPgwcTHD9Fth6
The core no longer references anything in the version-specific library:
the pt_abi table, the module-startup registration list and the shadow
plan registry moved into the new Core.cpp, and the functions Abi.cpp
implements per minor became function pointers in pt_abi. The few core
functions the version-specific library calls are PT_CORE_API, everything
else stays hidden.

PHP 8.6 broke the remaining assumptions, now read through pt_abi too:
- zend_class_entry gained ce_flags2 after ce_flags, moving every later
  member (PT_CE(), type-checked to take a zend_class_entry only)
- the ZPP_ERROR_* codes became a uint8_t enum without WRONG_COUNT and two
  Z_EXPECTED_* types were inserted; the parse slow paths now return the
  value (neutral ZPP macros, translated by Abi.cpp)
- zend_is_callable, zend_call_known_function, _php_stream_free and
  _php_stream_read are no longer exported
- zend_arg_info grew a doc_comment (PT_ARG_INFO())
and ZEND_COLD is pinned, since the older headers enable it for GCC only.

bin/shared-core/gate-linux.sh builds in the CI images (GCC, or clang with
GATE_IMAGE/GATE_MAKE_ARGS), link-split-linux.sh links one core from the
8.4 objects plus a 68 KB extension per minor and runs smoke.php with each:
ALL OK on 8.3, 8.4, 8.5 and 8.6. The macOS scripts got a -mac suffix.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017h1GfB7vfJPgwcTHD9Fth6
The last differences between the header sets, each pinned in abi.h or
moved into Abi.cpp:
- the fresh-C-stack switch (a zend_fiber_context, its switch and a
  zend_try, whose SETJMP became sigsetjmp in 8.6) is implemented per minor
- codegen pins for helpers 8.6 re-spelled without changing what they do:
  zend_string_equals_literal_ci / _equals_ci / starts_with_literal(_ci),
  ZSTR_IS_INTERNED, array_init(_size), Z_TRY_ADDREF_P / Z_TRY_DELREF_P
- the comparison treats the zend_result rename (a different mangling of
  the same enum) and the symbol/section name tables as names, not code

clang 14 in the CI images (gate-linux.sh with GATE_IMAGE=turbo-build-clang,
-fgnuc-version=4.3 so the older headers enable ZEND_COLD for it too) now
finds no differing function, data section or relocation outside Abi.o,
Shadow.o, TrustedTypes.o and main.o across 8.3, 8.4, 8.5 and 8.6. GCC
still differs in 15 string-building functions (8.6 const-qualified
zval_get_string() and friends, inline helpers compiled inside the headers).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017h1GfB7vfJPgwcTHD9Fth6
…ake SPLIT=1

`make SPLIT=1` builds phpstan_turbo_core.so (everything but the four
version-specific sources) and a phpstan_turbo.so linked against it, which
finds it next to itself ($ORIGIN, @loader_path); `make SPLIT=1 extension`
links only the extension against a core already in place, which is what a
per-PHP-version CI leg does. `make pgo` and `make strip` handle the pair; a
plain `make` (and the phpize build) still produce the one self-contained
extension.

The boundary is functions only — pt_abi, pt_globals and pt_type_op_infos
are reached through accessors from the version-specific side, since a
Windows DLL can delay-load functions but not data — and main.cpp is
version-specific like Abi/Shadow/TrustedTypes. The extension refuses a
core of another version: it registers nothing and reports "core-mismatch",
which TurboExtensionEnabler rejects. On Windows it loads the core from its
own directory in get_module() (PHP loads extensions with plain
LoadLibrary, which would search the application's directory instead), and
zend_is_true() goes through pt_abi there (no assembler names on MSVC).

Verified in the CI images: a PGO+LTO core built in the 8.5 image (6.73 MB
stripped) with extensions built against it in the 8.3, 8.4, 8.6 and 8.6
ZTS images (68 KB each): smoke.php ALL OK on each; a version-mismatched
pair reports core-mismatch and stays inactive. macOS: the same via make
SPLIT=1, ALL OK on 8.5.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017h1GfB7vfJPgwcTHD9Fth6
…HP version

phar.yml builds one profile-guided core per platform (turbo-compile-core,
-core-musl-arm64, -core-windows for NTS and TS; the jobs turbo-origins now
reuses from earlier runs) and links each PHP version's extension against
it in the existing turbo-compile* legs, whose artifacts carry the core
next to the extension so every test job finds it there.
.github/scripts/turbo-dist-layout.sh maps the artifacts onto the dist
layout (turbo-ext/<platform>/phpstan_turbo_core.so|.dll next to
phpstan_turbo-<minor>.so|.dll) for the aggregate artifact and the commit
job, refusing legs of one platform that carry different cores.
turbo-shared-core-gate (bin/shared-core/check-linux.sh) compiles the shared
sources against every supported PHP's headers with clang and fails on any
difference outside the four version-specific sources; the commit job waits
for it.

config.w32 gains --enable-phpstan-turbo=core / =extension; the extension
DLL delay-loads the core from its own directory. TurboExtensionSelector
offers no binary when the core is missing next to the extension. The
Makefile's default goal stays the extension (it had become `objects`), and
a plain `make strip` leaves a core put in place untouched.

README.md documents the layout, the mechanisms that keep shared code
version-neutral, the gate, and the procedure for supporting a new PHP
version; turbo-ext/CLAUDE.md the rules for code in shared sources.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017h1GfB7vfJPgwcTHD9Fth6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reduce the size of the Turbo binaries in the phpstan/phpstan package

3 participants