Repository navigation
Path to adding a Tier 2 build with pointer compression enabled #3204
Description
This is a follow up to nodejs/TSC#790. The project currently creates unofficial builds for Linux with pointer compression enabled. With pointer compression enabled, the main branch appears to pass the test suite (at least on macOS where I tested). From what I can tell, v19 and earlier are not in such good shape.
We are interested in getting pointer compression into the CI, and eventually hopefully get it to Tier 2 status (on Linux would be enough). cc: @nodejs/releasers.
I think the first step would be getting this into the CI so that we can have more confidence in the current unofficial build. Unfortunately, I don't have the slightest idea what needs to happen to add a new CI target - but I am willing to do the work if someone can provide guidance.
Activity
mhdawson commentedon Feb 23, 2023
I think two questions we should get feedback on to start are:
RafaelGSS commentedon Feb 23, 2023
As a Node.js TSC and Releaser, I'm +1. I don't see it impacting the release workflow directly. However, might make sense to produce a release candidate for LTS versions first (optional)
richardlau commentedon Feb 23, 2023
Yes, I have concerns about confusion to end users in producing a second Linux download, particularly as I seem to recall concerns about ABI compatibility between regular builds and pointer compression enabled ones (think about ecosystem packages using prebuilds, for example).
cjihrig commentedon Feb 23, 2023
@richardlau would you feel better about only adding pointer compression to the CI so that we can at least have more confidence in the existing unofficial build?
richardlau commentedon Feb 23, 2023
@cjihrig my response was specifically to the quoted question. I'm neutral on adding another job to the CI.
mhdawson commentedon Mar 2, 2023
Adding another job to the CI seems resonable to me as we already have machines of the required type available and my first guess is that we would not need additional machines to support the one extra job.
cjihrig commentedon Mar 3, 2023
@mhdawson so what would be the next step to add the CI job? I'm happy to help, but don't know where/how to get started.
mhdawson commentedon Mar 15, 2023
I don't see any objections to adding a CI job, so I'll set up a job Colin can edit/work on and then we can take it from there. It probably will mostly be a clone of some other job.
mhdawson commentedon Mar 16, 2023
Added this job with @cjihrig - https://ci.nodejs.org/job/node-test-commit-linux-pointer-compression.
Just letting it run to confirm it passes and then will add to the daily main run.
mhdawson commentedon Mar 16, 2023
To answer a question people might ask we did not add as a containerized build as I was not sure how to do that and only have it run in the nightly.
mhdawson commentedon Mar 16, 2023
Configured the job so that @cjihrig can edit/modify the job in cases changes/fixes are needed.
mhdawson commentedon Mar 17, 2023
Had to fixup the addition to the daily main job as it did not run today. Believe it should run next time.
cjihrig commentedon Mar 31, 2023
@mhdawson thanks for the help getting the pointer compression job into the daily job. It looks like it has passed every time it has run for the past few weeks with the exception of one time when GitHub changed the key.
As a next step, does it make sense to add pointer compression to the CI for pull requests on
main(from my testing, v19 and older do not work properly with pointer compression)?mhdawson commentedon Apr 4, 2023
The part about only running on v19 and later is one that may require something special. @richardlau I don't think the VersionSelectorSecript can help us on this one right?
richardlau commentedon Apr 5, 2023
That is exactly what the VersionSelectorScript is intended for. However it works on labels (specifically on a build parameter named "nodes"), so you may need to rework the job and/or introduce new labels.
20 remaining items
mikeduminy commentedon Apr 22, 2024
I'd love to be able to use an official build that enables pointer compression on mac. Would that need a separate build?
mcollina commentedon Apr 22, 2024
yes
ronag commentedon Aug 5, 2024
Did we give up on pointer compression? I've been unable to try it out. Haven't been able to find a recent build nor have been able to figure out how to create a build myself. Hard to give feedback (for anyone) with the current state of things.
richardlau commentedon Aug 5, 2024
Nobody appears to be paying any attention to it.
--experimental-enable-pointer-compressioncauses build failure node#51339 was last updated in January.jasnell commentedon Aug 5, 2024
No, we haven't given up and definitely still are paying attention. Cloudflare has engaged a contract with Igalia to look at a path that hopeful would enable, in part, Node.js to enable pointer compression by default in the future.
Specifically, one of the limiting factors with enabling pointer compression to this point has been the strict limit of a single 4gb pointer cage per-process. Limiting the entire process to a single 4gb limit would be too much of a breaking change.
To support the alternative configuration we have in Cloudflare Workers, for a different use case that touches on the same parts of v8, Igalia is working on a way to enable multiple pointer cages per process. We call the new API an
IsolateGroup. EachIsolateGrouprepresents one cage, withNIsolates allowed perIsolateCage. A process would allow any number ofIsolateGroups to be created. EachIsolateGroupstill has the 4gb limit but the ability to create multiple cages per process is a lot more tractable than a single 4gb cage for the entire process.While it would still definitely be a semver-major and potentially breaking change, I think this approach would make it far more likely that we can enable pointer compression by default in Node.js or at least provide a release distribution with pointer compression enabled.
ronag commentedon Aug 5, 2024
FWIW this is not a problem for us. I don't think it's a problem for a majority of users. IMHO two separate builds would be fine. Advanced users can use the pointer compression build.
mcollina commentedon Aug 6, 2024
In the past, I thought we (Platformatic) could sponsor some of this infra work, but startup life got in the way, and this is not a priority anymore. We would definitely use a Node.js build with pointer compression enabled, but the main driver for investing to make it happen is gone.
jasnell commentedon Aug 7, 2024
My hope is that the new
IsolateGroups will at least make it less of a potentially breaking change. We have users, however, that depend on extremely large heap sizes that, even with compression, go well beyond the 4gb limit so we have to be careful. Having theIsolateGroupAPI there should also make it easier for us to transition to this mode in general.SeanReece commentedon Aug 8, 2024
At the risk of beating a dead horse here I have to agree. The max-old-space-size defaults to 2GB on 64bit systems and I’m willing to bet most node developers have not had a use case to increase this. Even having pointer compression be the default would increase the default heap limit 2x and advanced developers could just switch to the non-compressed build for use cases requiring more heap space.
What @jasnell is working on sounds very interesting and promising. But I hope we can get the pointer compression build back on track regardless.
jdmarshall commentedon Sep 5, 2024
@jasnell What do you mean by well beyond 4GB? If you have lots of object graphs it's entirely possible for 6GB heap to comfortably fit into 4GB with compression enabled.
Also I haven't heard anybody petitioning for eliminating 64 bit pointers. The argument is which is the default case and which is the special case. Right now it's very much backward from reality.
jdmarshall commentedon Sep 5, 2024
@ronag I would take the opposite conclusion from this. If 4GB cage isn't a problem for most users, then the advanced users should be the ones looking at the uncompressed build, not the other way around.
jasnell commentedon Sep 5, 2024
I've talked with folks that run in excess of 20gb heaps in certain extreme cases. I think Netflix has also seen this but I'd need to confirm again. I do think the excessively sized heaps are not the norm and experience has shown that the vast majority of cases can fit well within a 4gb compressed heap.
jdmarshall commentedon Sep 25, 2024
@cjihrig completed?
cjihrig commentedon Sep 25, 2024
No. Clearing my issues list. If anyone needs this issue, please open a new one.