LoongArch to Experimental tier (CI environment)

I would like to discuss here how to put LoongArch machine access nodejs/build CI environment, test machine for LoongArch nodejs.

Best regards

@sxa

@sxa #3511 (comment)
Sorry, I've been testing nodejs(main v18. x v20.x), reply late

The question in my mind is what are we expecting to gain from adding it to the CI?

LoongArch platform can follow up the latest status of node, solve problems in the process, and promote the development of loong64-node.

What is the outlook for getting the tests passing cleanly? If it was added into the regular CI then we'd want it to run without errors otherwise it will cause problems interpreting the test results potentially resulting in delays to validating PRs.

OS: LoongArch Fedora remix

latest v18/v20/main

wget -c https://nodejs.org/download/release/latest-v18.x/node-v18.18.2.tar.gz
wget -c https://nodejs.org/download/release/latest-v20.x/node-v20.9.0.tar.gz
git clone https://github.com/nodejs/node.git

may need to apply this patch

nodejs/node#50209
nodejs/node#50363
#v20.x: may need to apply this patch
nodejs/node#50210

build

./configure --openssl-no-asm
time make -j4

check

export OPENSSL_CONF="./deps/openssl/openssl/apps/openssl.cnf"
time make test-ci-js 2>&1 | tee ../E.log-nodejs-ci

latest-v18/latest-v20/main

The latest test-ci-js version of v18/v20/main failed 3 tests

test/parallel/test-crypto-sign-verify.js
test/parallel/test-vm-global-setter.js
test/parallel/test-vm-module-synthetic.js

What is the performance of the systems compared to the other architectures (since that affects the execution time of CI jobs)? I suspect that it us unlikely that we could integrate it into the regular CI without having a significant negative impact on the run times of the PR testing.

latest-v18

(1) time make -j4

...
real    87m53.900s
user    215m25.687s
sys     6m22.889s

(2) time make test-ci-js 2>&1 | tee ../E.log-nodejs-ci

...
Failed tests:
out/Release/node /home/shipujin/node-v18.18.2/test/parallel/test-crypto-sign-verify.js
out/Release/node /home/shipujin/node-v18.18.2/test/parallel/test-vm-global-setter.js
out/Release/node --experimental-vm-modules /home/shipujin/node-v18.18.2/test/parallel/test-vm-module-synthetic.js
make: *** [Makefile:539：test-ci-js] Error 1

real    25m25.999s
user    37m11.585s
sys     1m44.890s

latest-v20

(1) time make -j4

...
real    95m7.574s
user    247m31.239s
sys     6m45.546s

(2) time make test-ci-js 2>&1 | tee ../E.log-nodejs-ci

...
Failed tests:
out/Release/node /home/shipujin/LTS/node-v20.9.0/test/parallel/test-crypto-sign-verify.js
out/Release/node /home/shipujin/LTS/node-v20.9.0/test/parallel/test-vm-global-setter.js
out/Release/node --experimental-vm-modules /home/shipujin/LTS/node-v20.9.0/test/parallel/test-vm-module-synthetic.js
make: *** [Makefile:535: test-ci-js] Error 1
                                                                    
real    21m28.480s
user    34m21.144s
sys     3m20.023s

main

(1) time make -j4

...

real    93m59.754s
user    354m6.669s
sys     9m37.371s

(2) time make test-ci-js 2>&1 | tee ../E.log-nodejs-ci

...
Failed tests:
out/Release/node /home/shipujin/node/test/parallel/test-crypto-sign-verify.js
out/Release/node /home/shipujin/node/test/parallel/test-vm-global-setter.js
out/Release/node --experimental-vm-modules /home/shipujin/node/test/parallel/test-vm-module-synthetic.js
make: *** [Makefile:535：test-ci-js] Error 1
                                                                    
real    27m55.096s
user    37m24.999s
sys     3m21.715s

Having said that, as Michael says, I'm not against having machines available for interested parties to progress the status of the port and work towards having it in place, and I can't think of any objection, given the assurances above, that it could be moved to being listed as an experimental platform.

There are two questions about the way LoongArch machine is provided, hope you can help me, thanks~
(1) How are LoongArch machines provided to nodejs?
I can provide a LoongArch machine, the machine runs in my company's machine room. Later I can provide an ssh access to the LoongArch machine ip, is this allowed?
(2) How to connect LoongArch machine to nodejs/build project for CI test?

Best regards

@nodejs/build @mhdawson @richardlau @sxa @targos

(2) How to connect LoongArch machine to nodejs/build project for CI test?

The machine must be able to run the Jenkins agent (Java 11 or 17)

(2) How to connect LoongArch machine to nodejs/build project for CI test?

The machine must be able to run the Jenkins agent (Java 11 or 17)

21'should also be acceptable now - I can't remember if the loongarch support was backported to JDK17

(2) How to connect LoongArch machine to nodejs/build project for CI test?

The machine must be able to run the Jenkins agent (Java 11 or 17)

Yes, the running system is java17.

[shipujin@bogon node]$ 
[shipujin@bogon node]$ java --version
openjdk 17.0.5 2022-10-18
OpenJDK Runtime Environment (Red_Hat-17.0.6.0.1-2.fc38) (build 17.0.5+1)
OpenJDK 64-Bit Zero VM (Red_Hat-17.0.6.0.1-2.fc38) (build 17.0.5+1, interpreted mode)
[shipujin@bogon node]$ 
[shipujin@bogon node]$ uname -a
Linux bogon 6.6.0-0.rc4.20231005git3006adf3be79.36.fc38.loongarch64 #1 SMP PREEMPT Sun Oct  8 14:18:09 CST 2023 loongarch64 GNU/Linux
[shipujin@bogon node]$

(1) How are LoongArch machines provided to nodejs?
I can provide a LoongArch machine, the machine runs in my company's machine room. Later I can provide an ssh access to the LoongArch machine ip, is this allowed?

There is still one question about the way of providing LoongArch machine, look forward to node-loong64 running successfully in the official CI process, thanks~

I think access by IP address and SSH is all we need to manage it and connect it to Jenkins.

Then it could be part of a nightly job like https://ci.nodejs.org/view/All/job/sxa-rvnodetest/

I think access by IP address and SSH is all we need to manage it and connect it to Jenkins.

Ok, thank you for your answer.

I will prepare the machine and the user password for accessing the LoongArch machine using ssh in these two days.
Which developer should I send the IP, user and password of the LoongArch machine to? Can I send it to you by email? Then configure "https://ci.nodejs.org/"

Best regards

In terms of configuring access we use ssh keys, it would be better if we could send you a public key that you can added to the machine versus a userid/password.

In terms of configuring access we use ssh keys, it would be better if we could send you a public key that you can added to the machine versus a userid/password.

It is possible to add a public key. I will tell you here after I configure the machine. I can help you put the public key into the nodejs-LoongArch machine.

Best regards

@mhdawson

@nodejs/build @mhdawson @richardlau @sxa @targos @nodejs/platform-loong64
Hi,
I have prepared the LoongArch machine and I am configuring the network.

There are two questions about the way LoongArch machine is provided, hope you can help me, thanks~

How is the ip address of the machine provided?
Is it to send email to send machine ip address
What public keys need to be put into the system? Configure build-loong64-CI
Please send me the download address of the public key, and I will put them into the system for ssh login.

Best regards

@mhdawson

Hello, everyone
I have finished LoongArch machine and network configuration is complete

OS (clfs): Yongbao-2.3-for-nodejs-ci.tar.gz

[shipujin@Sunhaiyong ~]$ 
[shipujin@Sunhaiyong ~]$ uname -a
Linux Sunhaiyong 6.6.0-rc7 #1 SMP PREEMPT Fri Oct 27 01:55:14 UTC 2023 loongarch64 GNU/Linux
[shipujin@Sunhaiyong ~]$ 
[shipujin@Sunhaiyong ~]$ nproc 
32
[shipujin@Sunhaiyong ~]$ free -m
               total        used        free      shared  buff/cache   available
Mem:          130176       24700       98261          23       16378      105476
Swap:              0           0           0
[shipujin@Sunhaiyong ~]$ 
[shipujin@Sunhaiyong ~]$ java --version
openjdk 17.0.8-internal 2023-07-18
OpenJDK Runtime Environment (build 17.0.8-internal+0-adhoc..jdk17-17ugit)
OpenJDK 64-Bit Server VM (build 17.0.8-internal+0-adhoc..jdk17-17ugit, mixed mode)

check nodejs/node 20231101-main

#### main
(1) time make -j32

...

real    17m38.731s                                                  
user    450m48.870s                                                 
sys     16m42.198s

(2) time make test-ci-js 2>&1 | tee ../E.log-nodejs-ci


  ...

Failed tests:
out/Release/node --expose-internals /home/shipujin/node/test/parallel/test-http-correct-hostname.js
make: *** [Makefile:535：test-ci-js] Error 1
                                                                    

real    16m2.169s
user    45m39.270s
sys     9m14.873s

Best regards

Can you guide me on what to do next? Thank you~

@nodejs/build @mhdawson @richardlau @sxa @targos @nodejs/platform-loong64

Then it could be part of a nightly job like https://ci.nodejs.org/view/All/job/sxa-rvnodetest/

Sorry for my a little too many questions, I would like to ask how "sxa-rvnodetest" accesses CI nodes, thank you

Best regards

I would like to ask how "sxa-rvnodetest" accesses CI nodes

Not sure I understand the question - like all jobs the rv-nodetest one specifies a label for machines to run on under "Restrict where this job can run", and the RISC-V machines that are connected to the CI have that label on it.

I would like to ask how "sxa-rvnodetest" accesses CI nodes

Not sure I understand the question - like all jobs the rv-nodetest one specifies a label for machines to run on under "Restrict where this job can run", and the RISC-V machines that are connected to the CI have that label on it.

Thanks for reply

I have a LoongArch machine ready, but I don't know how to connect this LoongArch to CI
Is there a document that describes this procedure, or can you guide me through it? Thank you

Best regards

I would like to ask how many machines are needed to join CI. I understand these so that I can apply for machines with my leader.

To test on every PR we typically need at least 3 machines and they should be fast enough that they complete within ~ 30 mins or so.

I'm very happy to join the build-WG to help nodejs-loong64 become an official support structure and help these machines run.

If possible I suggest you start by joining the Build WG meetings which are in the calendar (nodejs.org/calendar) , and and issue is created in the repo a few days in advance in https://github.com/nodejs/build. The next meeting is Mar 12 at 9ET -> #3646

To test on every PR we typically need at least 3 machines and they should be fast enough that they complete within ~ 30 mins or so.

Okay, thank you for your careful answer. I will apply with my leader to add the machine to CI.

If possible I suggest you start by joining the Build WG meetings which are in the calendar (nodejs.org/calendar) , and and issue is created in the repo a few days in advance in https://github.com/nodejs/build. The next meeting is Mar 12 at 9ET -> #3646

ok, I will submit an issue to apply to join build-WG laterhttps://github.com//issues/3647. I will also pay attention to the working group meetings, but my English speaking is not good, but it does not affect my ability to solve and respond to problems in github/nodejs or email. I will We will continue to promote the growth of nodejs-loong64 and solve the problems that arise.

Best regards

Hi,

Now I have successfully applied for 2 LoongArch servers, which can be provided to Node.js-CI remotely. Could you please provide me the public IP of Node.js-CI? I want to add the public IP of remote access by the Node.js-CI machine to the whitelist.
I will continue to maintain these LoongArch servers and promote the development of node-loong64.

Best regards

@shipujin

@shipujin there are quite a few different IPs in our infra. Do you mean the jenkins server? The agents running on the test nodes reach out to the Jenkins server so I don't think you necessarily need to add it to your allowlist.

In case you do need it all of the IPs are in - https://github.com/nodejs/build/blob/main/ansible/inventory.yml

On the other hand we do want build members to be able to ssh into the machines and we each some from different places

@shipujin

Thank you for your reply.

@shipujin there are quite a few different IPs in our infra. Do you mean the jenkins server? The agents running on the test nodes reach out to the Jenkins server so I don't think you necessarily need to add it to your allowlist.

Sorry for my misunderstanding. If LoongArch test machine connects to jenkins server, jenkins does not need to be whitelisted.

On the other hand we do want build members to be able to ssh into the machines and we each some from different places

Yes, I agree with your suggestion that build members should be able to ssh log in to the new LoongArch test machine, but our room network administrator requires to add whitelist and increase security.
I am thinking later, if an interested build member shows his IP here, I will communicate with our room network administrator to add it to the whitelist.

Sorry, I haven't thought of other better ways to solve this problem. There will be optimized plans in the future, and I will update them in time.

Best regards

@richardlau

I think we'd need something like a jump machine that can be in the whitelist. Build WG members connect to the jump machine and then into the LoongArch machines. What I'm not sure of is how we make that with Ansible. @richardlau any thoughts on that?

@richardlau

I think we'd need something like a jump machine that can be in the whitelist. Build WG members connect to the jump machine and then into the LoongArch machines. What I'm not sure of is how we make that with Ansible. @richardlau any thoughts on that?

Not sure. Rod set up the one we used to use for the Pi's -- I can't remember if the set up for that was in Ansibile.

I have prepared two LoongArch servers. What should I do to add them to Node.js-CI?

Best regards

@richardlau

Can you PR the ips ito the inventory here - https://github.com/nodejs/build/blob/main/ansible/inventory.yml and add the public test key to the services so we can ssh in?

I guess that gets us back to the earlier question of access. How about you get ubuntu1804-x64-1: {ip: 178.128.202.158, alias: gzemnid} added to the firewall so that some build team members would be able to ssh in from that machine.

I assume you have already run ansible on them to configure them, if so once I have access and know their ips from the inventory I can add them to jenkins, add the jenkins tokens to the machines.

@richardlau I don't think that machine is used too much, but wanted to check if you know its going to go away any time soon?

@richardlau

@richardlau I don't think that machine is used too much, but wanted to check if you know its going to go away any time soon?

I'm not aware of anything affecting that server's availability on the horizon.

Can you PR the ips ito the inventory here - https://github.com/nodejs/build/blob/main/ansible/inventory.yml and add the public test key to the services so we can ssh in?

Ok, I will submit the pr, thank you

I guess that gets us back to the earlier question of access. How about you get ubuntu1804-x64-1: {ip: 178.128.202.158, alias: gzemnid} added to the firewall so that some build team members would be able to ssh in from that machine.

Ok, I'll talk to the room administrator and add ubuntu1804-x64-1: {ip: 178.128.202.158, alias: gzemnid} to the whitelist

Best regards

Sorry that the network of the computer room is being adjusted, I will repair the network as soon as possible and promote the network recovery as soon as possible.

Best regards

ansible: add loongson-clfs-loongarch64-{1,2,3} (nodejs#3540)

Navigation Menu

Repository navigation

LoongArch to Experimental tier (CI environment) #3540

OS: LoongArch Fedora remix

latest v18/v20/main

may need to apply this patch

build

check

latest-v18/latest-v20/main

latest-v18

latest-v20

main

OS (clfs): Yongbao-2.3-for-nodejs-ci.tar.gz

check nodejs/node 20231101-main

84 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

LoongArch to Experimental tier (CI environment) #3540

Description

Activity

shipujin commented on Oct 25, 2023

OS: LoongArch Fedora remix

latest v18/v20/main

may need to apply this patch

build

check

latest-v18/latest-v20/main

latest-v18

latest-v20

main

targos commented on Oct 25, 2023

sxa commented on Oct 25, 2023

shipujin commented on Oct 25, 2023

shipujin commented on Oct 26, 2023

targos commented on Oct 26, 2023

targos commented on Oct 26, 2023

shipujin commented on Oct 26, 2023

mhdawson commented on Oct 26, 2023

shipujin commented on Oct 26, 2023

shipujin commented on Oct 31, 2023

shipujin commented on Nov 1, 2023

OS (clfs): Yongbao-2.3-for-nodejs-ci.tar.gz

check nodejs/node 20231101-main

shipujin commented on Nov 2, 2023

sxa commented on Nov 2, 2023

shipujin commented on Nov 2, 2023

84 remaining items

mhdawson commented on Mar 11, 2024

shipujin commented on Mar 12, 2024

shipujin commented on Apr 9, 2024

mhdawson commented on Apr 10, 2024

shipujin commented on Apr 11, 2024

mhdawson commented on Apr 12, 2024

richardlau commented on Apr 12, 2024

shipujin commented on Apr 19, 2024

mhdawson commented on Apr 19, 2024

richardlau commented on Apr 19, 2024

shipujin commented on Apr 20, 2024

shipujin commented on May 9, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions