Skip to content

PR check generator: add excludeOsAndVersionCombination #2350

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
angelapwen merged 4 commits into from
Jun 24, 2024
Merged

PR check generator: add excludeOsAndVersionCombination #2350

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
495201e
PR check generator: add `excludeOsAndVersionCombination`
angelapwen Jun 24, 2024
462c756
Switch `debug-artifacts` job back to `ubuntu`
angelapwen Jun 24, 2024
7c35b4f
Rename for clarity per PR review
angelapwen Jun 24, 2024
5d57660
Switch param order for clarity
angelapwen Jun 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
PR check generator: add excludeOsAndVersionCombination
  • Loading branch information
@angelapwen
angelapwen committed Jun 24, 2024
commit 495201e69c9f52491453cc4f3541839e33429dbe
6 changes: 6 additions & 0 deletions .github/workflows/__multi-language-autodetect.yml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions .github/workflows/__scaling-reserved-ram.yml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions .github/workflows/__swift-custom-build.yml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 4 additions & 10 deletions .github/workflows/__unset-environment.yml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 9 additions & 2 deletions pr-checks/checks/multi-language-autodetect.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,14 @@
name: "Multi-language repository"
description: "An end-to-end integration test of a multi-language repository using automatic language detection"
# TODO: Add ubuntu back for `nightly-latest` and `latest` once CLI v2.17.4 is available.
operatingSystems: ["macos"]
operatingSystems: ["macos", "ubuntu"]
excludeOsAndVersionCombination: [
# Known failure for Swift on Linux before CLI v2.17.4.
[ "ubuntu", "stable-20230403" ],
[ "ubuntu", "stable-v2.13.5" ],
[ "ubuntu", "stable-v2.14.6" ],
[ "ubuntu", "stable-v2.15.5" ],
[ "ubuntu", "stable-v2.16.6" ],
]
steps:
- uses: actions/setup-go@v5
with:
Expand Down
11 changes: 9 additions & 2 deletions pr-checks/checks/scaling-reserved-ram.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,14 @@
name: "Scaling reserved RAM"
description: "An end-to-end integration test of a multi-language repository with the scaling_reserved_ram feature flag enabled"
# TODO: Add ubuntu back for `nightly-latest` and `latest` once CLI v2.17.4 is available.
operatingSystems: ["macos"]
operatingSystems: ["macos", "ubuntu"]
excludeOsAndVersionCombination: [
# Known failure for Swift on Linux before CLI v2.17.4.
[ "ubuntu", "stable-20230403" ],
[ "ubuntu", "stable-v2.13.5" ],
[ "ubuntu", "stable-v2.14.6" ],
[ "ubuntu", "stable-v2.15.5" ],
[ "ubuntu", "stable-v2.16.6" ],
]
env:
CODEQL_ACTION_SCALING_RESERVED_RAM: true
steps:
Expand Down
3 changes: 1 addition & 2 deletions pr-checks/checks/swift-custom-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
name: "Swift analysis using a custom build command"
description: "Tests creation of a Swift database using custom build"
versions: ["linked", "default", "nightly-latest"]
# TODO: Add ubuntu back for `nightly-latest` and `latest` once CLI v2.17.4 is available.
operatingSystems: ["macos"]
operatingSystems: ["macos", "ubuntu"]
env:
DOTNET_GENERATE_ASPNET_CERTIFICATE: "false"
steps:
Expand Down
13 changes: 10 additions & 3 deletions pr-checks/checks/unset-environment.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,15 @@
name: "Test unsetting environment variables"
description: "An end-to-end integration test that unsets some environment variables"
# TODO: Switch back to all versions once CLI v2.17.4 is available and running on ubuntu again.
versions: ["stable-v2.14.6", "stable-v2.15.5", "stable-v2.16.6", "linked", "default", "nightly-latest"]
operatingSystems: ["macos"] # TODO: Switch back to ubuntu for `nightly-latest` and `latest` once CLI v2.17.4 is available.
operatingSystems: ["ubuntu"]
excludeOsAndVersionCombination: [
# Known failure for Swift on Linux before CLI v2.17.4.
[ "ubuntu", "stable-20230403" ],
[ "ubuntu", "stable-v2.13.5" ],
[ "ubuntu", "stable-v2.14.6" ],
[ "ubuntu", "stable-v2.15.5" ],
[ "ubuntu", "stable-v2.16.6" ],
]

steps:
- uses: ./../action/init
id: init
Expand Down
49 changes: 30 additions & 19 deletions pr-checks/sync.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,12 @@
"nightly-latest"
]

def is_version_and_os_excluded(version, os, exclude_params):
for exclude_param in exclude_params:
if exclude_param[0] == os and exclude_param[1] == version:
return True
return False

# When updating the ruamel.yaml version here, update the PR check in
# `.github/workflows/pr-checks.yml` too.
header = """# Warning: This file is generated automatically, and should not be modified.
Expand Down Expand Up @@ -56,27 +62,32 @@ def writeHeader(checkStream):
for file in (this_dir / 'checks').glob('*.yml'):
with open(file, 'r') as checkStream:
checkSpecification = yaml.load(checkStream)

matrix = []
excludedVersionsAndOses = checkSpecification.get('excludeOsAndVersionCombination', [])

@henrymercer henrymercer Jun 24, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could rename to excludedOsesAndVersions for clarity since it's OSes first and versions second

for version in checkSpecification.get('versions', defaultTestVersions):
runnerImages = ["ubuntu-latest", "macos-latest", "windows-latest"]
if checkSpecification.get('operatingSystems', None):
runnerImages = [image for image in runnerImages for operatingSystem in checkSpecification['operatingSystems']
if image.startswith(operatingSystem)]

for runnerImage in runnerImages:
# Prior to CLI v2.15.1, ARM runners were not supported by the build tracer.
# "macos-latest" is now an ARM runner, so we run tests on the old CLIs on Intel runners instead.
if version in ["stable-20230403", "stable-v2.13.4", "stable-v2.13.5", "stable-v2.14.6"] and runnerImage == "macos-latest":
matrix.append({
'os': "macos-12",
'version': version
})
else:
matrix.append({
'os': runnerImage,
'version': version
})
runnerImages = ["ubuntu-latest", "macos-latest", "windows-latest"]
operatingSystems = checkSpecification.get('operatingSystems', ["ubuntu", "macos", "windows"])

for operatingSystem in operatingSystems:
runnerImagesForOs = [image for image in runnerImages if image.startswith(operatingSystem)]

for runnerImage in runnerImagesForOs:
# Skip appending this combination to the matrix if it is explicitly excluded.
if is_version_and_os_excluded(version, operatingSystem, excludedVersionsAndOses):

@henrymercer henrymercer Jun 24, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly here consider using the same order of OS and version as in the YAML.

@angelapwen angelapwen Jun 24, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I actually misordered it myself when testing initially 😆

continue

# Prior to CLI v2.15.1, ARM runners were not supported by the build tracer.
# "macos-latest" is now an ARM runner, so we run tests on the old CLIs on Intel runners instead.
if version in ["stable-20230403", "stable-v2.13.4", "stable-v2.13.5", "stable-v2.14.6"] and runnerImage == "macos-latest":
matrix.append({
'os': "macos-12",
'version': version
})
else:
matrix.append({
'os': runnerImage,
'version': version
})

useAllPlatformBundle = "false" # Default to false
if checkSpecification.get('useAllPlatformBundle'):
Expand Down