Skip to content

Extend Upload Android SDK tooling to upload Preview SDKs #193690

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jesswrd wants to merge 1 commit into from
Closed

Extend Upload Android SDK tooling to upload Preview SDKs #193690

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
157 changes: 123 additions & 34 deletions engine/src/flutter/tools/android_sdk/create_cipd_packages.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,24 @@

print_usage () {
echo "Usage:"
echo " ./create_cipd_packages.sh <VERSION_TAG> [PATH_TO_SDK_DIR]"
echo " ./create_cipd_packages.sh [FLAGS] <VERSION_TAG> [PATH_TO_SDK_DIR]"
echo " Downloads, packages, and uploads Android SDK packages where:"
echo " - VERSION_TAG is the tag of the cipd packages, e.g. 28r6 or 31v1. Must contain"
echo " only lowercase letters and numbers."
echo " - VERSION_TAG is the tag of the cipd packages, e.g. 28r6, 31v1 or baklava_v1."
echo " Must contain only lowercase letters, numbers and underscores."
echo " - PATH_TO_SDK_DIR is the path to the sdk folder. If omitted, this defaults to"
echo " your ANDROID_SDK_ROOT environment variable."
echo " ./create_cipd_packages.sh list"
echo " ./create_cipd_packages.sh [FLAGS] list"
echo " Lists the available packages for use in 'packages.txt'"
echo ""
echo "This script downloads the packages specified in packages.txt and uploads"
echo "Flags:"
echo " --packages-file=<path> The file listing the packages to upload. Defaults to"
echo " 'packages.txt' in the same directory as this script."
echo " --dry-run Download and stage all packages, but print the 'cipd create'"
echo " commands instead of running them. The staged upload"
echo " directories are kept for inspection."
echo " -h, --help Print this message."
echo ""
echo "This script downloads the packages specified in the packages file and uploads"
echo "them to CIPD for linux, mac, and windows."
echo "To confirm you have write permissions run 'cipd acl-check flutter/android/sdk/all/ -writer'."
echo ""
Expand All @@ -27,20 +35,64 @@ print_usage () {
echo "For example, build-tools;31.0.0:build-tools"
echo "Multiple directories to upload can be specified by delimiting by additional ':'"
echo ""
echo "To upload an Android preview SDK, use 'preview_packages.txt'. For example:"
echo " ./create_cipd_packages.sh --packages-file=preview_packages.txt --dry-run \\"
echo " cinnamonbun_v1 \$ANDROID_SDK_ROOT"
echo ""
echo "This script expects the cmdline-tools to be installed in your specified PATH_TO_SDK_DIR"
echo "and should only be run on linux or macos hosts."
}

first_argument=$1
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

packages_file="$script_dir/packages.txt"
dry_run=false
positional=()
while [[ $# -gt 0 ]]; do
case "$1" in
--packages-file=*)
packages_file="${1#*=}"
;;
--dry-run)
dry_run=true
;;
-h|--help)
print_usage
exit 0
;;
--*)
echo "Unknown flag: $1"
print_usage
exit 1
;;
*)
positional+=("$1")
;;
esac
shift
done

first_argument="${positional[0]}"
# Validate version or argument is provided.
if [[ $first_argument == "" ]]; then
print_usage
exit 1
fi

# Validate version contains only lower case letters and numbers.
if ! [[ $first_argument =~ ^[[:lower:][:digit:]]+$ ]]; then
echo "Version tag can only consist of lower case letters and digits.";
# Validate version contains only lower case letters, numbers and underscores.
if [[ $first_argument != "list" ]] && ! [[ $first_argument =~ ^[[:lower:][:digit:]_]+$ ]]; then
echo "Version tag can only consist of lower case letters, digits and underscores.";
print_usage
exit 1
fi

# Validate the packages file exists. Relative paths are resolved against the
# current directory first, then against the directory of this script.
if [[ ! -f "$packages_file" && -f "$script_dir/$packages_file" ]]; then
packages_file="$script_dir/$packages_file"
fi
if [[ ! -f "$packages_file" ]]; then
echo "Packages file '$packages_file' not found."
print_usage
exit 1
fi
Comment on lines +89 to 98

@gemini-code-assist gemini-code-assist Bot Oct 1, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The validation of the packages file is executed before checking if the command is list. Since the list command only queries the available packages from sdkmanager and does not use the packages file, requiring it to exist is a regression. We should only validate the packages file if the command is not list.

Suggested change
# Validate the packages file exists. Relative paths are resolved against the
# current directory first, then against the directory of this script.
if [[ ! -f "$packages_file" && -f "$script_dir/$packages_file" ]]; then
packages_file="$script_dir/$packages_file"
fi
if [[ ! -f "$packages_file" ]]; then
echo "Packages file '$packages_file' not found."
print_usage
exit 1
fi
# Validate the packages file exists. Relative paths are resolved against the
# current directory first, then against the directory of this script.
if [[ "$first_argument" != "list" ]]; then
if [[ ! -f "$packages_file" && -f "$script_dir/$packages_file" ]]; then
packages_file="$script_dir/$packages_file"
fi
if [[ ! -f "$packages_file" ]]; then
echo "Packages file '$packages_file' not found."
print_usage
exit 1
fi
fi

Expand All @@ -51,7 +103,7 @@ if [[ `which cipd` == "" ]]; then
exit 1
fi

sdk_path=${2:-$ANDROID_SDK_ROOT}
sdk_path=${positional[1]:-$ANDROID_SDK_ROOT}

# Validate directory contains all SDK packages
if [[ ! -d "$sdk_path" ]]; then
Expand All @@ -68,7 +120,6 @@ if [[ ! -d "$sdk_path/cmdline-tools" ]]; then
fi

platforms=("linux" "macosx" "windows")
package_file_name="packages.txt"

# Find the sdkmanager in cmdline-tools. We default to using latest if available.
sdkmanager_path="$sdk_path/cmdline-tools/latest/bin/sdkmanager"
Expand All @@ -89,10 +140,41 @@ done

# list available packages
if [ $first_argument == "list" ]; then
$sdkmanager_path --list --include_obsolete
"$sdkmanager_path" --list --include_obsolete
exit 0
fi

# Returns the CIPD package name suffixes (e.g. mac-arm64) for a given sdkmanager platform.
cipd_names_for_platform () {
case "$1" in
macosx)
# Upload an arm64 version for M1 macs. Mac uses a different sdkmanager
# name than the platform name used in gn.
echo "mac-amd64 mac-arm64"
;;
*)
echo "$1-amd64"
;;
esac
}

# CIPD tags are immutable. Verify the tag is unused for every package before
# downloading anything, so a collision does not leave a partial upload.
for platform in "${platforms[@]}"; do
for cipd_name in $(cipd_names_for_platform "$platform"); do
if describe_output=$(cipd describe "flutter/android/sdk/all/$cipd_name" -version "version:$first_argument" 2>&1); then
echo "Tag version:$first_argument already exists for flutter/android/sdk/all/$cipd_name."
echo "CIPD tags are immutable. Please choose a new version tag."
exit 1
elif [[ $describe_output != *"no such tag"* ]]; then
echo "Failed to check tag version:$first_argument for flutter/android/sdk/all/$cipd_name:"
echo "$describe_output"
exit 1
fi
done
done
echo "Tag version:$first_argument is unused."
Comment on lines +161 to +176

@gemini-code-assist gemini-code-assist Bot Oct 1, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

When running with --dry-run, the script does not upload any packages to CIPD. However, the cipd describe check is still executed, which requires network access and valid CIPD credentials. If a developer is offline or lacks write/read permissions for these CIPD paths, the dry run will fail. We should skip this tag check when --dry-run is enabled.

Suggested change
# CIPD tags are immutable. Verify the tag is unused for every package before
# downloading anything, so a collision does not leave a partial upload.
for platform in "${platforms[@]}"; do
for cipd_name in $(cipd_names_for_platform "$platform"); do
if describe_output=$(cipd describe "flutter/android/sdk/all/$cipd_name" -version "version:$first_argument" 2>&1); then
echo "Tag version:$first_argument already exists for flutter/android/sdk/all/$cipd_name."
echo "CIPD tags are immutable. Please choose a new version tag."
exit 1
elif [[ $describe_output != *"no such tag"* ]]; then
echo "Failed to check tag version:$first_argument for flutter/android/sdk/all/$cipd_name:"
echo "$describe_output"
exit 1
fi
done
done
echo "Tag version:$first_argument is unused."
# CIPD tags are immutable. Verify the tag is unused for every package before
# downloading anything, so a collision does not leave a partial upload.
if ! $dry_run; then
for platform in "${platforms[@]}"; do
for cipd_name in $(cipd_names_for_platform "$platform"); do
if describe_output=$(cipd describe "flutter/android/sdk/all/$cipd_name" -version "version:$first_argument" 2>&1); then
echo "Tag version:$first_argument already exists for flutter/android/sdk/all/$cipd_name."
echo "CIPD tags are immutable. Please choose a new version tag."
exit 1
elif [[ $describe_output != *"no such tag"* ]]; then
echo "Failed to check tag version:$first_argument for flutter/android/sdk/all/$cipd_name:"
echo "$describe_output"
exit 1
fi
done
done
echo "Tag version:$first_argument is unused."
fi


# We create a new temporary SDK directory because the default working directory
# tends to not update/re-download packages if they are being used. This guarantees
# a clean install of Android SDK.
Expand All @@ -102,19 +184,19 @@ for platform in "${platforms[@]}"; do
sdk_root="$temp_dir/sdk_$platform"
upload_dir="$temp_dir/upload_$platform"
echo "Creating temporary working directory for $platform: $sdk_root"
mkdir $sdk_root
mkdir $upload_dir
mkdir $upload_dir/sdk
mkdir "$sdk_root"
mkdir "$upload_dir"
mkdir "$upload_dir/sdk"
export REPO_OS_OVERRIDE=$platform

# Download all the packages with sdkmanager.
for package in $(< $package_file_name); do
for package in $(< "$packages_file"); do
echo $package
split=(${package//:/ })
IFS=',' read -ra ADDR <<< "${split[0]}"
for i in "${ADDR[@]}"; do
echo "Installing $i"
yes "y" | $sdkmanager_path --sdk_root=$sdk_root "$i"
yes "y" | "$sdkmanager_path" --sdk_root="$sdk_root" "$i"
done

# We copy only the relevant directories to a temporary dir
Expand All @@ -126,30 +208,37 @@ for platform in "${platforms[@]}"; do
done

# Accept all licenses to ensure they are generated and uploaded.
yes "y" | $sdkmanager_path --licenses --sdk_root=$sdk_root
yes "y" | "$sdkmanager_path" --licenses --sdk_root="$sdk_root"
cp -a "$sdk_root/licenses" "$upload_dir/sdk"

archs=("amd64")
if [[ $platform == "macosx" ]]; then
# Upload an arm64 version for M1 macs
archs=("amd64" "arm64")
fi
for arch in "${archs[@]}"; do
# Mac uses a different sdkmanager name than the platform name used in gn.
cipd_name="$platform-$arch"
if [[ $platform == "macosx" ]]; then
cipd_name="mac-$arch"
fi
for cipd_name in $(cipd_names_for_platform "$platform"); do
cipd_command=(cipd create -in "$upload_dir" -name "flutter/android/sdk/all/$cipd_name" -install-mode copy -tag "version:$first_argument" -ref "$first_argument")

echo "Uploading $upload_dir as $cipd_name to CIPD"
cipd create -in $upload_dir -name "flutter/android/sdk/all/$cipd_name" -install-mode copy -tag version:$first_argument -ref $first_argument
if $dry_run; then
echo "[dry-run] ${cipd_command[*]}"
else
echo "Uploading $upload_dir as $cipd_name to CIPD"
"${cipd_command[@]}"
fi
done

rm -rf $sdk_root
rm -rf $upload_dir
rm -rf "$sdk_root"
if ! $dry_run; then
rm -rf "$upload_dir"
fi

# This variable changes the behvaior of sdkmanager.
# Unset to clean up after script.
unset REPO_OS_OVERRIDE
done
rm -rf $temp_dir

if $dry_run; then
echo ""
echo "[dry-run] No packages were uploaded. Staged upload directories:"
for platform in "${platforms[@]}"; do
du -sh "$temp_dir/upload_$platform/sdk"/*
done
echo "[dry-run] Delete $temp_dir when done inspecting."
else
rm -rf "$temp_dir"
fi
4 changes: 4 additions & 0 deletions engine/src/flutter/tools/android_sdk/preview_packages.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
platforms;android-37.2-beta1:platforms
build-tools;37.0.0:build-tools
cmdline-tools;latest:cmdline-tools
platform-tools:platform-tools
Loading